Supporting Posits in Deep Learning Frameworks: A PyTorch Implementation

Afonso Vieira da Silva Luz

Key information

Authors:

Afonso Vieira da Silva Luz (Afonso Vieira da Silva Luz)

Supervisors:

Pedro Filipe Zeferino Tomás (Pedro Filipe Zeferino Tomás); Nuno Filipe Valentim Roma (Nuno Filipe Valentim Roma)

Published in

11/29/2021

Abstract

Reducing the energy consumption of computationally intensive deep learning implementations has received a growing interest in the last years. This is particularly relevant in applications where there are strict energy restrictions, such as space and aerial devices. To this end, the posit number format has shown promising results as a more energy efficient replacement to the standard IEEE-754 floating-point for deep learning computations. Recent research suggests that 16-bit posits achieve similar results as 32-bit floating-point and even smaller posits can be used to train and evaluate deep learning models. However, to study the use of posits for deep learning, researchers have to develop customized functions, since the most popular deep learning frameworks do not yet support posits. This work aims at bridging this gap, by integrating posits natively in PyTorch, the most popular framework for deep learning research. The proposed implementation makes posits a built-in data type in the framework, which means that they can be used in the same way as any other data type that the framework already supports. To validate the implementation, the convolutional neural network LeNet-5 was trained and tested using posits on the MNIST and FashionMNIST datasets. The obtained results with 16-bit posits were similar to those with 32-bit floating-point, suggesting that the implementation of the considered posit operators is correct. To disseminate this contribution, the produced code and documentation was made available on a public GitHub repository.