Rosenblatt Perceptrons with Random Hidden Layer

10/30/2024

Often, when Rosenblatt’s perceptron is discussed, a perceptron with a single layer is considered, which can only solve linearly separable problems. Models such as deep neural networks have been proposed to overcome this limitation, but their greater complexity in terms of trainable parameters makes them prone to overfitting, which harms their ability to generalize and causes them to use more energy for training. However, Rosenblatt mainly investigated perceptrons with hidden layers, which allow them to solve non-linear problems. We have performed experiments testing the accuracy of the LImited Receptive Area (LIRA) classifier, a model for image classification based on Rosenblatt’s perceptron with randomly-chosen and fixed connections to the hidden layer, on a variety of image datasets, in both grayscale and color. We have also experimented with modifications to the LIRA model, such as a change to the process by which the connections to the hidden layer are chosen, support for classification of color images, and an added step of pruning and regenerating hidden neurons between each training iteration. We compared the model’s accuracy to that of the single-layer perceptron and logistic regression classifier, using cross-validation, and found that it was highest for the LIRA classifier, with an accuracy of 86.83% on the Fashion-MNIST dataset and 93.40% on the Kuzushiji-MNISTT dataset.

Authors in the community:

• 

  L

  Luís Miguel Almeida de Vasconcelos Amorim

  ist196890

Supervisors of this institution:

• 

  A

  Andreas Miroslaus Wichert

  ist24622

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

eng - English

Instituto Superior Técnico