Feedforward neural networks initialization based on discriminant learning

Research output: Contribution to journalArticleScientificpeer-review

2 Downloads (Pure)

Abstract

In this paper, a novel data-driven method for weight initialization of Multilayer Perceptrons and Convolutional Neural Networks based on discriminant learning is proposed. The approach relaxes some of the limitations of competing data-driven methods, including unimodality assumptions, limitations on the architectures related to limited maximal dimensionalities of the corresponding projection spaces, as well as limitations related to high computational requirements due to the need of eigendecomposition on high-dimensional data. We also consider assumptions of the method on the data and propose a way to account for them in a form of a new normalization layer. The experiments on three large-scale image datasets show improved accuracy of the trained models compared to competing random-based and data-driven weight initialization methods, as well as better convergence properties in certain cases.

Original languageEnglish
Pages (from-to)220-229
Number of pages10
JournalNeural Networks
Volume146
Early online date25 Nov 2021
DOIs
Publication statusPublished - Feb 2022
Publication typeA1 Journal article-refereed

Keywords

  • Discriminant learning
  • Neural networks initialization

Publication forum classification

  • Publication forum level 2

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Feedforward neural networks initialization based on discriminant learning'. Together they form a unique fingerprint.

Cite this