WLD-Reg: A Data-Dependent Within-Layer Diversity Regularizer

Tutkimustuotos: KonferenssiartikkeliScientificvertaisarvioitu

Abstrakti

Neural networks are composed of multiple layers arranged in a hierarchical structure jointly trained with a gradient-based optimization, where the errors are back-propagated from the last layer back to the first one. At each optimization step, neurons at a given layer receive feedback from neurons belonging to higher layers of the hierarchy. In this paper, we propose to complement this traditional’between-layer’ feedback with additional’within-layer’ feedback to encourage the diversity of the activations within the same layer. To this end, we measure the pairwise similarity between the outputs of the neurons and use it to model the layer’s overall diversity. We present an extensive empirical study confirming that the proposed approach enhances the performance of several state-of-the-art neural network models in multiple tasks. The code is publically available at https://github.com/firasl/AAAI-23WLD-Reg.

AlkuperäiskieliEnglanti
OtsikkoAAAI-23 Technical Tracks 7
ToimittajatBrian Williams, Yiling Chen, Jennifer Neville
Sivut8421-8429
Sivumäärä9
ISBN (elektroninen)9781577358800
DOI - pysyväislinkit
TilaJulkaistu - 27 kesäk. 2023
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaAAAI Conference on Artificial Intelligence - Washington, Yhdysvallat
Kesto: 7 helmik. 202314 helmik. 2023

Julkaisusarja

NimiProceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023
Numero7
Vuosikerta37
ISSN (elektroninen)2374-3468

Conference

ConferenceAAAI Conference on Artificial Intelligence
Maa/AlueYhdysvallat
KaupunkiWashington
Ajanjakso7/02/2314/02/23

Julkaisufoorumi-taso

  • Jufo-taso 2

!!ASJC Scopus subject areas

  • Artificial Intelligence

Sormenjälki

Sukella tutkimusaiheisiin 'WLD-Reg: A Data-Dependent Within-Layer Diversity Regularizer'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä