Hypersphere-Based Weight Imprinting for Few-Shot Learning on Embedded Devices

Nikolaos Passalis, Alexandros Iosifidis, Moncef Gabbouj, Anastasios Tefas

Research output: Contribution to journalArticleScientificpeer-review

16 Citations (Scopus)

Abstract

Weight imprinting (WI) was recently introduced as a way to perform gradient descent-free few-shot learning. Due to this, WI was almost immediately adapted for performing few-shot learning on embedded neural network accelerators that do not support back-propagation, e.g., edge tensor processing units. However, WI suffers from many limitations, e.g., it cannot handle novel categories with multimodal distributions and special care should be given to avoid overfitting the learned embeddings on the training classes since this can have a devastating effect on classification accuracy (for the novel categories). In this article, we propose a novel hypersphere-based WI approach that is capable of training neural networks in a regularized, imprinting-aware way effectively overcoming the aforementioned limitations. The effectiveness of the proposed method is demonstrated using extensive experiments on three image data sets.
Original languageEnglish
Pages (from-to)925-930
Number of pages6
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume32
Issue number2
Early online date2020
DOIs
Publication statusPublished - 2021
Publication typeA1 Journal article-refereed

Keywords

  • Prototypes
  • Training
  • Neural networks
  • Data visualization
  • Learning systems
  • Tensile stress
  • Image edge detection
  • Edge tensor processing unit (TPU)
  • embedded deep learning (DL)
  • few-shot learning
  • weight imprinting (WI).

Publication forum classification

  • Publication forum level 3

Fingerprint

Dive into the research topics of 'Hypersphere-Based Weight Imprinting for Few-Shot Learning on Embedded Devices'. Together they form a unique fingerprint.

Cite this