Robust Hypersphere-based Weight Imprinting for Few-Shot Learning

N. Passalis, A. Iosifidis, M. Gabbouj, A. Tefas

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Performing fast few-shot learning is increasingly important in a number of embedded applications. Among them, a form of gradient-descent free learning known as Weight Imprinting was recently established as an efficient way to perform few-shot learning on Deep Learning (DL) accelerators that do no support back-propagation, such as Edge Tensor Processing Units (Edge TPUs). Despite its efficiency, WI comes with a number of critical limitations. For example, WI cannot effectively handle multimodal novel categories, while it is especially prone to overfitting that can have devastating effects on the accuracy of the models on novel categorizes. To overcome these limitations, in this paper we propose a robust hypersphere-based WI approach that allows for regularizing the training process in an imprinting-aware way. At the same time, the proposed formulation provides a natural way to handle multimodal novel categories. Indeed, as demonstrated through the conducted experiments, the proposed method leads to significant improvements over the baseline WI approach.
Original languageEnglish
Title of host publication2020 28th European Signal Processing Conference (EUSIPCO)
PublisherIEEE
Pages1392-1396
Number of pages5
ISBN (Electronic)978-9-0827-9705-3
DOIs
Publication statusPublished - 2020
Publication typeA4 Article in conference proceedings
EventEuropean Signal Processing Conference -
Duration: 24 Aug 202028 Aug 2020

Publication series

NameEuropean Signal Processing Conference
Volume2021-January
ISSN (Print)2219-5491

Conference

ConferenceEuropean Signal Processing Conference
Period24/08/2028/08/20

Keywords

  • Training
  • Deep learning
  • Tensors
  • Neural networks
  • Stochastic processes
  • Signal processing
  • Task analysis
  • Weight Imprinting
  • Few-shot Learning
  • Edge TPU
  • Embedded Deep Learning

Publication forum classification

  • Publication forum level 1

Fingerprint

Dive into the research topics of 'Robust Hypersphere-based Weight Imprinting for Few-Shot Learning'. Together they form a unique fingerprint.

Cite this