Self-Supervised Learning of Audio Representations using Angular Contrastive Loss

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

In Self-Supervised Learning (SSL), various pretext tasks are designed for learning feature representations through contrastive loss. However, previous studies have shown that this loss is less tolerant to semantically similar samples due to the inherent defect of instance discrimination objectives, which may harm the quality of learned feature embeddings used in downstream tasks. To improve the discriminative ability of feature embeddings in SSL, we propose a new loss function called Angular Contrastive Loss (ACL), a linear combination of angular margin and contrastive loss. ACL improves contrastive learning by explicitly adding an angular margin between positive and negative augmented pairs in SSL. Experimental results show that using ACL for both supervised and unsupervised learning significantly improves performance. We validated our new loss function using the FSDnoisy18k dataset, where we achieved 73.6% and 77.1% accuracy in sound event classification using supervised and self-supervised learning, respectively.

Original languageEnglish
Title of host publicationICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, Proceedings
PublisherIEEE
ISBN (Electronic)978-1-7281-6327-7
DOIs
Publication statusPublished - 2023
Publication typeA4 Article in conference proceedings
EventIEEE International Conference on Acoustics, Speech, and Signal Processing - Rhodes Island, Greece
Duration: 4 Jun 202310 Jun 2023

Publication series

NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume2023-June
ISSN (Print)1520-6149

Conference

ConferenceIEEE International Conference on Acoustics, Speech, and Signal Processing
Country/TerritoryGreece
CityRhodes Island
Period4/06/2310/06/23

Keywords

  • angular margin loss
  • audio representation learning
  • contrastive loss
  • self-supervised learning

Publication forum classification

  • Publication forum level 2

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Self-Supervised Learning of Audio Representations using Angular Contrastive Loss'. Together they form a unique fingerprint.

Cite this