Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions

Shanshan Wang; Toni Heittola; Annamaria Mesaros; Tuomas Virtanen

doi:10.5281/zenodo.5770113

Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions

Shanshan Wang, Toni Heittola, Annamaria Mesaros, Tuomas Virtanen

Tietotekniikka

Tutkimustuotos: Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

21 Lataukset (Pure)

Abstrakti

This paper presents the details of the Audio-Visual Scene Classification task in the DCASE 2021 Challenge (Task 1 Subtask B). The task is concerned with classification using audio and video modalities, using a dataset of synchronized recordings. This task has attracted 43 submissions from 13 different teams around the world. Among all submissions, more than half of the submitted systems have better performance than the baseline. The common techniques among the top systems are the usage of large pretrained models such as ResNet or EfficientNet which are trained for the task-specific problem. Fine-tuning, transfer learning, and data augmentation techniques are also employed to boost the performance. More importantly, multi-modal methods using both audio and video are employed by all the top 5 teams. The best system among all achieved a logloss of 0.195 and accuracy of 93.8\%, compared to the baseline system with logloss of 0.662 and accuracy of 77.1%.

Alkuperäiskieli	Englanti
Otsikko	Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE2021)
Toimittajat	Frederic Font, Annamaria Mesaros, Daniel P.W. Ellis, Eduardo Fonseca, Magdalena Fuentes, Benjamin Elizalde
Kustantaja	DCASE
Sivut	45-49
ISBN (elektroninen)	978-84-09-36072-7
DOI - pysyväislinkit	https://doi.org/10.5281/zenodo.5770113
Tila	Julkaistu - 15 marrask. 2021
OKM-julkaisutyyppi	A4 Artikkeli konferenssijulkaisussa
Tapahtuma	Detection and Classication of Acoustic Scenes and Events - , Espanja Kesto: 15 marrask. 2021 → 19 marrask. 2021

Conference

Conference	Detection and Classication of Acoustic Scenes and Events
Maa/Alue	Espanja
Ajanjakso	15/11/21 → 19/11/21

Julkaisufoorumi-taso

Jufo-taso 0

Pääsy asiakirjaan

10.5281/zenodo.5770113Lisenssi: CC BY

DCASE2021Workshop_Wang_20-1Final published version, 647 KBLisenssi: CC BY

Siteeraa tätä

Wang, S., Heittola, T., Mesaros, A., & Virtanen, T. (2021). Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions. teoksessa F. Font, A. Mesaros, D. P.W. Ellis, E. Fonseca, M. Fuentes, & B. Elizalde (Toimittajat), Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE2021) (Sivut 45-49). DCASE. https://doi.org/10.5281/zenodo.5770113

Wang, Shanshan ; Heittola, Toni ; Mesaros, Annamaria et al. / Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions. Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE2021). Toimittaja / Frederic Font ; Annamaria Mesaros ; Daniel P.W. Ellis ; Eduardo Fonseca ; Magdalena Fuentes ; Benjamin Elizalde. DCASE, 2021. Sivut 45-49

@inproceedings{99f523461df840ecb5bdbc8deb769dce,

title = "Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions",

abstract = "This paper presents the details of the Audio-Visual Scene Classification task in the DCASE 2021 Challenge (Task 1 Subtask B). The task is concerned with classification using audio and video modalities, using a dataset of synchronized recordings. This task has attracted 43 submissions from 13 different teams around the world. Among all submissions, more than half of the submitted systems have better performance than the baseline. The common techniques among the top systems are the usage of large pretrained models such as ResNet or EfficientNet which are trained for the task-specific problem. Fine-tuning, transfer learning, and data augmentation techniques are also employed to boost the performance. More importantly, multi-modal methods using both audio and video are employed by all the top 5 teams. The best system among all achieved a logloss of 0.195 and accuracy of 93.8\%, compared to the baseline system with logloss of 0.662 and accuracy of 77.1%.",

author = "Shanshan Wang and Toni Heittola and Annamaria Mesaros and Tuomas Virtanen",

year = "2021",

month = nov,

day = "15",

doi = "10.5281/zenodo.5770113",

language = "English",

pages = "45--49",

editor = "Frederic Font and Annamaria Mesaros and {P.W. Ellis}, Daniel and Eduardo Fonseca and Magdalena Fuentes and Benjamin Elizalde",

booktitle = "Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE2021)",

publisher = "DCASE",

note = "Detection and Classication of Acoustic Scenes and Events ; Conference date: 15-11-2021 Through 19-11-2021",

}

Wang, S , Heittola, T , Mesaros, A & Virtanen, T 2021, Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions. julkaisussa F Font, A Mesaros, D P.W. Ellis, E Fonseca, M Fuentes & B Elizalde (toim), Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE2021). DCASE, Sivut 45-49, Detection and Classication of Acoustic Scenes and Events, Espanja, 15/11/21. https://doi.org/10.5281/zenodo.5770113

Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions. / Wang, Shanshan ; Heittola, Toni ; Mesaros, Annamaria et al.
Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE2021). toim. / Frederic Font; Annamaria Mesaros; Daniel P.W. Ellis; Eduardo Fonseca; Magdalena Fuentes; Benjamin Elizalde. DCASE, 2021. s. 45-49.

Tutkimustuotos: Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

TY - GEN

T1 - Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions

AU - Wang, Shanshan

AU - Heittola, Toni

AU - Mesaros, Annamaria

AU - Virtanen, Tuomas

PY - 2021/11/15

Y1 - 2021/11/15

N2 - This paper presents the details of the Audio-Visual Scene Classification task in the DCASE 2021 Challenge (Task 1 Subtask B). The task is concerned with classification using audio and video modalities, using a dataset of synchronized recordings. This task has attracted 43 submissions from 13 different teams around the world. Among all submissions, more than half of the submitted systems have better performance than the baseline. The common techniques among the top systems are the usage of large pretrained models such as ResNet or EfficientNet which are trained for the task-specific problem. Fine-tuning, transfer learning, and data augmentation techniques are also employed to boost the performance. More importantly, multi-modal methods using both audio and video are employed by all the top 5 teams. The best system among all achieved a logloss of 0.195 and accuracy of 93.8\%, compared to the baseline system with logloss of 0.662 and accuracy of 77.1%.

AB - This paper presents the details of the Audio-Visual Scene Classification task in the DCASE 2021 Challenge (Task 1 Subtask B). The task is concerned with classification using audio and video modalities, using a dataset of synchronized recordings. This task has attracted 43 submissions from 13 different teams around the world. Among all submissions, more than half of the submitted systems have better performance than the baseline. The common techniques among the top systems are the usage of large pretrained models such as ResNet or EfficientNet which are trained for the task-specific problem. Fine-tuning, transfer learning, and data augmentation techniques are also employed to boost the performance. More importantly, multi-modal methods using both audio and video are employed by all the top 5 teams. The best system among all achieved a logloss of 0.195 and accuracy of 93.8\%, compared to the baseline system with logloss of 0.662 and accuracy of 77.1%.

U2 - 10.5281/zenodo.5770113

DO - 10.5281/zenodo.5770113

M3 - Conference contribution

SP - 45

EP - 49

BT - Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE2021)

A2 - Font, Frederic

A2 - Mesaros, Annamaria

A2 - P.W. Ellis, Daniel

A2 - Fonseca, Eduardo

A2 - Fuentes, Magdalena

A2 - Elizalde, Benjamin

PB - DCASE

T2 - Detection and Classication of Acoustic Scenes and Events

Y2 - 15 November 2021 through 19 November 2021

ER -

Wang S , Heittola T , Mesaros A , Virtanen T. Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions. julkaisussa Font F, Mesaros A, P.W. Ellis D, Fonseca E, Fuentes M, Elizalde B, toimittajat, Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE2021). DCASE. 2021. s. 45-49 doi: 10.5281/zenodo.5770113

Audio-visual scene classification: analysis of DCASE 2021 Challenge submissions

Abstrakti

Conference

Julkaisufoorumi-taso

Pääsy asiakirjaan

Sormenjälki

Siteeraa tätä