A curated dataset of urban scenes for audio-visual scene analysis

Shanshan Wang; Annamaria Mesaros; Toni Heittola; Tuomas Virtanen

doi:10.48550/arXiv.2011.00030

A curated dataset of urban scenes for audio-visual scene analysis

Shanshan Wang, Annamaria Mesaros, Toni Heittola, Tuomas Virtanen

Tietotekniikka

Tutkimustuotos: Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

31 Sitaatiot (Scopus)

22 Lataukset (Pure)

Abstrakti

This paper introduces a curated dataset of urban scenes for audio-visual scene analysis which consists of carefully selected and recorded material. The data was recorded in multiple European cities, using the same equipment, in multiple locations for each scene, and is openly available. We also present a case study for audio-visual scene recognition and show that joint modeling of audio and visual modalities brings significant performance gain compared to state of the art uni-modal systems. Our approach obtained an 84.8% accuracy compared to 75.8% for the audio-only and 68.4% for the video-only equivalent systems.

Alkuperäiskieli	Englanti
Otsikko	ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Kustantaja	IEEE
Sivut	626-630
Sivumäärä	5
ISBN (elektroninen)	978-1-7281-7605-5
DOI - pysyväislinkit	https://doi.org/10.48550/arXiv.2011.00030 https://doi.org/10.1109/ICASSP39728.2021.9415085
Tila	Julkaistu - 2021
OKM-julkaisutyyppi	A4 Artikkeli konferenssijulkaisussa
Tapahtuma	IEEE International Conference on Acoustics, Speech and Signal Processing - Metro Toronto Convention Centre, Toronto, Kanada Kesto: 6 kesäk. 2021 → 11 kesäk. 2021 https://2021.ieeeicassp.org

Julkaisusarja

Nimi	Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing
ISSN (painettu)	1520-6149

Conference

Conference	IEEE International Conference on Acoustics, Speech and Signal Processing
Maa/Alue	Kanada
Kaupunki	Toronto
Ajanjakso	6/06/21 → 11/06/21
www-osoite	https://2021.ieeeicassp.org

Julkaisufoorumi-taso

Jufo-taso 1

!!ASJC Scopus subject areas

Software
Signal Processing
Electrical and Electronic Engineering

Pääsy asiakirjaan

wang_ICASSP2021
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Accepted author manuscript, 127 KBLisenssi: Unspecified

https://urn.fi/URN:NBN:fi:tuni-202203032286

Siteeraa tätä

Wang, S., Mesaros, A., Heittola, T., & Virtanen, T. (2021). A curated dataset of urban scenes for audio-visual scene analysis. teoksessa ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (Sivut 626-630). (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing). IEEE. https://doi.org/10.48550/arXiv.2011.00030, https://doi.org/10.1109/ICASSP39728.2021.9415085

@inproceedings{80ab5a44f68545b09081153cf22f27a1,

title = "A curated dataset of urban scenes for audio-visual scene analysis",

abstract = "This paper introduces a curated dataset of urban scenes for audio-visual scene analysis which consists of carefully selected and recorded material. The data was recorded in multiple European cities, using the same equipment, in multiple locations for each scene, and is openly available. We also present a case study for audio-visual scene recognition and show that joint modeling of audio and visual modalities brings significant performance gain compared to state of the art uni-modal systems. Our approach obtained an 84.8% accuracy compared to 75.8% for the audio-only and 68.4% for the video-only equivalent systems.",

keywords = "Acoustic scene, Audio-visual data, Pattern recognition, Scene analysis, Transfer learning",

author = "Shanshan Wang and Annamaria Mesaros and Toni Heittola and Tuomas Virtanen",

note = "Funding Information: This work was supported in part by the European Research Council under the European Unions H2020 Framework Programme through ERC Grant Agreement 637422 EVERYSOUND. The authors wish to thank CSC-IT Centre of Science Ltd., Finland, for providing computational resources. Funding Information: This work was supported in part by the European Research Council under the European Unions H2020 Framework Programme through ERC Grant Agreement 637422 EVERYSOUND. The authors wish to thank CSCIT Centre of Science Ltd., Finland, for providing computational resources. Publisher Copyright: {\textcopyright} 2021 IEEE. jufoid=57409; IEEE International Conference on Acoustics, Speech and Signal Processing ; Conference date: 06-06-2021 Through 11-06-2021",

year = "2021",

doi = "10.48550/arXiv.2011.00030",

language = "English",

series = "Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing",

publisher = "IEEE",

pages = "626--630",

booktitle = "ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

address = "United States",

url = "https://2021.ieeeicassp.org",

}

Wang, S , Mesaros, A , Heittola, T & Virtanen, T 2021, A curated dataset of urban scenes for audio-visual scene analysis. julkaisussa ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE, Sivut 626-630, IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, Ontario, Kanada, 6/06/21. https://doi.org/10.48550/arXiv.2011.00030, https://doi.org/10.1109/ICASSP39728.2021.9415085

A curated dataset of urban scenes for audio-visual scene analysis. / Wang, Shanshan ; Mesaros, Annamaria ; Heittola, Toni et al.
ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021. s. 626-630 (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing).

Tutkimustuotos: Konferenssiartikkeli › Tieteellinen › vertaisarvioitu

TY - GEN

T1 - A curated dataset of urban scenes for audio-visual scene analysis

AU - Wang, Shanshan

AU - Mesaros, Annamaria

AU - Heittola, Toni

AU - Virtanen, Tuomas

N1 - Funding Information: This work was supported in part by the European Research Council under the European Unions H2020 Framework Programme through ERC Grant Agreement 637422 EVERYSOUND. The authors wish to thank CSC-IT Centre of Science Ltd., Finland, for providing computational resources. Funding Information: This work was supported in part by the European Research Council under the European Unions H2020 Framework Programme through ERC Grant Agreement 637422 EVERYSOUND. The authors wish to thank CSCIT Centre of Science Ltd., Finland, for providing computational resources. Publisher Copyright: © 2021 IEEE. jufoid=57409

PY - 2021

Y1 - 2021

N2 - This paper introduces a curated dataset of urban scenes for audio-visual scene analysis which consists of carefully selected and recorded material. The data was recorded in multiple European cities, using the same equipment, in multiple locations for each scene, and is openly available. We also present a case study for audio-visual scene recognition and show that joint modeling of audio and visual modalities brings significant performance gain compared to state of the art uni-modal systems. Our approach obtained an 84.8% accuracy compared to 75.8% for the audio-only and 68.4% for the video-only equivalent systems.

AB - This paper introduces a curated dataset of urban scenes for audio-visual scene analysis which consists of carefully selected and recorded material. The data was recorded in multiple European cities, using the same equipment, in multiple locations for each scene, and is openly available. We also present a case study for audio-visual scene recognition and show that joint modeling of audio and visual modalities brings significant performance gain compared to state of the art uni-modal systems. Our approach obtained an 84.8% accuracy compared to 75.8% for the audio-only and 68.4% for the video-only equivalent systems.

KW - Acoustic scene

KW - Audio-visual data

KW - Pattern recognition

KW - Scene analysis

KW - Transfer learning

U2 - 10.48550/arXiv.2011.00030

DO - 10.48550/arXiv.2011.00030

M3 - Conference contribution

AN - SCOPUS:85108360575

T3 - Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing

SP - 626

EP - 630

BT - ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - IEEE

T2 - IEEE International Conference on Acoustics, Speech and Signal Processing

Y2 - 6 June 2021 through 11 June 2021

ER -

Wang S , Mesaros A , Heittola T , Virtanen T. A curated dataset of urban scenes for audio-visual scene analysis. julkaisussa ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2021. s. 626-630. (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing). doi: 10.48550/arXiv.2011.00030, 10.1109/ICASSP39728.2021.9415085

A curated dataset of urban scenes for audio-visual scene analysis

Abstrakti

Julkaisusarja

Conference

Julkaisufoorumi-taso

!!ASJC Scopus subject areas

Pääsy asiakirjaan

Sormenjälki

Siteeraa tätä