NN-VVC: Versatile Video Coding boosted by self-supervisedly learned image coding for machines

Jukka I. Ahonen, Nam Le, Honglei Zhang, Antti Hallapuro, Francesco Cricri, Hamed Rezazadegan Tavakoli, Miska M. Hannuksela, Esa Rahtu

Tutkimustuotos: KonferenssiartikkeliTieteellinenvertaisarvioitu

1 Sitaatiot (Scopus)
4 Lataukset (Pure)

Abstrakti

The recent progress in artificial intelligence has led to an ever-increasing usage of images and videos by machine analysis algorithms, mainly neural networks. Nonetheless, compression, storage and transmission of media have traditionally been designed considering human beings as the viewers of the content. Recent research on image and video coding for machine analysis has progressed mainly in two almost orthogonal directions. The first is represented by end-to-end (E2E) learned codecs which, while offering high performance on image coding, are not yet on par with state-of-the-art conventional video codecs and lack interoperability. The second direction considers using the Versatile Video Coding (VVC) standard or any other conventional video codec (CVC) together with pre- and post-processing operations targeting machine analysis. While the CVC-based methods benefit from interoperability and broad hardware and software support, the machine task performance is often lower than the desired level, particularly in low bitrates. This paper proposes a hybrid codec for machines called NN-VVC, which combines the advantages of an E2E-learned image codec and a CVC to achieve high performance in both image and video coding for machines. Our experiments show that the proposed system achieved up to -43.20% and -26.8% Bjontegaard Delta rate reduction over VVC for image and video data, respectively, when evaluated on multiple different datasets and machine vision tasks. To the best of our knowledge, this is the first research paper showing a hybrid video codec that outperforms VVC on multiple datasets and multiple machine vision tasks.

AlkuperäiskieliEnglanti
OtsikkoProceedings - 2023 IEEE International Symposium on Multimedia, ISM 2023
KustantajaIEEE
Sivut10-19
Sivumäärä10
ISBN (elektroninen)979-8-3503-9576-1
DOI - pysyväislinkit
TilaJulkaistu - 2023
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaIEEE International Symposium on Multimedia - Laguna Hills, Yhdysvallat
Kesto: 11 jouluk. 202313 jouluk. 2023

Conference

ConferenceIEEE International Symposium on Multimedia
Maa/AlueYhdysvallat
KaupunkiLaguna Hills
Ajanjakso11/12/2313/12/23

Julkaisufoorumi-taso

  • Jufo-taso 1

!!ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Modelling and Simulation

Sormenjälki

Sukella tutkimusaiheisiin 'NN-VVC: Versatile Video Coding boosted by self-supervisedly learned image coding for machines'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä