Content-Adaptive convolutional neural network post-processing filter

Maria Santamaria, Yat Hong Lam, Francesco Cricri, Jani Lainema, Ramin G. Youvalari, Honglei Zhang, Miska M. Hannuksela, Esa Rahtu, Moncef Gabbouj

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Neural Network (NN)-based coding techniques are being developed for hybrid video coding schemes, such as the Versatile Video Coding (VVC) standard. In-loop filters and postprocessing filters are two types of coding tools that aim to improve the visual quality of the reconstructed content. These tools are usually trained on large video or image datasets with varying content, but they are rarely adaptive to different content types. This problem is addressed with the proposed content-Adaptive Convolutional Neural Network (CNN) post-processing filter. The proposed approach is content-Adaptive in two ways. Firstly, a relatively simple CNN is pre-Trained on a general video dataset and then fine-Tuned on the video to be coded. Since only the bias terms of the CNN are fine-Tuned, the signalling overhead is reduced. Secondly, a scaling factor indicates the influence of the CNN post-processing filter on the final reconstruction. The CNN post-processing filter is evaluated on top of VVC Test Model (VTM) 11.0 with NN-based Video Coding (NNVC) 1.0 and, overall, it can save 2.37% (Y), 3.63% (U), 2.24% (V) Bjontegaard Delta rate (BD-rate) in the Random Access (RA) configuration.

Original languageEnglish
Title of host publicationProceedings - 23rd IEEE International Symposium on Multimedia, ISM 2021
PublisherIEEE
Pages99-106
Number of pages8
ISBN (Electronic)9781665437349
DOIs
Publication statusPublished - 2021
Publication typeA4 Article in a conference publication
EventIEEE International Symposium on Multimedia: ISM - Virtual, Online, Italy
Duration: 29 Nov 20211 Dec 2021

Publication series

NameProceedings - 23rd IEEE International Symposium on Multimedia, ISM 2021

Conference

ConferenceIEEE International Symposium on Multimedia
Country/TerritoryItaly
CityVirtual, Online
Period29/11/211/12/21

Keywords

  • Artefact reduction
  • content-Adaptation
  • fine-Tuning
  • post-processing filter
  • video coding

Publication forum classification

  • Publication forum level 1

ASJC Scopus subject areas

  • Computer Science Applications
  • Human-Computer Interaction
  • Media Technology
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Content-Adaptive convolutional neural network post-processing filter'. Together they form a unique fingerprint.

Cite this