TY - GEN
T1 - Content-Adaptive convolutional neural network post-processing filter
AU - Santamaria, Maria
AU - Lam, Yat Hong
AU - Cricri, Francesco
AU - Lainema, Jani
AU - Youvalari, Ramin G.
AU - Zhang, Honglei
AU - Hannuksela, Miska M.
AU - Rahtu, Esa
AU - Gabbouj, Moncef
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Neural Network (NN)-based coding techniques are being developed for hybrid video coding schemes, such as the Versatile Video Coding (VVC) standard. In-loop filters and postprocessing filters are two types of coding tools that aim to improve the visual quality of the reconstructed content. These tools are usually trained on large video or image datasets with varying content, but they are rarely adaptive to different content types. This problem is addressed with the proposed content-Adaptive Convolutional Neural Network (CNN) post-processing filter. The proposed approach is content-Adaptive in two ways. Firstly, a relatively simple CNN is pre-Trained on a general video dataset and then fine-Tuned on the video to be coded. Since only the bias terms of the CNN are fine-Tuned, the signalling overhead is reduced. Secondly, a scaling factor indicates the influence of the CNN post-processing filter on the final reconstruction. The CNN post-processing filter is evaluated on top of VVC Test Model (VTM) 11.0 with NN-based Video Coding (NNVC) 1.0 and, overall, it can save 2.37% (Y), 3.63% (U), 2.24% (V) Bjontegaard Delta rate (BD-rate) in the Random Access (RA) configuration.
AB - Neural Network (NN)-based coding techniques are being developed for hybrid video coding schemes, such as the Versatile Video Coding (VVC) standard. In-loop filters and postprocessing filters are two types of coding tools that aim to improve the visual quality of the reconstructed content. These tools are usually trained on large video or image datasets with varying content, but they are rarely adaptive to different content types. This problem is addressed with the proposed content-Adaptive Convolutional Neural Network (CNN) post-processing filter. The proposed approach is content-Adaptive in two ways. Firstly, a relatively simple CNN is pre-Trained on a general video dataset and then fine-Tuned on the video to be coded. Since only the bias terms of the CNN are fine-Tuned, the signalling overhead is reduced. Secondly, a scaling factor indicates the influence of the CNN post-processing filter on the final reconstruction. The CNN post-processing filter is evaluated on top of VVC Test Model (VTM) 11.0 with NN-based Video Coding (NNVC) 1.0 and, overall, it can save 2.37% (Y), 3.63% (U), 2.24% (V) Bjontegaard Delta rate (BD-rate) in the Random Access (RA) configuration.
KW - Artefact reduction
KW - content-Adaptation
KW - fine-Tuning
KW - post-processing filter
KW - video coding
U2 - 10.1109/ISM52913.2021.00025
DO - 10.1109/ISM52913.2021.00025
M3 - Conference contribution
AN - SCOPUS:85125019523
T3 - Proceedings - 23rd IEEE International Symposium on Multimedia, ISM 2021
SP - 99
EP - 106
BT - Proceedings - 23rd IEEE International Symposium on Multimedia, ISM 2021
PB - IEEE
T2 - IEEE International Symposium on Multimedia
Y2 - 29 November 2021 through 1 December 2021
ER -