FiveNet: joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement

Mykola Ponomarenko; Vladimir Marchuk; Karen Egiazarian

doi:10.2352/EI.2022.34.14.COIMG-218

FiveNet: joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement

Mykola Ponomarenko, Vladimir Marchuk, Karen Egiazarian

Computing Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

Abstract

In this paper, a convolutional neural network for joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement is proposed. The network inputs are four-channel Bayer CFA image (R, G, G, B) and three channels of the same size containing distortions maps, namely, noise level map, blur level map, and clarity degradation map. It is shown that the designed network FiveNet can effectively process images with the mix of five different distortions. It is also demonstrated that adding clarity enhancement into the processing chain can additionally increase image quality (by up to 3-4 dB in PSNR). A small dataset ClarityDegr120 of color images with different clarity degradations and enhancements is designed using images processed by FiveNet. Mean opinion scores (MOS) for the test set are collected. The MOS prove that clarity enhancement can significantly increase image visual quality. A comparative analysis using the MOS demonstrates a low correspondence between image quality metrics and human perception for the clarity enhancement task.

Original language	English
Title of host publication	Proc. IS&T Int’l. Symp. on Electronic Imaging: Computational Imaging, 2022
Number of pages	6
Volume	34
Edition	14
DOIs	https://doi.org/10.2352/EI.2022.34.14.COIMG-218
Publication status	Published - 2022
Publication type	A4 Article in conference proceedings
Event	IS and T International Symposium on Electronic Imaging: Computational Imaging - Duration: 17 Jan 2022 → 26 Jan 2022

Publication series

Name	IS and T International Symposium on Electronic Imaging Science and Technology
ISSN (Print)	2470-1173

Conference

Conference	IS and T International Symposium on Electronic Imaging: Computational Imaging
Period	17/01/22 → 26/01/22

Funding

Vladimir Marchuk would like to acknowledge the financial support of the Russian Federation represented by the Ministry of Science and Higher Education of the Russian Federation (Agreement No. 075-15-2021-997 of 09/28/2021), and Mykola Ponomarenko - the financial support of Huawei-Tampere University project 3114100158, FlexISP.

Publication forum classification

Publication forum level 1

ASJC Scopus subject areas

Computer Graphics and Computer-Aided Design
Computer Science Applications
Human-Computer Interaction
Software
Electrical and Electronic Engineering
Atomic and Molecular Physics, and Optics

Access to Document

10.2352/EI.2022.34.14.COIMG-218

Cite this

Ponomarenko, M., Marchuk, V., & Egiazarian, K. (2022). FiveNet: joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement. In Proc. IS&T Int’l. Symp. on Electronic Imaging: Computational Imaging, 2022 (14 ed., Vol. 34). (IS and T International Symposium on Electronic Imaging Science and Technology). https://doi.org/10.2352/EI.2022.34.14.COIMG-218

@inproceedings{aeaec92045d64ab0a16d76d7236a2bbc,

title = "FiveNet: joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement",

abstract = "In this paper, a convolutional neural network for joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement is proposed. The network inputs are four-channel Bayer CFA image (R, G, G, B) and three channels of the same size containing distortions maps, namely, noise level map, blur level map, and clarity degradation map. It is shown that the designed network FiveNet can effectively process images with the mix of five different distortions. It is also demonstrated that adding clarity enhancement into the processing chain can additionally increase image quality (by up to 3-4 dB in PSNR). A small dataset ClarityDegr120 of color images with different clarity degradations and enhancements is designed using images processed by FiveNet. Mean opinion scores (MOS) for the test set are collected. The MOS prove that clarity enhancement can significantly increase image visual quality. A comparative analysis using the MOS demonstrates a low correspondence between image quality metrics and human perception for the clarity enhancement task.",

author = "Mykola Ponomarenko and Vladimir Marchuk and Karen Egiazarian",

note = "Funding Information: Vladimir Marchuk would like to acknowledge the financial support of the Russian Federation represented by the Ministry of Science and Higher Education of the Russian Federation (Agreement No. 075-15-2021-997 of 09/28/2021), and Mykola Ponomarenko - the financial support of Huawei-Tampere University project 3114100158, FlexISP. Publisher Copyright: {\textcopyright} 2022, Society for Imaging Science and Technology. jufoid=84313; IS and T International Symposium on Electronic Imaging: Computational Imaging ; Conference date: 17-01-2022 Through 26-01-2022",

year = "2022",

doi = "10.2352/EI.2022.34.14.COIMG-218",

language = "English",

volume = "34",

series = "IS and T International Symposium on Electronic Imaging Science and Technology",

booktitle = "Proc. IS&T Int{\textquoteright}l. Symp. on Electronic Imaging: Computational Imaging, 2022",

edition = "14",

}

Ponomarenko, M, Marchuk, V & Egiazarian, K 2022, FiveNet: joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement. in Proc. IS&T Int’l. Symp. on Electronic Imaging: Computational Imaging, 2022. 14 edn, vol. 34, IS and T International Symposium on Electronic Imaging Science and Technology, IS and T International Symposium on Electronic Imaging: Computational Imaging, 17/01/22. https://doi.org/10.2352/EI.2022.34.14.COIMG-218

FiveNet: joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement. / Ponomarenko, Mykola; Marchuk, Vladimir; Egiazarian, Karen.
Proc. IS&T Int’l. Symp. on Electronic Imaging: Computational Imaging, 2022. Vol. 34 14. ed. 2022. (IS and T International Symposium on Electronic Imaging Science and Technology).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - FiveNet

T2 - IS and T International Symposium on Electronic Imaging: Computational Imaging

AU - Ponomarenko, Mykola

AU - Marchuk, Vladimir

AU - Egiazarian, Karen

N1 - Funding Information: Vladimir Marchuk would like to acknowledge the financial support of the Russian Federation represented by the Ministry of Science and Higher Education of the Russian Federation (Agreement No. 075-15-2021-997 of 09/28/2021), and Mykola Ponomarenko - the financial support of Huawei-Tampere University project 3114100158, FlexISP. Publisher Copyright: © 2022, Society for Imaging Science and Technology. jufoid=84313

PY - 2022

Y1 - 2022

N2 - In this paper, a convolutional neural network for joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement is proposed. The network inputs are four-channel Bayer CFA image (R, G, G, B) and three channels of the same size containing distortions maps, namely, noise level map, blur level map, and clarity degradation map. It is shown that the designed network FiveNet can effectively process images with the mix of five different distortions. It is also demonstrated that adding clarity enhancement into the processing chain can additionally increase image quality (by up to 3-4 dB in PSNR). A small dataset ClarityDegr120 of color images with different clarity degradations and enhancements is designed using images processed by FiveNet. Mean opinion scores (MOS) for the test set are collected. The MOS prove that clarity enhancement can significantly increase image visual quality. A comparative analysis using the MOS demonstrates a low correspondence between image quality metrics and human perception for the clarity enhancement task.

AB - In this paper, a convolutional neural network for joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement is proposed. The network inputs are four-channel Bayer CFA image (R, G, G, B) and three channels of the same size containing distortions maps, namely, noise level map, blur level map, and clarity degradation map. It is shown that the designed network FiveNet can effectively process images with the mix of five different distortions. It is also demonstrated that adding clarity enhancement into the processing chain can additionally increase image quality (by up to 3-4 dB in PSNR). A small dataset ClarityDegr120 of color images with different clarity degradations and enhancements is designed using images processed by FiveNet. Mean opinion scores (MOS) for the test set are collected. The MOS prove that clarity enhancement can significantly increase image visual quality. A comparative analysis using the MOS demonstrates a low correspondence between image quality metrics and human perception for the clarity enhancement task.

U2 - 10.2352/EI.2022.34.14.COIMG-218

DO - 10.2352/EI.2022.34.14.COIMG-218

M3 - Conference contribution

AN - SCOPUS:85132397327

VL - 34

T3 - IS and T International Symposium on Electronic Imaging Science and Technology

BT - Proc. IS&T Int’l. Symp. on Electronic Imaging: Computational Imaging, 2022

Y2 - 17 January 2022 through 26 January 2022

ER -

FiveNet: joint image demosaicing, denoising, deblurring, super-resolution and clarity enhancement

Abstract

Publication series

Conference

Funding

Publication forum classification

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this