Semantic-Guided Multi-mask Image Harmonization

Xuqian Ren, Yifan Liu

Tutkimustuotos: KonferenssiartikkeliTieteellinenvertaisarvioitu

6 Sitaatiot (Scopus)
4 Lataukset (Pure)

Abstrakti

Previous harmonization methods focus on adjusting one inharmonious region in an image based on an input mask. They may face problems when dealing with different perturbations on different semantic regions without available input masks. To deal with the problem that one image has been pasted with several foregrounds coming from different images and needs to harmonize them towards different domain directions without any mask as input, we propose a new semantic-guided multi-mask image harmonization task. Different from the previous single-mask image harmonization task, each inharmonious image is perturbed with different methods according to the semantic segmentation masks. Two challenging benchmarks, HScene and HLIP, are constructed based on 150 and 19 semantic classes, respectively. Furthermore, previous baselines focus on regressing the exact value for each pixel of the harmonized images. The generated results are in the ‘black box’ and cannot be edited. In this work, we propose a novel way to edit the inharmonious images by predicting a series of operator masks. The masks indicate the level and the position to apply a certain image editing operation, which could be the brightness, the saturation, and the color in a specific dimension. The operator masks provide more flexibility for users to edit the image further. Extensive experiments verify that the operator mask-based network can further improve those state-of-the-art methods which directly regress RGB images when the perturbations are structural. Experiments have been conducted on our constructed benchmarks to verify that our proposed operator mask-based framework can locate and modify the inharmonious regions in more complex scenes. Our code and models are available at https://github.com/XuqianRen/Semantic-guided-Multi-mask-Image-Harmonization.git.
AlkuperäiskieliEnglanti
OtsikkoComputer Vision – ECCV 2022
Alaotsikko17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXVII
KustantajaSpringer
Sivut564-579
Sivumäärä16
ISBN (elektroninen)978-3-031-19836-6
ISBN (painettu)978-3-031-19835-9
DOI - pysyväislinkit
TilaJulkaistu - 2022
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaEuropean Conference on Computer Vision - Tel Aviv, Israel
Kesto: 23 lokak. 202227 lokak. 2022

Julkaisusarja

NimiLecture Notes in Computer Science
Vuosikerta13697
ISSN (elektroninen)1611-3349

Conference

ConferenceEuropean Conference on Computer Vision
Maa/AlueIsrael
KaupunkiTel Aviv
Ajanjakso23/10/2227/10/22

Julkaisufoorumi-taso

  • Jufo-taso 2

Sormenjälki

Sukella tutkimusaiheisiin 'Semantic-Guided Multi-mask Image Harmonization'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä