Unsupervised Facial Image De-occlusion with Optimized Deep Generative Models

Lei Xu, Honglei Zhang, Jenni Raitoharju, Moncef Gabbouj

    Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

    1 Citation (Scopus)

    Abstract

    In recent years, Generative Adversarial Networks (GANs) or various types of Auto-Encoders (AEs) have gained attention on facial image de-occlusion and/or in-painting tasks. In this paper, we propose a novel unsupervised technique to remove occlusion from facial images and complete the occluded parts simultaneously with optimized Deep Convolutional Generative Adversarial Networks (DCGANs) in an iterative way. Generally, GANs, as generative models, can estimate the distribution of images using a generator and a discriminator. DCGANs, as its variant, are proposed to conquer its instability during training. Existing facial image in-painting methods manually define a block of pixels as the missing part and the potential content of this block is semantically generated using generative models, such as GANs or AEs. In our method, a mask is inferred from an occluded facial image using a novel loss function, and then this mask is utilized to in-paint the occlusions automatically by pre-trained DCGANs. We evaluate the performance of our method on facial images with various occlusions, such as sunglasses and scarves. The experiments demonstrate that our method can effectively detect certain kinds of occlusions and complete the occluded parts in an unsupervised manner.
    Original languageEnglish
    Title of host publication2018 Eighth International Conference on Image Processing Theory, Tools and Applications (IPTA)
    PublisherIEEE
    ISBN (Electronic)978-1-5386-6428-5
    ISBN (Print)978-1-5386-6429-2
    DOIs
    Publication statusPublished - Nov 2018
    Publication typeA4 Article in conference proceedings
    EventInternational Conference on Image Processing Theory, Tools and Applications -
    Duration: 1 Jan 1900 → …

    Conference

    ConferenceInternational Conference on Image Processing Theory, Tools and Applications
    Period1/01/00 → …

    Publication forum classification

    • Publication forum level 1

    Fingerprint

    Dive into the research topics of 'Unsupervised Facial Image De-occlusion with Optimized Deep Generative Models'. Together they form a unique fingerprint.

    Cite this