Densely Sampled Light Field Reconstruction

The emerging light-field and holographic displays aim at providing an immersive visual experience, which in turn requires processing a substantial amount of visual information. In this endeavour, the concept of plenoptic or light-field function plays a very important role as it quantifies the light coming from a visual scene through the multitude of rays going in any direction, at any intensity and at any instant in time. Such a comprehensive function is multi-dimensional and highly redundant at the same time, which raises the problem of its accurate sampling and reconstruction.

In this thesis, we develop a novel method for light field reconstruction from a limited number of multi-perspective images (views).

First, we formalize the light field function in the epipolar image domain in terms of a directional frame representation. We construct a frame (i.e. a dictionary) based on the previously developed shearlet system. The constructed dictionary efficiently represents the structural properties of the continuous light field function. This allows us to formulate the light field reconstruction problem as a variational optimization problem with a sparsity constraint.

Second, we develop an iterative optimization procedure by adapting the variational in-painting method originally developed for 2D image reconstruction. The designed algorithm employs an iterative thresholding and yields an accurate reconstruction using a relatively sparse set of samples in the angular domain.

Finally, we extended the method using various acceleration approaches. More specifically, we improve its robustness by an additional overrelaxation step and make use of the redundancy between different color channels and between epipolar images through colorization and wavelet decomposition techniques.

Extensive experiments have demonstrated that these methods constitute the state of the art for light field reconstruction. The resulting densely-sampled light fields have high visual quality which is beneficial in applications such as holographic stereograms, super-multiview displays, and light field compression.