Description
Light field cameras capture both the intensity and the direction of light rays passing through a scene, enabling capabilities such as post-capture refocusing and view synthesis. However, tasks such as denoising a noisy light field, super-resolving it in the angular or spatial dimension, and refocusing using a stereo pair, have traditionally been treated as separate problems, each solved with its own dedicated algorithm.
We introduce a single, common framework that addresses all of these light field processing tasks using one underlying patch prior. The key observation is that light field patches sharing the same disparity, that is, patches lying at the same depth from the focal plane, live on a low-dimensional subspace whose dimensionality grows quadratically with the disparity value.
Modeling each light field patch as a Gaussian random variable conditioned on its disparity gives rise to a Gaussian Mixture Model over the space of light field patches. This GMM patch prior can then be plugged into a standard MAP estimation pipeline, giving a unified solution for light field denoising, angular and spatial super-resolution, and stereo-camera-based refocusing, all from the same underlying model.