My research interests are in Computational Imaging, Image Processing and Computer Vision. Traditionally, camera optics design and imaging inference (image processing, computer vision algorithms)have been done more or less independently. However, in Computational Imaging ,the camera optics and the processing/inference algorithms are co-designed so that we can capture more information about
the visual scene and we can significantly enhance the performance of imaging systems.
Our research focuses on the following aspects of Computational Imaging (CI):
- Develop novel CI systems
- Develop theory to explore the performance limits of CI systems
- Design deep learning algorithms for solving inverse problems in CI, image processing and computer vision
In the following, I highlight the research achievements of my lab in these areas:
Light Field Reconstruction from Compressive and Sparse Measurements:
A classical example of a computational imaging system is a Light Field (LF) camera. A conventional
camera only captures 2D spatial resolution, whereas, a LF camera captures both 2D spatial and 2D
angular resolutions. There are various advantages in capturing the LF of a scene. It is predicted that LF displays are going to revolutionize VR headsets in the next few years. For
generating LF contents for such displays, it is essential to make LF capture easy and affordable. Current
LF cameras require specialized hardware and hence it is not easily accessible. Thus, an active area of research is to reconstruct LF from sparse views such as those captured using a stereo camera or even
from a monocular camera. This will enable LF reconstruction from smartphone captures. Recently, we
have looked at the challenging problem of reconstructing LF videos from stereo and monocular videos.
Some important publications from this line of work are as follows:
Lensless Imaging: Reconstruction, Inference and Design:
Emerging applications such as wearables, augmented reality, virtual reality, biometrics, and many
others are driving an acute need for highly miniaturized imaging systems. Unfortunately, current-
generation cameras are based on lenses – and these lenses typically account for more than 90 percent
of the cost, volume and weight of cameras. Over the last decade, lensless imaging systems have emerged
as a potential solution for light-weight, ultra-compact, inexpensive imaging. The basic idea in lensless
imaging is to replace the lens with an amplitude or a phase mask; typically placed quite close to the
sensor.Because of the absence of the focusing lens, the lensless camera captures a highly multiplexed
measurement of the scene and the main challenge is to reconstruct a sharp image from it. There are
traditional methods of recovering the image such as Wiener deconvolution, however, the recovered
images are not sharp enough. We have proposed a two-stage network with the first stage being an
inversion network that produces an intermediate image followed by a refinement network to produce
the final image. We have further proposed a learning based algorithm for finding optimal phase
masks for reconstruction as well as for solving inference tasks such as face detection and optical flow
estimation. We have also extended our approach to estimate depth and intensity from a single lensless
measurement.
The following publications have resulted from this line of research work:
Real-time Restoration of Extremely Dark (Single and Multiview) Images:
Images captured in extreme low-light conditions such as night-time suffer from significant sensor noise
and poor color capture. Restoring such low-light images will have immense applications in surveillance
and advanced driver assistance systems. In recent years, researchers have tried to tackle this problem
using deep learning architectures. Existing methods, however, only target restoration quality and
compromise on speed and memory requirements, raising concerns about their real-world deployability.
We have proposed light-weight and fast models with low memory utilization. We have further proposed
light-weight and fast models for restoration of stereo and light field images while preserving their
epipolar geometry.
The following publications have resulted from this line of research work:
