3D reconstruction from coded plenoptic sampling
Date
2019
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
The plenoptic function describes a scene in terms of light rays, it is a 7-dimensional function with spectral, directional, spatial, and temporal variation. Traditional plenoptic sampling is acquired either by employing a standard plenoptic camera or a camera array, and
the spatial-angular sampling can be potentially used to model 3D surface. ☐ In this dissertation, I present three coded plenoptic sampling schemes, i.e., the rotational cross-slit (R-XSlit) plenoptic sampling, the wavelength coded plenoptic sampling, and the polarimetric plenoptic sampling. The additional coded sampling information, such
as non-centric sampling, spectral sampling, and polarization sampling, are conducive to 3D reconstruction. Therefore, I also develop the corresponding 3D reconstruction framework for each of them. ☐ First, I introduce the R-XSlit plenoptic sampling scheme by exploiting a special noncentric camera called the crossed-slit or XSlit camera. An XSlit camera acquires rays that simultaneously pass through two oblique slits. I show that instead of translating the camera
as in the pinhole case, we can effectively sample the 4D plenoptic sampling by rotating individual or both slits while keeping the camera fixed, which makes the plenoptic sampling coded in the spatial-angular domain. The theoretical analysis shows that it provides denser spatial-angular sampling, which is beneficial for scene reconstruction and rendering. I develop a volumetric reconstruction scheme for scene reconstruction. ☐ Second, I present two wavelength coded plenoptic sampling schemes in the visible and infrared spectrum respectively. I firstly design a compact system with lights and cameras arranged on concentric circles to acquire a concentric wavelength coded plenoptic sampling
in the visible spectrum, the cameras on each ring capture images in a unique spectrum. I employ the Phong dichromatic model onto its plenoptic function for 3D reconstruction and spectral reflectance map estimation. Experiments show that our technique can achieve high accuracy and robustness in geometry recovery. Moreover, I present an infrared wavelength coded plenoptic sampling and develop a hybrid sensing framework to efficiently achieve pose estimation and face reconstruction by exploiting the captured reflected infrared rays
from human eyes. ☐ Finally, I present a polarimetric plenoptic sampling framework for recovering 3D surfaces, the polarization of light is included in its plenoptic function. I employ a new analysis analogous to the optical flow to correlate the polarization radiance function with both surface normal and depth. The proposed framework effectively resolves the azimuth-zenith ambiguity by forming an over-determined system. Extensive experiments on both synthetic and real data demonstrate that the technique is capable of recovering extremely challenging glossy and textureless objects.