Three-dimensional profilometric reconstruction using flexible sensing integral imaging and occlusion removal.

We present a method for three-dimensional (3D) profilometric reconstruction using flexible sensing integral imaging with object recognition and automatic occlusion removal. Two-dimensional images, known as elemental images (EIs), of a scene containing an object behind occlusion are captured by flexible sensing integral imaging using a moving camera randomly placed on a non-planar surface with unknown camera position and orientation. After 3D image acquisition, the unknown camera poses are estimated using the EIs and 3D reconstruction is performed based on flexible sensing integral imaging. Object recognition using the 3D reconstructed images is conducted to detect the object behind occlusion and estimate the object depth and position. Occlusion removal is then performed on the 2D EIs for the occluded object by computing variance maps of the scene. For each EI, occluded object pixels with low variance are replaced by object pixels from other perspectives using multi-view geometry. The new set of elemental images may be used to visualize the 3D profile of the scene containing the object without occlusion. Experiments are performed to validate the feasibility of the proposed method. To the best of our knowledge, this is the first report of applying flexible sensing integral imaging to profilometric reconstruction with object recognition and occlusion removal.