A Framework for Depth Video Reconstruction From a Subset of Samples and Its Applications

作者：Liu Lee Kang^*; Nguyen Truong Q

来源：IEEE Transactions on Image Processing, 2016, 25(10): 4873-4887.

DOI：10.1109/TIP.2016.2598484

• Sparse reconstruction
• spatio-temporal volume reconstruction
• spatio-temporal sampling
• dense depth estimation
• alternating direction method of multipliers

The growth of depth sensing systems in this decade has facilitated a variety of applications in computer vision. Depending on the systematic configurations, both direct and indirect sensing techniques encounter image processing issues, such as hole filling and depth map super resolution. In this paper, a framework for depth video reconstruction from a subset of samples is proposed. By redefining classical dense depth estimation into two individual problems, sensing and synthesis, we propose a motion compensation-assisted sampling (MCAS) scheme and a spatio-temporal depth reconstruction (STDR) algorithm for reconstructing depth video sequences from a subset of samples. Using the 3-dimensional extensible dictionary, discrete wavelet transform (DWT), and applying alternating direction method of multiplier technique, the proposed STDR algorithm possesses scalability for temporal volume and efficiency for processing large scale depth data. Exploiting the temporal information and corresponding RGB images, the proposed MCAS achieves an efficient one-stage sampling scheme. Experimental results show that the proposed depth reconstruction framework outperforms the existing methods and is competitive compared with our previous work, which requires a pilot signal in the two-stage sampling scheme. Finally, to estimate missing reliable depth samples from varying input sources, we present an inference approach using geometrical and color similarities. Applications for depth video super resolution from uniform-grid subsampled data and dense disparity video estimation from a subset of reliable samples are presented.

• 出版日期2016-10