{"id":5242,"date":"2019-06-12T20:04:40","date_gmt":"2019-06-12T20:04:40","guid":{"rendered":"https:\/\/labsites.rochester.edu\/gsharma\/?page_id=5242"},"modified":"2019-06-12T20:04:40","modified_gmt":"2019-06-12T20:04:40","slug":"joint-hole-filling-and-depth-upsampling-for-rgb-d-images","status":"publish","type":"page","link":"https:\/\/labsites.rochester.edu\/gsharma\/research\/computer-vision\/joint-hole-filling-and-depth-upsampling-for-rgb-d-images\/","title":{"rendered":"Joint Hole Filling and Depth Upsampling for RGB-D Images"},"content":{"rendered":"

\"\"<\/p>\n

 <\/p>\n

We propose an approach for jointly filling holes and upsampling depth information for RGB-D images, where RGB color information is available at all pixel locations whereas depth information is only available at lower resolution and entirely missing in small regions referred to as \u201choles.\u201d Depth information completion is formulated as a minimization of an objective function composed of two additive terms. The first data fidelity term penalizes disagreement with the observed low-resolution data. The second regularization term penalizes weighted depth deviations from a local linear model in spatial coordinates, where the weights are experimentally determined to ensure consistency between the RGB color image and the estimated depth image. We also propose a memory-efficient implementation of the proposed method based on the conjugate gradient method. Importantly, statistical analysis, which we present in this paper, also reveals that prior evaluations of depth upsampling accuracy are potentially biased because the evaluations inappropriately used preprocessed hole-filled data as \u201cground truth.\u201d<\/p>\n