Google Patent | System and Method for Active Stereo Depth Sensing

Patent: System and Method for Active Stereo Depth Sensing

Publication Number: 20180350087

Publication Date: 2018-12-06

Applicants: Google

Abstract

An electronic device estimates a depth map of an environment based on stereo depth images captured by depth cameras having exposure times that are offset from each other in conjunction with illuminators pulsing illumination patterns into the environment. A processor of the electronic device matches small sections of the depth images from the cameras to each other and to corresponding patches of immediately preceding depth images (e.g., a spatio-temporal image patch “cube”). The processor computes a matching cost for each spatio-temporal image patch cube by converting each spatio-temporal image patch into binary codes and defining a cost function between two stereo image patches as the difference between the binary codes. The processor minimizes the matching cost to generate a disparity map, and optimizes the disparity map by rejecting outliers using a decision tree with learned pixel offsets and refining subpixels to generate a depth map of the environment.

Background

Depth cameras are used as inputs for computer vision tasks such as hand, body, or object tracking, 3D reconstruction, and simultaneous localization and mapping (SLAM). For such tasks, each new frame of depth and image data is correlated to a previous frame, allowing for pose or geometric reconstruction over time. However, for depth cameras operating at relatively low speed (i.e., capturing a low number of frames per second), high frame-to-frame movements in a scene and artifacts such as motion blur make correlations between frames difficult to solve.