Facebook Patent | Tileable structured light projection for wide field-of-view depth sensing

Patent: Tileable structured light projection for wide field-of-view depth sensing

Publication Number: 10158845

Publication Date: 2018-12-18

Applicants: Facebook

Abstract

A depth camera assembly (DCA) includes a projector, a detector and a controller. The projector emits a tiled structured light (SL) pattern onto a local area. Each illumination source of the projector includes one or more light emitters and an augmented diffractive optical element (ADOE) designed with a pattern mask. The ADOE diffracts at least a portion of light beams emitted from the light emitters to form a first SL pattern projection having a field-of-view corresponding to a first tileable boundary. The pattern mask prevents projection of light that would otherwise be diffracted outside the first tileable boundary. The first SL pattern projection is combined with at least a second SL pattern projection into the tiled SL pattern illuminating objects in the local area. The detector captures images of the objects illuminated by the SL pattern. The controller determines depth information for the objects using the captured images.

Background

The present disclosure generally relates to design of a structured light pattern, and specifically relates to generating a tileable structured light projection for wide field-of-view (FOV) depth sensing.

Current solutions for achieving structured light illumination with a wide FOV for depth sensing include an approach based on a single wide FOV diffractive optical element (DOE), an approach based on a dual DOE, and an approach based on tiling multiple projectors with projection distortion. However, each of these solutions has its own disadvantages. For example, the approach based on a single wide FOV DOE suffers a relative large zero-order value compared to other diffraction orders. The large zero-order value can lead to issues in laser safety compliance and algorithm performance. Also, the large pincushion like distortion in the light projection causes inefficient use of a projector power. For the approach based on the single DOE for achieving wide FOV, the zero-order value is largely limited by a process error, such as the depth etching error. For example, a depth etching error of 2% leads to a zero-order value of approximately 0.1%. Achieving lower zero-order values than 0.1% based on the single wide FOV DOE means low yield and high price for fabricating a DOE.

The approach based on dual DOE that utilizes a tile DOE and a tiler DOE can further reduce the zero-order value by tiling a light pattern generated by the tile DOE to fill the entire FOV. The zero-order value of the tile DOE can be effectively reduced by a factor equal to a number of tiling projections. For example, 3.times.3 tiling provides effective reduction of the zero-order value by the factor of nine. However, the approach based on dual DOE increases the DOE complexity and cost, while still having an issue with pincushion distortion. The approach based on dual DOE also restricts the space of achievable pattern to the one that repeats across the field which results in additional algorithmic complexity.

The DOE-based structured light projection suffers intrinsic distortion (i.e., the pincushion like distortion) due to the law of diffraction, which increases rapidly as FOV increases. The intrinsic distortion causes overlapping of light patterns and/or voids (i.e., gaps in light patterns) when tiling multiple light projections to achieve wider FOV illumination, which leads to increased complexity of depth sensing algorithms and less efficient use of a projector power. A simple tileable projection shape is a rectangular or square shape. For achieving such a rectangular or square shape of light projection, the rectangular design pattern can be shaped on finer grids (e.g., each grid corresponding to a diffraction order) such that the projected light pattern exhibits desired rectangular shape after illuminating a DOE designed with the re-shaped rectangular light pattern. However, the distortion pre-compensation approach uses more diffraction orders in DOE design and demands more advanced fabrication process, leading to a higher price.

Summary

Embodiments of the present disclosure support a depth camera assembly (DCA). The DCA includes at least one illumination source, a detector and a controller. The at least one illumination source is configured to emit a structured light (SL) pattern onto one or more objects in a local area. The illumination sources of the DCA emit light that are diffracted by one or more augmented diffractive optical elements (ADOEs) to form corresponding SL patterns (also referred to as SL pattern projections). An ADOE is a diffractive optical element that is designed to diffract light into a SL pattern projection that has a field-of-view (FOV) corresponding to a tiling boundary (e.g., rectangular, hexagonal, square, etc.). The ADOE prevents projection of portions of the SL pattern that would otherwise lie outside the tiling boundary. In some embodiments, the ADOE is designed by, e.g., masking a design pattern such that light diffracted from the ADOE is within a FOV that is bounded by the tiling boundary, and then proceeding with a normal lithography process to form a diffractive element. The purpose of masking is to not project portions of the SL pattern that would lie outside a desired tiling boundary.

The ADOE is illuminated with at least a portion of light beams emitted from light emitters of the illumination source. The ADOE generates a SL pattern that is projected into the local area, forming a SL pattern projection. The generated SL pattern projection is combined with at least one other SL pattern projection into a tiled light projection. Two or more ADOEs can be physically arranged so that emitted SL patterns appear in a space as if the SL patterns were coming from the same origin, i.e., angular separations between the generated SL patterns does not depend on a distance between ADOEs. The tiled light projection represents the SL pattern that illuminates the one or more objects in the local area achieving a wide FOV without distortions. The detector of the DCA is configured to capture one or more images of the one or more objects in the local area illuminated by the tiled light projection. The controller of the DCA is configured to determine depth information for the one or more objects in the local area using the one or more images.

Embodiments of the present disclosure further support a head-mounted display (HMD) that integrates the DCA. The HMD further includes an electronic display and an optical assembly. The HMD may be, e.g., a virtual reality (VR) system, an augmented reality (AR) system, a mixed reality (MR) system, or some combination thereof. The electronic display is configured to emit image light. The optical assembly is configured to direct the image light to an exit pupil of the HMD corresponding to a location of a user’s eye, the image light comprising the depth information of the one or more objects in the local area determined by the DCA.

Embodiments of the present disclosure further support a station separate from a HMD, wherein the ADOE is part of an illumination source of a DCA integrated into the station. The station may be a console or some other device interfaced (through a wired or wireless connection) with the HMD. The station can perform three-dimensional depth sensing of an environment surrounding some or all of the station including a user wearing the HMD. The station may provide the depth information to the HMD, e.g., for presentation to the user.