Microsoft Patent | Reducing Blur in a Depth Camera System

Patent: Reducing Blur in a Depth Camera System

Publication Number: 20180343432

Publication Date: 2018-11-29

Applicants: Microsoft Technology Licensing, LLC

Abstract

A technique is described herein for reducing blur caused by an imaging assembly of a depth camera system. In a runtime phase, the technique involves receiving a sensor image that is generated in response to return radiation reflected from a scene. The return radiation passes through an optical element (such as a visor element) of the imaging assembly, which produces blur due to the scattering of radiation. The technique then deconvolves the sensor image with a kernel, to provide a blur-reduced image. The kernel represents a point spread function that describes the distortion-related characteristics of at least the optical element. The technique then uses the blur-reduced image to calculate a depth image. The technique also encompasses a calibration-phase process for generating the kernel by modeling blur that occurs near an edge of a test object within a test image.

BACKGROUND 

A time-of-flight (ToF) depth camera system includes an illumination source and a sensor operating in coordination with each other. The illumination source projects infrared radiation onto a scene. The sensor receives resultant infrared radiation that is reflected from the scene, and, in response thereto, provides a plurality of sensor signals. The signals provide information which relates to an amount of time it takes the radiation to travel from the illumination source to the sensor, for a plurality of points in the scene. A processing component converts the sensor signals into depth values, each of which describes the distance between a point in the scene and a reference point. The depth values collectively correspond to a depth image. A post-processing component may thereafter leverage the depth image to perform some context-specific task, such as providing a mixed-reality experience in a head-mounted display (HMD), controlling the navigation of a vehicle, producing a three-dimensional reconstruction of the scene, etc.

A ToF depth camera system is highly susceptible to noise that originates from various sources. The noise can cause the depth camera system to generate inaccurate depth values, which, in turn, may degrade the performance of any post-processing component that relies on the depth values. This makes a depth camera system different from a conventional video camera, in which noise only causes an aesthetic degradation of an image.

SUMMARY

A technique is described herein for reducing blur caused by an imaging assembly of a depth camera system. More specifically, in one implementation, the technique reduces blur principally caused by the light-scattering behavior of an optical element (OE) of a time-of-flight depth camera system. In one non-limiting example, the optical element corresponds to a transparent visor element of a head-mounted display (HMD), through which radiation passes to and from the HMD's depth camera system.

In a runtime phase, the technique generates a sensor image in response to return radiation that is reflected from an object in a scene. The return radiation is scattered as it passes through the optical element, which causes blur in the sensor image. The technique then deconvolves the sensor image with a kernel, to provide a blur-reduced image. The kernel represents a point spread function (PSF) that describes the distortion-related characteristics of at least the optical element. The technique then uses the blur-reduced image (together with other blur-reduced images) to calculate a depth image.

In a calibration phase, the technique generates the PSF based on a line spread function. The technique generates the line spread function, in turn, by modeling blur that occurs near an edge of a test object within a test image. That blur is principally caused by radiation scattered by the optical element.

The above technique can be manifested in various types of systems, devices, components, methods, computer-readable storage media, data structures, articles of manufacture, and so on.

This Summary is provided to introduce a selection of concepts in a simplified form; these concepts are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.