Microsoft Patent | Color Management Of Display Device

Patent: Color Management Of Display Device

Publication Number: 10650785

Publication Date: 20200512

Applicants: Microsoft

Abstract

MicroLED arrays offer a small form factor solution for the HMD image sources since they do not need a separate illumination optics. Features of the present disclosure implement a MicroLED display system that incorporate a plurality of monochrome projectors (e.g., three MicroLED projectors) to generate three monochrome images (e.g., red, blue, and green images) that are separately input into a single waveguide of the HMD and combined to form an image that is displayed to the user. By utilizing a single waveguide that includes a plurality of spatially separated input regions (e.g., a region for inputting blue light, a region for inputting red light, a region for inputting green light), the MicroLED display system of the present disclosure may reduce the form factor of the HMD device because of the reduced number of plates that may be required to combine the three monochrome images.

BACKGROUND

The present disclosure relates to computer graphics systems, and more particularly, to a Micro light emitting diode (MicroLED) display system and the color management of the display device.

One area of computing devices that has grown in recent years is the area of virtual reality (VR) and augmented reality (AR) devices, which use a graphics processing unit (GPU) to render graphics from a computing device to a display device. Such technology may be incorporated into a head-mounted display (HMD) device in the form of eyeglasses, goggles, a helmet, a visor, or other eyewear. As used herein, a HMD device may include a device that generates and/or displays virtual reality images (e.g., from at least one virtual environment input), and/or mixed reality (MR) images or augmented reality (AR) images (e.g., from at least one virtual environment input and one real environment input). In such devices, a scene produced on a display device can be oriented or modified based on user input (e.g., movement of a gamepad button or stick to cause movement of the orientation of the scene, introduction of items into the scene, etc.).

One challenge with incorporating display devices into HMD or mobile devices is the size constraints that limit some of the optical or display components that can be integrated into the HMD devices while miniaturizing the overall size of the HMD devices to improve user mobility. Current HMDs usually use illuminated micro displays such as reflective liquid crystal on silicon (hereafter “LCoS”) or digital light processing (DLP) projectors as they provide a high standard of display performance. These displays offer advantages such as high resolution, a wide color gamut, high brightness and a high contrast ratio. However, such digital projection systems that rely on LCoS or DLP technology require large form factors to create a uniform illumination of panels. Thus, there is a need in the art for improvements in presenting images on a display with miniaturized components without compromising the display quality or user experience.

SUMMARY

MicroLED arrays offer a small form factor solution for the HMD image sources since they do not need separate illumination optics. Features of the present disclosure implement a MicroLED display system that incorporates a plurality of monochrome projectors (e.g., three MicroLED projectors) to generate three monochrome images (e.g., red, blue, and green images) that are separately input into a single waveguide of the HMD and combined to form an image that is displayed to the user. By utilizing a single waveguide that includes a plurality of spatially separated input regions (e.g., a region for inputting blue light, a region for inputting red light, and a region for inputting green light), the MicroLED display system of the present disclosure may reduce the form factor of the HMD device because of the reduced number and/or size of optical components, such as a reduced number of plates that may be required to combine the three monochrome images.

In one example, a display device is disclosed. The display device may include an optical waveguide having a plurality of input regions for receiving at least a portion of light corresponding to an image, wherein the plurality of input regions include at least a first input region, a second input region, and a third input region. The display device may also include a plurality of monolithic micro light emitting diode (MicroLED) projectors that each generate a different monochrome color version of the image. In some examples, the plurality of MicroLED projectors may include at least a first projector generating a first version of the image in a first color, a second projector generating a second version of the image in a second color, and a third projector generating a third version of the image in a third color. In some aspects, the first input region of the optical waveguide may be physically aligned with the first projector to receive light corresponding to the first version of the image, the second input region of the optical waveguide may be physically aligned with the second projector to receive light corresponding to the second version of the image, and the third input region of the optical waveguide may be physically aligned with the third projector to receive light corresponding to the third version of the image. In some examples, the optical waveguide may further include an output region configured to guide light from each of the plurality of MicroLED projectors toward a target to make a final image visible to user, wherein the final image is a fully colored image that combines the different monochrome color versions of the image.

In another example, a method for displaying an image on a display device is disclosed. The method may include generating a plurality of monochrome color versions of an image using a plurality of monolithic MicroLED projectors, wherein the plurality of MicroLED projectors includes at least a first projector generating a first version of the image in a first color, a second projector generating a second version of the image in a second color, and a third projector generating a third version of the image in a third color. The method may further include receiving, at an optical waveguide, light inputs from each of the plurality of MicroLED projectors into a plurality of different input regions of the optical waveguide such that a first input region of the optical waveguide is physically aligned with a first color projector to receive light corresponding to the first version of the image, the second input region of the optical waveguide is physically aligned with the second color projector to receive light corresponding to the second version of the image, and the third input region of the optical waveguide is physically aligned with the third color projector to receive light corresponding to the third version of the image. The method may further include outputting, through the optical waveguide, light from each of the plurality of MicroLED projectors toward a target to make a final image visible to user, wherein the final image is a fully colored image that combines the plurality of monochrome color versions of an image.

In another example, another method for calibrating an image on a display device is disclosed. The method may include detecting, at a camera, light rays output from an optical waveguide, wherein the light rays correspond to a plurality of monochrome color versions of an image generated by a plurality of monolithic MicroLED projectors. The method may further include determining a first position of a first version of the image from the plurality of monochrome color versions of the image, wherein the first version of the image is generated in first monochrome color by a first projector. The method may further include determining a second position of a second version of the image from the plurality of monochrome color versions of the image, wherein the second version of the image is generated in second monochrome color by a second projector. The method may further include measuring an image displacement between the first position of the first version of the image and the second position of the second version of the image to calculate an offset value between the first position and the second position, wherein the image displacement is one or both of lateral displacement or angular rotational displacement. The method may further include configuring the display device to adjust at least one image output parameter of the plurality of MicroLED projectors such that the output from the optical waveguide of the plurality of monochrome color versions of the image have an overlapping alignment that meets an alignment threshold.

In another example, an apparatus for calibrating an image on a display device is disclosed. The apparatus may include a memory to store data and instructions. The apparatus may further include a processor in communication with the memory to execute the instructions to detect, at a camera, light rays output from an optical waveguide, wherein the light rays correspond to a plurality of monochrome color versions of an image generated by a plurality of monolithic MicroLED projectors. The apparatus may further include instructions to determine a first position of a first version of the image from the plurality of monochrome color versions of the image, wherein the first version of the image is generated in first monochrome color by a first projector. The apparatus may further include instructions to determine a second position of a second version of the image from the plurality of monochrome color versions of the image, wherein the second version of the image is generated in second monochrome color by a second projector. The apparatus may further include instructions to measure an image displacement between the first position of the first version of the image and the second position of the second version of the image to calculate an offset value between the first position and the second position, wherein the image displacement is one or both of lateral displacement or angular rotational displacement. The apparatus may further include instructions to configure the display device to adjust at least one image output parameter of the plurality of MicroLED projectors such that the output from the optical waveguide of the plurality of monochrome color versions of the image have an overlapping alignment that meets an alignment threshold.

The above presents a simplified summary of one or more implementations of the present disclosure in order to provide a basic understanding of such implementations. This summary is not an extensive overview of all contemplated implementations, and is intended to neither identify key or critical elements of all implementations nor delineate the scope of any or all implementations. Its sole purpose is to present some concepts of one or more implementations of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

To the accomplishment of the foregoing and related ends, the one or more aspects comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative features of the one or more aspects. These features are indicative, however, of but a few of the various ways in which the principles of various aspects may be employed, and this description is intended to include all such aspects and their equivalents.

DESCRIPTION OF THE FIGURES

The disclosed aspects of the present disclosure will hereinafter be described in conjunction with the appended drawings, provided to illustrate and not to limit the disclosed aspects, wherein like designations denote like elements, where a dashed line may indicate an optional component, and in which:

FIG. 1A is a schematic diagram of a display device (e.g., HMD device) in accordance with an implementation of the present disclosure;

FIGS. 1B and 1C are a schematic diagram of a side view of the display device, and more particularly the waveguide aligned with a plurality of monochrome MicroLED projectors in accordance with an implementation of the present disclosure;

FIG. 2 is an example of light projection from a MicroLED projector to an optical waveguide in accordance with an implementation of the present disclosure;

FIG. 3 is an example MicroLED display system that incorporate a plurality of monochrome MicroLED projectors to generate three monochrome images that are separately input into a single waveguide of the HMD in accordance with an implementation of the present disclosure;

FIG. 4 is another example MicroLED display system that incorporate a plurality of monochrome MicroLED projectors to generate three monochrome images that are separately input into a single waveguide of the HMD while allowing at least two of the MicroLED projectors to share the same grating structure in accordance with an implementation of the present disclosure;

FIG. 5 is a flow chart of a method for displaying images on a display device in accordance with an implementation of the present disclosure;

FIG. 6 is a flow chart of a method for calibrating images on a display device in accordance with an implementation of the present disclosure;

FIG. 7 is a schematic block diagram of an example device in accordance with an implementation of the present disclosure.

DETAILED DESCRIPTION

Various aspects are now described with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It may be evident, however, that such aspect(s) may be practiced without these specific details. Additionally, the term “component” as used herein may be one of the parts that make up a system, may be hardware, firmware, and/or software stored on a computer-readable medium, and may be divided into other components.

The present disclosure provides devices and methods for presentation of images such as virtual reality (VR) or mixed reality (MR)/augmented reality (AR) images on a display that is incorporated into mobile display devices, such as displays implemented for head mounted display (HMD). It should be appreciated by those of ordinary skill in the art that while the present disclosure references HMD, the display techniques implemented herein may be adaptable for any mobile device, including but not limited to, mobile phones, tablets, or laptops.

As discussed above, one challenge with incorporating display devices into mobile devices is the size constraints that limit the components that can be integrated into the display systems while miniaturizing the overall size of the HMD devices or mobile display to improve user mobility. Current HMDs generally use illuminated micro displays such as reflective LCoS or DLP projectors as they provide a high standard of display performance. While these displays offer advantages such as high resolution, a wide color gamut, high brightness and a high contrast ratio, such digital projection systems that rely on LCoS or DLP technology also require large form factors to create a uniform illumination of panels.

In contrast, MicroLED arrays offer a small form factor solution for the HMD image sources since they do not need separate illumination optics. However, the materials required to create semiconductors with either blue-green or red LEDs are generally incompatible in the manufacturing process of LEDs for specific colors (e.g., Indium, Gallium and Nitrogen (InGaN) to produce green, blue and white colors and/or Aluminum Gallium Arsenide (AlGaAs) used to generate red and amber portions of the visible spectrum). Specifically, from the manufacturing perspective, it may be difficult to get all three colors into the same array without either the labor intensive process of picking and placing each LED individually or developing an expensive growth/lithography process.

One solution to the above-identified problem with MicroLED micro displays may be to separate each of the red, green, and blue arrays and combine the different colors with optics. However, optical combiners require large prism cubes to combine three different images together and relay optics to prevent the divergence of the pixels when combining the colors. Thus, such optical systems increase the optics footprint and prevent small form factor HMD development with micro LEDs. Furthermore, the divergence angle of the microLEDs may be so large (E.g., +/-40 degrees) that it may impact the optical power.

Features of the present disclosure implement a MicroLED display system that incorporates a plurality of monochrome projectors (e.g., three MicroLED projectors) to generate three monochrome images (e.g., red, blue, and green images) that are separately input into a single waveguide of the HMD and combined to form an image that is displayed to the user. By utilizing a single waveguide that includes a plurality of spatially separated input regions (e.g., a region for inputting blue light, a region for inputting red light, a region for inputting green light), the MicroLED display system of the present disclosure may reduce the size and form factor of the HMD device because of the reduced number of plates that may be required to combine the three monochrome images and/or because of the reduced number of size of the optical components. However, it should be appreciated by those of ordinary skill in the art that the features of the present disclosure are not limited to plurality of projectors generating monochrome images. Instead, in some examples, a single projector without external illumination may also generate and input the full color image into the waveguide. Such system would provide benefit of simplistic hardware and reduced form factor in comparison to the current digital projection systems that rely on LCoS or DLP technology that require large form factors to create a uniform illumination of panels.

Specifically, in accordance with aspects of the present disclosure, waveguides with gratings may provide a mechanism to combine several images together with the smallest possible form factor. With a separate microLED display for each primary color, the exit pupil of each image/color may be coupled into a single optical waveguide that brings the pupils on top of each other using pupil replication. Further, since the waveguide may be designed to act as a pupil expander of an augmented reality (AR) display, the same optical waveguide may be used to relay the image into the user’s eye, thereby saving on additional hardware by using the same integrated waveguide to combine the three colors together and to act as an AR-display. This may include configuring the waveguide to steer the incoming field-of-view (FoV) at least into two or more directions in order to enable pupil expansion with a large FoV.

Additional advantages of the present disclosure may be the hardware simplicity that is required for HMD. For example, by incorporating a plurality of monochrome projectors (e.g., three MicroLED projectors) to generate three monochrome images (e.g., red, blue, and green images) that are separately input into a single waveguide of the HMD, the optics of each MicroLED projectors may be simpler because no mirrors or doublet lenses may be needed to avoid or correct chromatic aberration.

In addition, in some implementations, the present disclosure may further reduce the form factor of the HMD by implementing a color management scheme that may include modifying the color resolution of each MicroLED projector. Specifically, the physical size of the MicroLED projector may depend on the resolution/pixel count of the imaging system. Recognizing that the human eye has the highest sensitivity for green light, the overall form factor of the HMD may be further reduced by lowering the color resolution of blue and/or red colors, while increasing the color resolution of green light for full resolution. To this end, due to lower color resolution requirements of one or more colors (blue and/or red color images), features of the present disclosure may further reduce the size and complexity of hardware of the monolithic MicroLED projectors. Thus, the lower color resolution of the blue and/or red colors may be compensated by a higher resolution of green color to provide a user with a full resolution image while benefiting from reduced form factor of the overall display system. Although the above example discusses lowering the color resolution of blue and/or red color images, it should be appreciated that the modification of the color resolution is not limited to only blue and/or red colors, but instead may be adapted for any number of colors. Additionally, in some examples, the techniques of the present disclosure may be applied to lowering the color resolution of only one color.

Finally, features of the present disclosure also provide techniques of calibrating the image to be displayed via the optical guide by configuring one or more MicroLED projectors. For example, any mechanical mounting issues with the MicroLED projectors may change the angle of the projected image, which in turn may cause pixels from that MicroLED projector to be in misplaced. Thus, any displacement between the three monochrome images may result in a blurry image being displayed on the display device. One technique to solve this problem may include manipulating the incoming image in order to pre-correct the location of one or more pixels in order to account for any disparity. Thus, in some aspects of the manufacturing process, a calibration procedure may be implemented that may utilize a camera to be placed in front of the optical waveguide (e.g., where a user’s eye would otherwise be) to identify how each color field is separated from one another, and to provide configuration parameters to calibrate the image from the three MicroLED projectors such that the three images overlap, generating a cohesive final image to be displayed. To this end, features of the present disclosure may include techniques for pre-rotating and offsetting at least one of the three monochromic images to correct for any misalignment in the mechanical assembly of the display device, including the MicroLED projectors.

The following description provides examples, and is not limiting of the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as appropriate. For instance, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Also, features described with respect to some examples may be combined in other examples.

Turning first to FIGS. 1A-1C, an example display device 100 may implement display techniques in accordance with an present disclosure. In some examples, as illustrated, the display device 100 may be a head mounted device (HMD). However, for purposes of this disclosure, it should be appreciated that the techniques described herein are not limited only to HMD, but may also be implemented in other display devices, including, but not limited to, mobile phones, laptops, televisions, etc. For purposes of this disclosure, features of FIGS. 1A-1C may be discussed contemporaneously.

A display device 100 may be configured to provide virtual reality images (e.g., from at least one virtual environment input), and/or mixed reality (MR) images or augmented reality (AR) images (e.g., from at least one virtual environment input and one real environment input). The display device 100, when implemented as a HMD, may comprises a headpiece arranged to be worn on the user’s head using a frame 105 (in the manner of conventional spectacles), helmet or other fit system. The purpose of the fit system is to support the display and provide stability to the display and other head borne systems such as tracking systems and cameras.

The display device 100 may include one or more optical components 115 (e.g., one or more lenses), including one or more optical waveguides 130 (see FIGS. 1B and 1C) that may allow the HMD to project images generated by one or more MicroLED projectors 120. The one or more MicroLED projectors 120 may project images to be displayed on the optical components 115. In accordance with aspects of the present disclosure, the optical component 115 may include three monolithic MicroLED projectors 120 (e.g., a first monolithic MicroLED projector 120-a, a second monolithic MicroLED projector 120-b, and a third monolithic MicroLED projector 120-c–See FIGS. 1B and 1C) that each generate a different monochrome color version of the same image. For purposes of this disclosure, the phrase “monochrome color version of an image” may refer to an image (e.g., photograph, video frame, picture, etc.) developed in varying tones of only one color. For example, a first projector 120-a may generate a first version of the image in a first color (e.g., red), a second projector 120-b may generate a second version of the image in a second color (e.g., blue), and a third projector may generate a third version of the image in a third color (e.g., green). In some examples, the MicroLED projectors 120 may be positioned either at the temple portion of the HMD or near the nasal cavity. Thus, in some aspects, two MicroLED projectors 120 may be positioned near the temple portion of the HMD device (e.g., the side of the head between the forehead and the ear–see dashed MicroLED projector 120-N) and one MicroLED projector 120 at the nasal side of the HMD (e.g., at the nose bridge portion when the HMD is worn by the user). Alternatively, in some aspects, two MicroLED projectors 120 may be positioned near the nasal side of the HMD, while the third MicroLED projector 120 may be positioned at the temple portion of the HMD. In each instance, the MicroLED projectors 120 may project the respective monochrome color versions of an image into one or more waveguides of the HMD. It should be further appreciated that, in such configuration, the plurality of MicroLED projectors 120 may be included on either the same side of the waveguide (see e.g., FIG. 1B) or on opposite sides of the waveguide (see e.g., FIG. 1C)

As illustrated in FIGS. 1B and 1C, the three MicroLED projectors 120 may be either positioned together on one side, or in different groups on opposing sides, of the optical waveguide 130 to the extent that the projectors 120 may be physically aligned with the input regions 140 of the optical waveguide 130. In some examples, the first MicroLED projector 120-a may generate the first version of the image in only red color tone, while the second MicroLED projector 120-b may generate the second version of the image in only blue color tone, and the third MicroLED projector 120-c may generate the third version of the image in only green color tone, such that the images can be combined to form a gamut of colors.

The optical components 115 may focus a user’s vision on one or more portions of one or more display panels 125, as shown in FIG. 1B. The display panels 125 may display one or more images (e.g., left eye image and right eye image) based on signals received from the plurality of monolithic MicroLED projectors 120. The optics 115 may include left eye optics 115-L for focusing the user’s left eye on the left eye image and right eye optics 115-R for focusing the user’s right eye on the right eye image. For example, the optics 115 may focus the user’s eyes on a central portion of each of the left eye image and the right eye image. The user’s brain may combine the images viewed by each eye to create the perception that the user is viewing a combined image.

In some examples, the optical components 115 may include a left and right optical components (e.g., left optical component 115-L and right optical component 115-R). The optical components 115 may use plate-shaped (usually planar) waveguides 130 for transmitting angular image information to users’ eyes as virtual images from image sources (e.g., light engine and/or MicroLED projectors 120) located out of the user’s line of sight 135. The image information may be input near one end of the waveguides 130 and is output near another end of the waveguides 130.

In some examples, the image information may propagate along the optical waveguides 130 as a plurality of angularly related beams that are internally reflected along the waveguide. The optical waveguide 130 can be either a hollow pipe with reflective inner surfaces or an integrator rod with total or partial internal reflection. Additionally or alternatively, the optical waveguide 130 may be a single optical waveguide with different input regions for inputting different color lights from the three MicroLED projectors 120 or multiple optical waveguides or plates stacked on top of each other such that each optical waveguide may input a single color light (e.g., first optical waveguide for accepting red color light from a first MicroLED projector, a second optical waveguide for accepting a blue color light from a second MicroLED projector, and a third optical waveguide for accepting a green color light from the third MicroLED projector). In either instance, the optical waveguide 130 may include an inside surface (facing the user’s eye) and an outside surface (facing the ambient environment), with both the inside and outside surfaces being exposed to air or another lower refractive index medium. As such the optical waveguide 130 may be at least partially transparent so that the user can also view the ambient environment through the waveguide.