Magic Leap Patent | Ghost image mitigation in see-through displays with pixel arrays
Publication Number: 20250208425
Publication Date: 2025-06-26
Assignee: Magic Leap
Abstract
A head-mounted apparatus include an eyepiece that include a variable dimming assembly and a frame mounting the eyepiece so that a user side of the eyepiece faces a towards a user and a world side of the eyepiece opposite the first side faces away from the user. The dynamic dimming assembly selectively modulates an intensity of light transmitted parallel to an optical axis from the world side to the user side during operation. The dynamic dimming assembly includes a variable birefringence cell having multiple pixels each having an independently variable birefringence, a first linear polarizer arranged on the user side of the variable birefringence cell, the first linear polarizer being configured to transmit light propagating parallel to the optical axis linearly polarized along a pass axis of the first linear polarizer orthogonal to the optical axis, a quarter wave plate arranged between the variable birefringence cell and the first linear polarizer, a fast axis of the quarter wave plate being arranged relative to the pass axis of the first linear polarizer to transform linearly polarized light transmitted by the first linear polarizer into circularly polarized light, and a second linear polarizer on the world side of the variable birefringence cell.
Claims
1.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a continuation of U.S. application Ser. No. 18/296,238, entitled GHOST IMAGE MITIGATION IN SEE-THROUGH DISPLAYS WITH PIXEL ARRAYS, filed on Apr. 5, 2023, now U.S. Pat. No. 11,644,675, which is a continuation of U.S. application Ser. No. 17/450,627, entitled GHOST IMAGE MITIGATION IN SEE-THROUGH DISPLAYS WITH PIXEL ARRAYS, filed on Oct. 12, 2021, which is a continuation of U.S. application Ser. No. 16/994,509, entitled GHOST IMAGE MITIGATION IN SEE-THROUGH DISPLAYS WITH PIXEL ARRAYS, filed on Aug. 14, 2020, now U.S. Pat. No. 11,169,380, which claims priority to Provisional Application No. 62/887,639, entitled GHOST IMAGE MITIGATION IN SEE-THROUGH DISPLAYS WITH PIXEL ARRAYS, filed on Aug. 15, 2019, the entire contents of which are incorporated by reference.
INCORPORATION BY REFERENCE
This application incorporates by reference the entirety of each of the following patent applications: U.S. patent application Ser. No. 15/479,700, filed on Apr. 5, 2017, published on Oct. 12, 2017 as U.S. Publication No. 2017/0293141; U.S. patent application Ser. No. 16/214,575, filed on Dec. 10, 2018, published on Jun. 13, 2019 as U.S. Publication No. 2019/0179057; U.S. Provisional Patent Application Ser. No. 62/725,993, entitled SPATIALLY-RESOLVED DYNAMIC DIMMING FOR AUGMENTED REALITY DEVICE, filed on Aug. 31, 2018; U.S. Provisional Patent Application Ser. No. 62/731,755, entitled SYSTEMS AND METHODS FOR EXTERNAL LIGHT MANAGEMENT, filed on Sep. 14, 2018; U.S. Provisional Patent Application Ser. No. 62/858,252, entitled SPATIALLY-RESOLVED DYNAMIC DIMMING FOR AUGMENTED REALITY DEVICE, filed on Jun. 6, 2019; and U.S. Provisional Patent Application Ser. No. 62/870,896, entitled GEOMETRIES FOR MITIGATING ARTIFACTS IN SEE-THROUGH PIXEL ARRAYS, filed on Jul. 5, 2019. For every document incorporated herein, in case of conflict, the current specification controls.
BACKGROUND OF THE INVENTION
Modern computing and display technologies have facilitated the development of systems for so called “virtual reality” or “augmented reality” experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or “VR,” scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or “AR,” scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user.
Despite the progress made in these display technologies, there is a need in the art for improved methods, systems, and devices related to augmented reality systems, particularly, display systems.
SUMMARY OF THE INVENTION
Segmented attenuation using a polarized TFT-LCD panel can greatly increase the visibility and solidity of content without dimming the full field of view of the world. The display light in diffractive waveguide type see through AR displays send light in two directions: one towards the user and one towards the world. The back reflection of the light (e.g., off of reflective components of the dimming assembly, such as metal traces, conductors, layer index mismatches, and TFTs) going toward the world may appear as a “ghost” image next to the primary display image. This ghost image impacts the effective contrast and viewing fidelity of the virtual content and therefore quality of immersion. Mitigating ghost images and stray light paths due to the metal traces, TFTs, layer index mismatches, elements/objects beyond the dimmer, etc. is a difficult problem.
The systems and techniques disclosed herein leverage polarization films of the dimmer as both a system optical isolator and an intra-dimmer optical isolator to effectively suppress such “ghost” images. The addition of index matching fluid/gel between the eyepiece cover glass and the dimmer, where the index is close to the first layer of the dimmer, can mitigate ghosting from the first dimmer surface.
Using quarter waveplates (QWPs) with specifically chosen achromatic properties can allow a liquid crystal dimmer, such as those using an electrically controlled birefringence (ECB) cell, to have less polarization leakage, better chromatic performance and be more color neutral across a relatively wide range of operating conditions.
The present disclosure relates generally to techniques for improving optical systems in varying ambient light conditions. More particularly, embodiments of the present disclosure provide systems and methods for operating an augmented reality (AR) device comprising a dimming element. Although the present invention is described in reference to an AR device, the disclosure is applicable to a variety of applications in computer vision and image display systems.
In general, in a first aspect, the invention features head-mounted apparatus that include an eyepiece that include a variable dimming assembly and a frame mounting the eyepiece so that, during use of the head-mounted apparatus, a user side of the eyepiece faces a towards a user of the head-mounted apparatus, and a world side of the eyepiece opposite the first side faces away from the user. The dynamic dimming assembly is configured to selectively modulate an intensity of light transmitted parallel to an optical axis from the world side of the eyepiece to the user side of the eyepiece during operation of the head-mounted apparatus. The dynamic dimming assembly includes a variable birefringence cell having multiple pixels each having an independently variable birefringence, a first linear polarizer arranged on the user side of the variable birefringence cell, the first linear polarizer being configured to transmit light propagating parallel to the optical axis linearly polarized along a pass axis of the first linear polarizer orthogonal to the optical axis, a quarter wave plate arranged between the variable birefringence cell and the first linear polarizer, a fast axis of the quarter wave plate being arranged relative to the pass axis of the first linear polarizer to transform linearly polarized light transmitted by the first linear polarizer into circularly polarized light, and a second linear polarizer on the world side of the variable birefringence cell.
Implementations of the head-mounted apparatus can include one or more of the following features and/or features of other aspects. For example, the dynamic dimming assembly further includes an optical retarder arranged between the variable birefringence cell and the second linear polarizer. The optical retarder can be a second quarter wave plate. The optical retarder is an A-plate with a retardation greater than a retardation of the quarter wave plate. A difference between a retardation of the optical retarder and a retardation of the quarter wave plate can correspond to a residual retardation of the variable birefringent cell in a minimum birefringence state.
The variable birefringence cell can include a layer of a liquid crystal. The liquid crystal can be configured in an electrically controllable birefringence mode. In some embodiments, the layer of the liquid crystal is a vertically aligned liquid crystal layer. The liquid crystal can be a nematic phase liquid crystal.
The pixels of the variable birefringence cell can be actively addressed pixels.
The eyepiece can further include a see-through display mounted in the frame on the user side of the variable dimming assembly. The see-through display can include one or more waveguide layers arranged to receive light from a light projector during operation of the head-mounted apparatus and direct the light toward the user. The head-mounted apparatus can include one or more index-matching layers arranged between the see-through display and the variable dimming assembly.
In some embodiments, the dynamic dimming assembly includes one or more antireflection layers.
In another aspect, the invention features an augmented reality system including the head-mounted apparatus.
In general, in another aspect, the invention features head-mounted apparatus that include an eyepiece having a variable dimming assembly and a frame mounting the eyepiece so that, during use of the head-mounted apparatus, a user side of the eyepiece faces a towards a user of the head-mounted apparatus, and a world side of the eyepiece opposite the first side faces away from the user. The dynamic dimming assembly is configured to selectively modulate an intensity of light transmitted parallel to an optical axis from the world side of the eyepiece to the user side of the eyepiece during operation of the head-mounted apparatus. The dynamic dimming assembly includes a layer of a liquid crystal, a circular polarizer arranged on the user side of the liquid crystal layer, and a linear polarizer on the world side of the liquid crystal layer.
Embodiments of the head-mounted apparatus can include one or more of the following features and/or features of other aspects. For example, the circular polarizer can include a linear polarizer and a quarter wave plate. The head-mounted apparatus can include an A-plate arranged between the linear polarizer on the world side of the liquid crystal layer.
The head-mounted apparatus can include a pixelated cell including the layer of the liquid crystal, the pixelated cell being an actively addressed pixelated cell.
In general, in a further aspect, the invention features a method of operating an optical system. The method may include receiving, at the optical system, light associated with a world object. The method may also include projecting a virtual image onto an eyepiece. The method may further include determining a portion of a system field of view of the optical system to be at least partially dimmed based on detected information. The method may further include adjusting a dimmer to reduce an intensity of the light associated with the world object in the portion of the system field of view.
In some embodiments, the optical system includes a light sensor configured to detect light information corresponding to the light associated with the world object. In some embodiments, the detected information includes the light information. In some embodiments, the light information includes multiple spatially-resolved light values. In some embodiments, the light information includes a global light value. In some embodiments, the optical system includes an eye tracker configured to detect gaze information corresponding to an eye of a user of the optical system. In some embodiments, the detected information includes the gaze information. In some embodiments, the gaze information includes a pixel location that intersects with a gaze vector of the eye of the user. In some embodiments, the gaze information includes one or more of a pupil position of the eye of the user, a center of rotation of the eye of the user, a pupil size of the eye of the user, a pupil diameter of the eye of the user, and cone and rod locations of the eye of the user. In some embodiments, the method further includes detecting image information corresponding to the virtual image. In some embodiments, the detected information includes the image information. In some embodiments, the image information includes a plurality of spatially-resolved image brightness values. In some embodiments, the image information includes a global image brightness value.
In some embodiments, the method further includes determining multiple spatially-resolved dimming values for the portion of the system field of view based on the detected information. In some embodiments, the dimmer is adjusted according to the plurality of dimming values. In some embodiments, the dimmer comprises a plurality of pixels. In some embodiments, the dimmer is adjusted to completely block the intensity of the light associated with the world object in all of the system field of view. In some embodiments, the method further includes adjusting a brightness associated with the virtual image. In some embodiments, the virtual image is characterized by an image field of view. In some embodiments, the image field of view is equal to the system field of view. In some embodiments, the image field of view is a subset of the system field of view.
In general, in another aspect, the invention features an optical system. The optical system may include a projector configured to project a virtual image onto an eyepiece. The optical system may also include a dimmer configured to dim light associated with a world object. The optical system may further include a processor communicatively coupled to the projector and the dimmer. In some embodiments, the processor is configured to perform operations including determining a portion of a system field of view of the optical system to be at least partially dimmed based on detected information. In some embodiments, the operations may also include adjusting the dimmer to reduce an intensity of the light associated with the world object in the portion of the system field of view.
In some embodiments, the optical system further includes a light sensor configured to detect light information corresponding to the light associated with the world object. In some embodiments, the detected information includes the light information. In some embodiments, the light information includes a plurality of spatially-resolved light values. In some embodiments, the light information includes a global light value. In some embodiments, the optical system further includes an eye tracker configured to detect gaze information corresponding to an eye of a user of the optical system. In some embodiments, the detected information includes the gaze information. In some embodiments, the gaze information includes a pixel location that intersects with a gaze vector of the eye of the user. In some embodiments, the gaze information includes one or more of a pupil position of the eye of the user, a center of rotation of the eye of the user, a pupil size of the eye of the user, a pupil diameter of the eye of the user, and cone and rod locations of the eye of the user. In some embodiments, the operations further include detecting image information corresponding to the virtual image. In some embodiments, the detected information includes the image information. In some embodiments, the image information includes a plurality of spatially-resolved image brightness values. In some embodiments, the image information includes a global image brightness value.
In some embodiments, the operations further include determining multiple spatially-resolved dimming values for the portion of the system field of view based on the detected information. In some embodiments, the dimmer is adjusted according to the plurality of dimming values. In some embodiments, the dimmer comprises a plurality of pixels. In some embodiments, the dimmer is adjusted to completely block the intensity of the light associated with the world object in all of the system field of view. In some embodiments, the operations further include adjusting a brightness associated with the virtual image. In some embodiments, the virtual image is characterized by an image field of view. In some embodiments, the image field of view is equal to the system field of view. In some embodiments, the image field of view is a subset of the system field of view.
Numerous benefits can be achieved by way of the present disclosure over conventional techniques. For example, the AR device described herein may be used in varying light levels, from dark indoors to bright outdoors, by globally dimming and/or selectively dimming the ambient light reaching the user's eyes. Embodiments of the present invention allow for AR and virtual reality (VR) capabilities in a single device by using the pixelated dimmer to attenuate the world light by greater than 99%. Embodiments of the present invention also mitigate vergence accommodation conflict using a variable focal element with discrete or continuous variable depth plane switching technologies. Embodiments of the present invention improve the battery life of the AR device by optimizing the projector brightness based on the amount of detected ambient light. Other benefits will be readily apparent to those skilled in the art.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates an augmented reality (AR) scene as viewed through a wearable AR device according to some embodiments described herein.
FIG. 2A illustrates one or more general features of an AR device according to the present invention.
FIG. 2B illustrates an example of an AR device in which a dimmed area is determined based on detected light information.
FIG. 2C illustrates an example of an AR device in which a dimmed area is determined based on a virtual image.
FIG. 2D illustrates an example of an AR device in which a dimmed area is determined based on gaze information.
FIG. 3 illustrates a schematic view of a wearable AR device according to the present invention.
FIG. 4 illustrates a method for operating an optical system.
FIG. 5 illustrates an AR device with an eyepiece and a pixelated dimming element.
FIG. 6A is a front view of an example optically-transmissive spatial light modulator (“SLM”) or display assembly for a see-through display system, according to some embodiments of the present disclosure.
FIG. 6B is a schematic side view of the example SLM or display assembly of FIG. 6A.
FIG. 7A illustrates an example of a see-through display system in a first state.
FIG. 7B illustrates an example of the see-through display system of FIG. 7A in a second state different from the first state.
FIG. 8A illustrates an example of another see-through display system in a first state.
FIG. 8B illustrates an example of the see-through display system of FIG. 8A in a second state different from the first state.
FIG. 9 illustrates an example of a see-through display system according to some embodiments of the present disclosure.
FIG. 10 illustrates an example of a see-through display system according to other embodiments of the present disclosure.
FIG. 11 illustrates an example of a see-through display system according to yet other embodiments of the present disclosure.
FIG. 12 is a diagram of an example computer system useful with a see-through display system.
