Magic Leap Patent | Display System And Method For Providing Variable Accommodation Cues Using Multiple Intra-Pupil Parallax Views Formed By Light Emitter Arrays
Publication Number: 20200301147
Publication Date: 20200924
Applicants: Magic Leap
Abstract
A display system is configured to direct a plurality of parallactically-disparate intra-pupil images into a viewer’s eye. The parallactically-disparate intra-pupil images provide different parallax views of a virtual object, and impinge on the pupil from different angles. In the aggregate, the wavefronts of light forming the images approximate a continuous divergent wavefront and provide selectable accommodation cues for the user, depending on the amount of parallax disparity between the intra-pupil images. The amount of parallax disparity may be selected using an array of shutters that selectively regulate the entry of image light into an eye. Each opened shutter in the array provides a different intra-pupil image, and the locations of the open shutters provide the desired amount of parallax disparity between the images. In some other embodiments, the images may be formed by an emissive micro-display. Each pixel formed by the micro-display may be formed by one of a group of light emitters, which are at different locations such that the emitted light takes different paths to the eye, the different paths providing different amounts of parallax disparity.
PRIORITY CLAIM
[0001] This application claims priority to: U.S. Provisional Application No. 62/812,142, filed on Feb. 28, 2019; and U.S. Provisional Application No. 62/815,225, filed on Mar. 7, 2019, which are incorporated herein by reference.
INCORPORATION BY REFERENCE
[0002] This application incorporates by reference the entirety of each of the following patent applications and publications: U.S. Application Publ. No. 2018/0113311, published Apr. 26, 2018; U.S. application Ser. No. 14/555,585, filed on Nov. 27, 2014; U.S. application Ser. No. 14/690,401, filed on Apr. 18, 2015; U.S. application Ser. No. 14/212,961, filed on Mar. 14, 2014; U.S. application Ser. No. 14/331,218, filed on Jul. 14, 2014; U.S. application Ser. No. 15/072,290, filed on Mar. 16, 2016; and WO 2016/179246, published Nov. 10, 2016; and U.S. Prov. Application No. 62/800,363, filed Feb. 1, 2019.
BACKGROUND
Field
[0003] The present disclosure relates to optical devices, including augmented reality and virtual reality imaging and visualization systems.
Description of the Related Art
[0004] Modern computing and display technologies have facilitated the development of systems for so called “virtual reality” or “augmented reality” experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or “VR”, scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or “AR”, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user. A mixed reality, or “MR”, scenario is a type of AR scenario and typically involves virtual objects that are integrated into, and responsive to, the natural world. For example, in an MR scenario, AR image content may be blocked by or otherwise be perceived as interacting with objects in the real world.
[0005] Referring to FIG. 1, an augmented reality scene 10 is depicted wherein a user of an AR technology sees a real-world park-like setting 20 featuring people, trees, buildings in the background, and a concrete platform 30. In addition to these items, the user of the AR technology also perceives that he “sees” “virtual content” such as a robot statue 40 standing upon the real-world platform 30, and a cartoon-like avatar character 50 flying by, which seems to be a personification of a bumble bee, even though these elements 40, 50 do not exist in the real world. Because the human visual perception system is complex, it is challenging to produce an AR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements.
[0006] Systems and methods disclosed herein address various challenges related to AR and VR technology.
SUMMARY
[0007] In some embodiments, a head-mounted display system is provided. The display system includes an image projection system comprising: a micro-display configured to output image light defining images; and projection optics configured to direct the image light from the micro-display for propagation to an eye of a viewer. The display system also comprises an array of selectively-activated shutters for selectively transmitting the image light to the eye from different locations. The array of selectively-activated shutters is disposed within an eye-box volume of the projection optics.
[0008] In some other embodiments, a method for displaying image content is provided. The method comprises injecting, from a head-mounted display system, a set of parallactically-disparate intra-pupil images of a virtual object into an eye of a viewer. Each image of the intra-pupil images is provided by: forming the image on a micro-display of the head-mounted display system; outputting image light from the micro-display through projection optics; and opening a shutter of an array of shutters to propagate image light through the opened shutter to the eye. The array of shutters is disposed within an eye box volume of the projection optics. Different images of the set of parallactically-disparate intra-pupil images propagate through different opened shutters.
[0009] In yet other embodiments, a head-mounted display system is provided. The display system comprises a micro-display comprising an array of groups of light emitters; an array of light collimators overlying the light emitters; and projection optics. Each light collimator is associated with one of the groups of light emitters and extends across all light emitters of the associated group of light emitters. The array of light collimators is between the light emitters and the projection optics. The display system is configured to display a virtual object on a depth plane by injecting a set of parallactically-disparate intra-pupil images of the object into an eye of a viewer.
[0010] In some other embodiments, a method for displaying image content is provided. The method comprises injecting, from a head-mounted display system, a set of parallactically-disparate intra-pupil images into an eye of a viewer. Injecting the set of parallactically-disparate intra-pupil images comprises: providing an array of groups of light emitters; providing an array of light collimators overlying the light emitters, wherein each light collimator is associated with a group of the light emitters; providing projection optics, wherein the array of light collimators is between the array of groups of light emitters and the projection optics, injecting a first parallactically-disparate intra-pupil image into the eye by emitting light from a first light emitter of the groups of light emitters; and injecting a second parallactically-disparate intra-pupil image into the eye by emitting light from a second light emitter of the groups of light emitters.
[0011] In addition, various innovative aspects of the subject matter described in this disclosure may be implemented in the following examples:
Example 1
[0012] A head-mounted display system comprising: [0013] an image projection system comprising: [0014] a micro-display configured to output image light defining images; and [0015] projection optics configured to direct the image light from the micro-display for propagation to an eye of a viewer; and [0016] an array of selectively-activated shutters for selectively transmitting the image light to the eye from different locations, wherein the array of selectively-activated shutters is disposed within an eye-box volume of the projection optics.
Example 2
[0017] The display system of Example 1, further comprising a control system comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the display system to perform operations comprising: [0018] determining a desired depth plane for a virtual object; [0019] determining shutters of the array of selectively-activated shutters to be opened based upon the desired depth plane; [0020] synchronizing presentation of different images, by the image projection system, with opening of different ones of the shutters, wherein the different images provide different views of the virtual object.
Example 3
[0021] The display system of any of Examples 1 or 2, wherein the shutters are moveable physical structures.
Example 4
[0022] The display system of Example 3, wherein the physical structures are mems-based micro-mechanical structures.
Example 5
[0023] The display system of Example 3, wherein the shutters are ferro-electric shutters.
Example 6
[0024] The display system of any of Examples 1 or 2, wherein the shutters comprise chemical species having reversibly changeable states, the states providing different amounts of light transmission.
Example 7
[0025] The display system of Example 6, wherein the chemical species comprise liquid crystals, wherein the shutters are formed by pixels of a pixelated liquid crystal display.
Example 8
[0026] The display system of any of Examples 1-7, wherein the micro-display is an emissive micro-display comprising an array of light emitters.
Example 9
[0027] The display system of Example 8, wherein the light emitters are micro-LEDs.
Example 10
[0028] The display system of any of Examples 1-9, further comprising an array of light collimators between the light emitters and the projection optics.
Example 11
[0029] The display system of Example 10, wherein each of the array of light collimators extends across a plurality of the light emitters, wherein each light collimator corresponds to a pixel in images outputted by the image projection system.
Example 12
[0030] The display system of any of Examples 1-11, wherein the micro-display is one of a plurality of monochrome micro-displays forming the projection system, wherein each of the monochrome micro-displays is configured to emit light of a different component color.
Example 13
[0031] The display system of Example 12, further comprising an X-cube prism, wherein each of the monochrome micro-displays is arranged to output image light into a different face of the X-cube prism.
Example 14
[0032] The display system of any of Examples 1-13, further comprising a pupil relay combiner eyepiece configured to relay the image light to the eye of the viewer, wherein the array of selectively-activated shutters are configured to regulate propagation of the image light to the pupil relay combiner eyepiece.
Example 15
[0033] The display system of Example 14, wherein the pupil relay combiner eyepiece comprises a waveguide comprising: [0034] in-coupling optical elements for in-coupling the image light into the waveguide; and [0035] out-coupling optical elements for out-coupling in-coupled image light out of the waveguide.
Example 16
[0036] The display system of Example 15, wherein the waveguide is one of a plurality of waveguides comprising in-coupling optical elements and out-coupling optical elements.
Example 17
[0037] The display system of any of Examples 1-16, wherein the projection system has a pupil diameter of 0.2-0.5 mm.
Example 18
[0038] A method for displaying image content, the method comprising: [0039] injecting, from a head-mounted display system, a set of parallactically-disparate intra-pupil images of a virtual object into an eye of a viewer, [0040] wherein each image of the intra-pupil images is provided by: [0041] forming the image on a micro-display of the head-mounted display system; [0042] outputting image light from the micro-display through projection optics; and [0043] opening a shutter of an array of shutters to propagate image light through the opened shutter to the eye, wherein the array of shutters is disposed within an eye box volume of the projection optics, [0044] wherein different images of the set of parallactically-disparate intra-pupil images propagate through different opened shutters.
Example 19
[0045] The method of Example 18, wherein all images of the set of parallactically-disparate intra-pupil images are injected into the eye within a flicker fusion threshold.
Example 20
[0046] The method of Example 18, wherein the flicker fusion threshold is 1/60 of a second.
Example 21
[0047] The method of any of Examples 18-20, further comprising: [0048] determining a desired depth plane for the virtual object to be displayed to the viewer; [0049] determining shutters of the array of selectively-activated shutters to be opened based upon the desired depth plane; and [0050] synchronizing presentation of different ones of the set of parallactically-disparate intra-pupil images with opening of different ones of the shutters.
Example 22
[0051] The method of any of Examples 18-21, further comprising: [0052] determining a gaze of the eye using an eye tracking sensor; and [0053] selecting content for the intra-pupil images based upon the determined gaze of the eye.
Example 23
[0054] The method of any of Examples 18-22, wherein the micro-display is an emissive micro-display.
Example 24
[0055] The method of any of Examples 18-23, wherein the array of shutters comprises selectively-movable physical structures.
Example 25
[0056] The method of any of Examples 18-23, wherein the array of shutters comprises chemical species having reversibly changeable states, the states providing different amounts of light transmission.
Example 26
[0057] The method of any of Examples 18-25, wherein the different images provide different views of the virtual object.
Example 27
[0058] A head-mounted display system comprising: [0059] a micro-display comprising an array of groups of light emitters; [0060] an array of light collimators overlying the light emitters, wherein each light collimator is associated with one of the groups of light emitters and extends across all light emitters of the associated group of light emitters; [0061] projection optics, wherein the array of light collimators is between the light emitters and the projection optics, [0062] wherein the display system is configured to display a virtual object on a depth plane by injecting a set of parallactically-disparate intra-pupil images of the object into an eye of a viewer.
Example 28
[0063] The display system of Example 27, further comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the display system to perform operations comprising: [0064] determining light emitters of each of the groups of light emitters to activate based upon a desired level of parallax disparity for images formed by light emitters; [0065] activating a first light emitter of the groups of light emitters to form a first parallactically-disparate intra-pupil image; and [0066] activating a second light emitter of the groups of light emitters to form a second parallactically-disparate intra-pupil image, [0067] wherein the first and second parallactically-disparate intra-pupil images provide different views of the virtual object.
Example 29
[0068] The display system of Example 27, wherein activating the first light emitter of the groups of light emitters overlaps in time with activating the second light emitter of the groups of light emitters, to inject the first and second parallactically-disparate intra-pupil images into the eye simultaneously.
Example 30
[0069] The display system of any of Examples 27-29, wherein the light collimators are lenslets.
Example 31
[0070] The display system of any of Examples 27-30, further comprising an array of selectively-activated shutters for selectively transmitting the image light to the eye from different locations, wherein the array of selectively-activated shutters is disposed within an eye-box volume of the projection optics.
Example 32
[0071] The display system of Example 31, wherein the array of shutters comprises selectively-movable physical structures.
Example 33
[0072] The display system of Example 31, wherein the array of shutters comprises chemical species having reversibly changeable states, the states providing different amounts of light transmission.
Example 34
[0073] The display system of any of Examples 31-33, further comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the display system to perform operations comprising: [0074] determining a desired depth plane for a virtual object; [0075] determining shutters of the array of selectively-activated shutters to be opened based upon the desired depth plane; [0076] synchronizing presentation of different images, by the image projection system, with opening of different ones of the shutters, wherein the different images provide different views of the virtual object.
Example 35
[0077] The display system of any of Examples 27-34, wherein the light collimators are lenticular lenslets configured to provide different beams of light, from light emitters of an associated group of light emitters, to different locations along a first axis, [0078] wherein the array of shutters are arranged to form subpupils along a second axis orthogonal to the first axis.
Example 36
[0079] The display system of any of Examples 27-35, wherein the micro-display is an emissive micro-display, wherein the light emitters are micro-LEDs.
Example 37
[0080] The display system of any of Examples 27-36, wherein the micro-display is one of a plurality of monochrome micro-displays, wherein each of the monochrome micro-displays is configured to emit light of a different component color.
Example 38
[0081] The display system of Example 37, further comprising an X-cube prism, wherein each of the monochrome micro-displays is arranged to output image light into a different face of the X-cube prism.
Example 39
[0082] The display system of any of Examples 27-38, further comprising a pupil relay combiner eyepiece configured to relay the image light to the eye of the viewer, wherein the array of selectively-activated shutters are configured to regulate propagation of the image light to the pupil relay combiner eyepiece.
Example 40
[0083] The display system of Example 39, wherein the pupil relay combiner eyepiece comprises a waveguide comprising: [0084] in-coupling optical elements for in-coupling the image light into the waveguide; and [0085] out-coupling optical elements for out-coupling in-coupled image light out of the waveguide.
Example 41
[0086] The display system of Example 40, wherein the waveguide is one of a plurality of waveguides comprising in-coupling optical elements and out-coupling optical elements.
Example 42
[0087] A method for displaying image content, the method comprising: [0088] injecting, from a head-mounted display system, a set of parallactically-disparate intra-pupil images into an eye of a viewer, [0089] wherein injecting the set of parallactically-disparate intra-pupil images comprises: [0090] providing an array of groups of light emitters; [0091] providing an array of light collimators overlying the light emitters, wherein each light collimator is associated with a group of the light emitters; [0092] providing projection optics, wherein the array of light collimators is between the array of groups of light emitters and the projection optics, [0093] injecting a first parallactically-disparate intra-pupil image into the eye by emitting light from a first light emitter of the groups of light emitters; and [0094] injecting a second parallactically-disparate intra-pupil image into the eye by emitting light from a second light emitter of the groups of light emitters.
Example 43
[0095] The method of Example 42, wherein each of the images of the set of parallactically-disparate intra-pupil images are injected into the eye at different angles and all images of the set of parallactically-disparate intra-pupil images are injected into the eye within a flicker fusion threshold.
Example 44
[0096] The method of Example 43, wherein the flicker fusion threshold is 1/60 of a second.
Example 45
[0097] The method of any of Examples 42-43, wherein the different images provide different views of the virtual object.
Example 46
[0098] The method of any of Examples 42-45, wherein injecting the first parallactically-disparate intra-pupil image and injecting the second parallactically-disparate intra-pupil image are performed simultaneously.
Example 47
[0099] The method of any of Examples 42-46, further comprising providing an array of selectively-activated shutters for selectively transmitting the image light to the eye from different locations, wherein the array of selectively-activated shutters is disposed within an eye-box volume of the projection optics.
Example 48
[0100] The method of any of Examples 42-47, wherein the light collimators are lenticular lenslets configured to provide different beams of light, from light emitters of an associated group of light emitters, to different locations along a first axis, wherein the array of shutters are arranged to form subpupils along a second axis orthogonal to the first axis.
Example 49
[0101] The method of any of Examples 47-48, further comprising spatially-multiplexing multiple images formed by different light emitters of the groups of light emitters to localize a display subpupil along the first axis, and temporally-multiplexing multiple images by synchronizing opening of the shutters with activation of corresponding light emitters.
Example 50
[0102] The method of any of Examples 47-49, wherein the array of shutters comprises selectively-movable physical structures.
Example 51
[0103] The method of any of Examples 47-49, wherein the array of shutters comprises chemical species having reversibly changeable states, the states providing different amounts of light transmission.
Example 52
[0104] The method of any of Examples 42-51, wherein injecting the first parallactically-disparate intra-pupil image and injecting the second parallactically-disparate intra-pupil image comprise routing light from the light emitters to the eye through a pupil relay combiner eyepiece.
Example 53
[0105] The method of Example 52, wherein the pupil relay combiner eyepiece comprises a waveguide comprising: [0106] in-coupling optical elements for in-coupling the image light into the waveguide; and [0107] out-coupling optical elements for out-coupling in-coupled image light out of the waveguide.
Example 54
[0108] The method of any of Examples 42-53, further comprising injecting, from the head-mounted display system, a second set of parallactically-disparate intra-pupil images into a second eye of a viewer.
BRIEF DESCRIPTION OF THE DRAWINGS
[0109] FIG. 1 illustrates a user’s view of augmented reality (AR) through an AR device.
[0110] FIG. 2 illustrates a conventional scheme for simulating three-dimensional imagery for a user.
[0111] FIG. 3 illustrates aspects of an approach for simulating three-dimensional imagery using multiple depth planes.
[0112] FIGS. 4A-4C illustrate relationships between wavefront curvature and focal distance.
[0113] FIG. 5 illustrates an example of a waveguide stack for outputting image information to a user.
[0114] FIG. 6A illustrates pre-accommodation and post-accommodation conditions of an eye for a continuous incoming wavefront.
[0115] FIG. 6B illustrates pre-accommodation and post-accommodation conditions of an eye to a piecewise approximation for a continuous incoming wavefront.
[0116] FIG. 7A illustrates an eye accommodating to a divergent wavefront emanating from a finite focal-distance virtual image provided by a projection system.
[0117] FIG. 7B illustrates a system for forming an approximation of the divergent wavefront of FIG. 7A utilizing wavefront segments formed by infinity-focused virtual images.
[0118] FIG. 8 illustrates examples of parallax views forming the divergent wavefront approximation of FIG. 7B.
[0119] FIG. 9 illustrates an example of a display system comprising a projection system for forming the divergent wavefront approximation of FIGS. 7B and 8.
[0120] FIGS. 10A-10C illustrate examples of arrays of shutters for providing different parallax views.
[0121] FIG. 11 illustrates an example of the projection system of FIG. 9 providing a different parallactically-disparate intra-pupil image at a different time than that shown in FIG. 9.
[0122] FIG. 12 illustrates an example of the projection systems of FIGS. 9 and 11 with an array of light collimators between an emissive micro-display and projection optics.
[0123] FIG. 13 illustrates another example of a display system comprising a projection system for forming the divergent wavefront approximation of FIGS. 7B and 8.
[0124] FIG. 14 illustrates an example of a projection system having an array of light collimators and an array of shutters for forming different parallactically-disparate intra-pupil images.
[0125] FIG. 15 illustrates an example of a range of depth planes provided by projection systems according to various embodiments.
[0126] FIG. 16A illustrates an example of a display system comprising a projection system, comprising an array of shutters, and a pupil relay combiner eyepiece for superimposing image content on a user’s view of the world.
[0127] FIG. 16B illustrates an example of a display system comprising a projection system, comprising an array of light collimators for providing different intra-pupil images, and a pupil relay combiner eyepiece for superimposing image content on a user’s view of the world.
[0128] FIG. 17A illustrates another example of a display system comprising a projection system, comprising an array of shutters, and a pupil relay combiner eyepiece for superimposing image content on a user’s view of the world.
[0129] FIG. 17B illustrates another example of a display system comprising a projection system, comprising an array of light collimators for providing different intra-pupil images, and a pupil relay combiner eyepiece for superimposing image content on a user’s view of the world.
[0130] FIG. 18 illustrates an example of a display system comprising an eye tracking system and a combiner eyepiece with a pupil expander.
[0131] FIG. 19 illustrates an example of a display system comprising an eye tracking system and a pupil rely combiner eyepiece with a pupil expander configured to produce a non-infinity depth plane.
[0132] FIGS. 20A-20B illustrate examples of projection systems having multiple micro-displays.
[0133] FIG. 21 illustrates an example of an eyepiece comprising a stacked waveguide assembly for outputting light of different wavelengths corresponding to different component colors.
[0134] FIG. 22 illustrates an example of a wearable display system.
DETAILED DESCRIPTION
[0135] The human visual system may be made to perceive images presented by a display as being “3-dimensional” by providing slightly different presentations of the image to each of a viewer’s left and right eyes. Depending on the images presented to each eye, the viewer perceives a “virtual” object in the images as being at a selected distance (e.g., at a certain “depth plane”) from the viewer (also referred to as “user” herein). Simply providing different presentations of the image to the left and right eyes, however, may cause viewer discomfort. As discussed further herein, viewing comfort may be increased by causing the eyes to accommodate to the images as the eyes would accommodate to a real object at the depth plane on which the virtual object is placed.
[0136] The proper accommodation for a virtual object on a given depth plane may be elicited by presenting images to the eyes with light having a wavefront divergence that matches the wavefront divergence of light coming from a real object on that depth plane. Some display systems use distinct structures having distinct optical powers to provide the appropriate wavefront divergence. For example, one structure may provide a specific amount of wavefront divergence (to place virtual objects on one depth plane) and another structure may provide a different amount of wavefront divergence (to place virtual objects on a different depth plane). Thus, there may be a one-to-one correspondence between physical structures and the depth planes in these display systems. Due to the need for a separate structure for each depth plane, such display systems may be bulky and/or heavy, which may be undesirable for some applications, such as portable head-mounted displays. In addition, such display systems may be limited in the numbers of different accommodative responses they may elicit from the eyes, due to practical limits on the number of structures of different optical powers that may be utilized.
[0137] It has been found that a continuous wavefront, e.g. a continuous divergent wavefront, may be approximated by injecting parallactically-disparate intra-pupil images directed into an eye. For example, a display system may provide a range of accommodative responses without requiring a one-to-one correspondence between the accommodative response and the optical structures in the display. The display system may output light with a selected amount of perceived wavefront divergence, corresponding to a desired depth plane, by injecting a set of parallactically-disparate intra-pupil images into the eye. These images may be referred to as “parallactically-disparate” intra-pupil images since each image may be considered to be a different parallax view of the same virtual object or scene, on a given depth plane. These are “intra-pupil” images since a set of images possessing parallax disparity is projected into the pupil of a single eye, e.g., the right or left eye of a viewer. Although the images may have some overlap, the light beams forming these images will have at least some areas without overlap and will impinge on the pupil from slightly different angles. In some embodiments, the other eye of the viewer, e.g., the left eye, may be provided with its own set of parallactically-disparate intra-pupil images. The sets of parallactically-disparate intra-pupil images projected into each eye may be slightly different, e.g., the images may show slightly different views of the same scene due to the slightly different perspectives provided by each eye.
[0138] The wavefronts of light forming each of the intra-pupil images of the different views, when projected into a pupil of an eye, may, in the aggregate, approximate a continuous divergent wavefront. The amount of perceived divergence of this approximated wavefront may be varied by varying the amount of parallax disparity between the intra-pupil images; changes in the parallax disparity change the angular range spanned by the wavefronts of light forming the intra-pupil images. Preferably, this angular range mimics the angular range spanned by the continuous wavefront being approximated. In some embodiments, the wavefronts of light forming the individual intra-pupil images are collimated or quasi-collimated, as discussed herein. Examples of systems for providing intra-pupil images are disclosed in U.S. Application Publ. No. 2018/0113311 published Apr. 26, 2018. Some embodiments disclosed in that application utilize light emitters to illuminate a spatial light modulator which encodes light from the light emitters with image information; multiple light emitters may be provided and the different intra-pupil images may be formed using light emitters at different locations to provide desired amounts of parallax disparity between images.
[0139] Preferably, the set of intra-pupil images for approximating a particular continuous wavefront is injected into the eye sufficiently rapidly for the human visual system to not detect that the images were provided to the eye at different times. Without being limited by theory, the term flicker fusion threshold may be used to denote the duration within which images presented to the human eye are perceived as being present simultaneously; that is, the visual system may perceive images formed on the retina within a flicker fusion threshold as being present simultaneously. In some embodiments, approximating a continuous wavefront may include sequentially injecting beams of light for each of a set of intra-pupil images into the eye, with the total duration for injecting all of the beams of light being less than the flicker fusion threshold. It will be appreciated that presenting the set of images over a duration greater than the flicker fusion threshold may result in the human visual system perceiving at least some of the images as being separately injected into the eye. As an example, the flicker fusion threshold may be about 1/60 of a second. Consequently, each set of intra-pupil images may consist of a particular number of parallax views, e.g., two or more views, three or more views, four or more views, etc. and preferably all of these views are provided to the eye within the flicker fusion threshold.
[0140] Providing all of the desired views within the flicker fusion threshold presents a challenge for some display technologies, such as those using spatial light modulators in which physical elements are moved to modulate the intensity of outputted light. The need to physically move these elements may limit the speed at which individual pixels may change states and also constrains the frame rate of displays using these optical elements. In addition, spatial light modulators may require a separate light source, which can undesirably add to the complexity, size, and weight of the display system, and may potentially limit the brightness of displayed images.
[0141] In some embodiments, a display system includes an emissive micro-display, which advantageously may provide different intra-pupil images at exceptionally high rates. In addition, the display system may include an array of shutters. The shutters of the array of shutters may be individually selectively opened, or activated to allow light transmission, to allow light to propagate into the retina from different locations. The emissive micro-display emits image light for forming an intra-pupil images and the image light propagates to the array of shutters. Different ones of the shutters at different locations may be selectively opened (made transmissive to the image light) to allow the image light to further propagate into a viewer’s eye from those different locations. The amount of parallax disparity between the intra-pupil images may be varied by changing the locations at which an shutter is opened. Consequently, spatial differences in the opened shutter locations may translate into differences in the paths that the light takes into the eye. The different paths may correspond to different amounts of parallax disparity. In some embodiments, an array of light collimators may be disposed proximate the emissive micro-display. For example, each pixel of the emissive micro-display may have an associated light collimator. The light collimators narrow the angular emission profile of light emitted by the emissive micro-display, and thereby nay increase the amount of the emitted light that ultimately reaches the eyes of the viewer.
[0142] It will be appreciated that the images formed by the emissive micro-display may be temporally synchronized with the shutter that is opened in the array of shutters. For example, the opening of one shutter (or multiple adjacent or contiguous shutters) corresponding to one intra-pupil image may be synchronized, or simultaneous, with the activation of pixels in the micro-display. Once another shutter, at a location desired for a second intra-pupil image, is opened, the micro-display may emit light for forming that second intra-pupil image. Additional intra-pupil images may be formed by synchronizing with the opening of shutters at different locations. This time-based sequential injection of intra-pupil images to the eye may be referred to as temporal multiplexing or temporally multiplexed display of the intra-pupil images. As a result, in some embodiments, the presentation of the intra-pupil images by the micro-display may be temporally multiplexed, such that different parallax views may be provided by the emissive micro-display at different times and synchronized with the opening of different shutters providing the desired parallax disparity.
[0143] Preferably, the emissive micro-displays are micro-LED displays, which provide advantages for high brightness and high pixel density. In some other embodiments, the micro-displays are micro-OLED displays.
[0144] In some embodiments, the emissive micro-displays comprise arrays of light emitters having a pitch of, e.g., less than 10 .mu.m, less than 8 .mu.m, less than 6 .mu.m, less than 5 .mu.m, or less than 2 .mu.m, including 1-5 .mu.m, 1-4 .mu.m or 1-2 .mu.m; and an emitter size of 2 .mu.m or less, 1.7 .mu.m or less, or 1.3 .mu.m or less. In some embodiments, the emitter size is within a range having an upper limit of the above-noted sizes and a lower limit of 1 .mu.m. Examples of the ratio of emitter size to pitch include 1:1 to 1:5, 1:2 to 1:4, or 1:2 to 1:3,
[0145] In some embodiments, the display system may utilize an emissive micro-display in conjunction with a light collimator array to provide different amounts of parallax disparity. The collimator array may be configured to direct light, emitted by the micro-display, along different paths which correspond to the different amounts of parallax disparity. For example, the collimator array may be positioned proximate to or directly on the emissive micro-display. In some embodiments, the light collimators are lenslets. Each collimator of the collimator array may include a group of associated subpixels, each disposed at a different location relative to the collimator. As a result, light from different subpixels of the group of subpixels interfaces differently with the collimator and is directed along slightly different paths by the collimator. These different paths may correspond to different amounts of parallax disparity. Thus, each collimator may correspond to a different pixel of an intra-pupil image, and each subpixel may provide a different light path for that pixel, such that the parallax disparity between two or more pixels may be selected by appropriate activation of the sub-pixels forming those pixels. In some embodiments, advantageously, different intra-pupil images may be formed and provided to the eye simultaneously, with the parallax disparity determined by the locations of the subpixels forming the images and with the collimator array directing the propagation of light from those subpixels.
[0146] As noted above, light from different subpixels will take different paths to the projection optic and thus to the viewer’s eyes. Consequently, lateral and/or vertical displacement of the active subpixels translates into angular displacement in the light leaving the light collimator array and ultimately propagating towards the viewer’s pupil through the projection optic. In some embodiments, increases in lateral displacement between the activated subpixels used to form different images may be understood to translate to increases in angular displacement as measured with respect to the micro-display. In some embodiments, each of the intra-pupil images, used to approximate a particular wavefront, may be formed by outputting light from a different subpixel, thereby providing the angular displacement between the beams of light forming each of the images.
[0147] In some embodiments, the display systems, whether utilizing a collimator array or an array of shutters, may include projection optics for injecting light into the eye. The emissive micro-display may be configured to output the light, encoded with image information, to form an intra-pupil image. Light subsequently impinges on and propagates through the projection optics and, ultimately, to the eye of a viewer.
[0148] In some embodiments, the display system may include a combiner eyepiece, which allows virtual image content to be overlaid with the viewer’s view of the world, or ambient environment. For example, the combiner eyepiece may be an optically transmissive waveguide that allows the viewer to see the world. In addition, the waveguide may be utilized to receive, guide, and ultimately output light, forming the intra-pupil images, to the viewer’s eyes. Because the waveguide may be positioned between the viewer and the world, the light outputted by the waveguide may be perceived to form virtual images that are placed on various depth planes in the world. In essence, the combiner eyepiece allows the viewer to receive a combination of light from the display system and light from the world.
[0149] In some embodiments, the display system may also include an eye tracking system to detect the viewer’s gaze direction. Such an eye tracking system allows appropriate content to be selected and displayed based upon where the viewer is looking.
[0150] Preferably, the display system has a sufficiently small exit pupil that the depth of field provided by light forming individual intra-pupil images is substantially infinite and the visual system operates in an “open-loop” mode in which the eye is unable to accommodate to an individual intra-pupil image. In some embodiments, the light beams forming individual images occupy an area having a width or diameter less than about 0.5 mm when incident on the eye. It will be appreciated, however, that light beams forming a set of intra-pupil images are at least partially non-overlapping and the set of light beams preferably define an area larger than 0.5 mm, to provide sufficient information to the lens of the eye to elicit a desired accommodative response based on the wavefront approximation formed by the wavefronts of the light forming the intra-pupil images.
[0151] Without being limited by theory, the area defined by a set of beams of light may be considered to mimic a synthetic aperture through which an eye views a scene. It will be appreciated that viewing a scene through a sufficiently small pinhole in front of the pupil provides a nearly infinite depth of field. Given the small aperture of the pinhole, the lens of the eye is not provided with adequate scene sampling to discern distinct depth of focus. As the pinhole enlarges, additional information is provided to the eye’s lens, and natural optical phenomena allow a limited depth of focus to be perceived. Advantageously, the area defined by the set of beams of light and the corresponding sets of parallactically-disparate intra-pupil images may be made larger than the pinhole producing the infinite depth of field and the multiple intra-pupil images may produce an approximation of the effect provided by the enlarged pinhole noted above.
[0152] Some embodiments disclosed herein may provide various advantages. For example, because the micro-displays are emissive, no external illumination is required, thereby facilitating reductions in the size and weight of the projection system. The small size of these micro-displays allows the use of a single projector with separate component color (e.g., red, green, blue) micro-display panels, without requiring an unduly large or complicated projector. In some embodiments, because of the advantageously small size and weight of various micro-display disclosed herein, different projectors may be used for different component colors. In addition, in contrast to typical displays, such as LCOS displays, polarization is not needed to provide light with image information. As a result, light loss associated with polarization may be avoided. Also, the individual light emitters of the micro-displays have high etendue and, as a result, light from each pixel naturally fills a large super-pupil area, which can provide a desirably large eye-box volume. In some embodiments, the emissive micro-display is a micro-LED display, which may have exceptionally high frame rates (e.g., frame rate of 1 kHz or more, including 1-2 kHz). In addition, the emissive micro-displays may have exceptionally small pixel pitches (e.g., 1-4 .mu.m, including 2-4 .mu.m or 1-2 .mu.m) and high pixel density, which may provide desirably high image resolutions.
[0153] Reference will now be made to the figures, in which like reference numerals refer to like parts throughout.
[0154] As discussed herein, the perception of an image as being “three-dimensional” or “3-D” may be achieved by providing slightly different presentations of the image to each eye of the viewer. FIG. 2 illustrates a conventional scheme for simulating three-dimensional imagery for a user. Two distinct images 190, 200–one for each eye 210, 220–are outputted to the user. The images 190, 200 are spaced from the eyes 210, 220 by a distance 230 along an optical or z-axis that is parallel to the line of sight of the viewer. The images 190, 200 are flat and the eyes 210, 220 may focus on the images by assuming a single accommodated state. Such 3-D display schemes rely on the human visual system to combine the images 190, 200 to provide a perception of depth and/or scale for the combined image.
[0155] It will be appreciated, however, that the human visual system is more complicated and providing a realistic perception of depth is more challenging. For example, many viewers of conventional “3-D” display systems find such systems to be uncomfortable or may not perceive a sense of depth at all. Without being limited by theory, it is believed that viewers of an object may perceive objects as being “three-dimensional” due to a combination of vergence and accommodation. Vergence movements (i.e., rotation of the eyes so that the pupils move toward or away from each other to converge the lines of sight of the eyes to fixate upon an object) of the two eyes relative to each other are closely associated with focusing (or “accommodation”) of the lenses and pupils of the eyes. Under normal conditions, changing the focus of the lenses of the eyes (or accommodating the eyes) from one object to another object at a different distance will automatically cause a matching change in vergence to the same distance, under a relationship known as the “accommodation-vergence reflex”. Likewise, under normal conditions, a change in vergence will trigger a matching change in accommodation, with changes in lens shape and pupil size. As noted herein, many stereoscopic or “3-D” display systems display a scene using slightly different presentations (and, so, slightly different images) to each eye such that a three-dimensional perspective is perceived by the human visual system. Such systems are uncomfortable for many viewers, however, since they, among other things, simply provide a different presentation of a scene; the eyes view all the image information at a single accommodated state, even image information for objects at different depths. This, however, which works against the “accommodation-vergence reflex.” Display systems that provide a better match between accommodation and vergence may form more realistic and comfortable simulations of three-dimensional imagery, which may facilitate users wearing the displays for longer durations.
……
……
……
