Microsoft Patent | Reduced bandwidth holographic near-eye display
Patent: Reduced bandwidth holographic near-eye display
Drawings: Click to check drawins
Publication Number: 20180188688
Publication Date: 20180705
Applicants: Microsoft Technology Licensing
Assignee: Microsoft Technology Licensing
Abstract
Examples are disclosed that relate to holographic near-eye display systems. One example provides a near-eye display device, comprising a diverging light source, an image producing dynamic digital hologram panel configured to receive light from the diverging light source and form an image. The near-eye display device also includes and a combiner comprising a holographic optical element positioned to receive light from the dynamic digital hologram panel and to redirect the light toward an eyebox, the holographic optical element being positioned between the eyebox and a view of an external environment to combine a view of the image formed by the dynamic digital hologram panel and the view of the external environment.
Claims
1. A near-eye display device, comprising: a diverging light source (DLS); an image-producing Dynamic Digital Hologram (DDH) panel configured to receive light from the DLS and form an image; and a combiner comprising a holographic optical element (HOE) positioned to receive light from the image-producing DDH panel and to redirect the light toward an eyebox, the HOE being positioned between the eyebox and a view of an external environment to combine a view of the image formed by the DDH panel and the view of the external environment.
2. The near-eye display device of claim 1, further comprising a controller in communication with the DDH panel, the controller being configured to control the DDH panel to control a focal depth of each pixel in the image.
3. The near-eye display device of claim 2, wherein the controller is configured to control the DDH panel to correct for aberrations in one or more of the HOE and a viewer's eye.
4. The near-eye display device of claim 3, wherein the controller is configured to control the DDH panel to correct for aberrations by calculating a Computer Generated Hologram (CGH) as an integration of sub-CGHs.
5. The near-eye display device of claim 4, wherein the controller is configured to control the DDH panel to compute the sub-CGHs by tracing a bundle of rays for each point in the image from an eye to the DDH panel, and from the DLS to the DDH panel, and to compute an associated phase delay.
6. The near-eye display device of claim 4, wherein the controller is configured to control the DDH panel to compute the sub-CGH to produce a wavefront based on an optical aberration model for the optical system.
7. The near-eye display device of claim 6, further comprising a camera, and wherein the controller is configured to select model parameters of the optical aberration model based on observing the wavefront via the camera, and adjust the model parameters to reduce a spot size.
8. The near-eye display device of claim 4, wherein the controller is configured to select the phase of each sub-CGH such that the phase is within a threshold consistency over the image between sub-CGHs.
9. The near-eye display device of claim 3, wherein the controller is configured to control the DDH panel to correct for aberrations individually for each color channel in the display device.
10. The near-eye display device of claim 3, wherein the controller is configured to control the DDH panel to correct for one or more of focus and aberrations in a portion of the image near a tracked position of a fovea and not in another portion of the image.
11. The near-eye display device of claim 10, wherein the controller is configured to control the DDH panel to compute a CGH as an inverse fast Fourier transform of a product of a fast Fourier transform of the image multiplied with a fast Fourier transform of a phase function that corrects for aberrations and focuses light at a depth in the portion of the image near the tracked position of the fovea.
12. The near-eye display device of claim 2, wherein the DDH panel is controllable to compute a CGH based on a separable convolution or a fast Fourier transform convolution, and wherein the phase modulating panel is further controllable to compute the hologram by using a larger area of the CGH to calculate portions of the image near the tracked position of the fovea, and a smaller area of the CGH in other portions of the image.
13. The near-eye display device of claim 2, wherein the controller is configured to shift an exit pupil by electrically switching between holograms in the HOE.
14. The near-eye display device of claim 2, wherein the controller is configured to shift an exit pupil via an HOE sensitive to a property of incoming light and configured to steer light to a different exit pupil position based upon the property.
15. The near-eye display device of claim 2, wherein the controller is configured to control the DDH panel to correct for aberrations based on a position of an exit pupil.
16. The near-eye display device of claim 1, wherein the DDH panel is positioned and oriented to provide an even distribution of resolution over a field of view of the display device.
17. A near-eye display device, comprising: a DLS; a DDH panel configured to receive light from the DLS, form an image from the light, and redirect the light; a combiner comprising a HOE positioned to receive light from the DDH panel and to redirect the light toward an eyebox, the HOE being positioned between the eyebox and a view of an external environment to combine a view of the image formed by the DDH panel and the view of the external environment; and a controller configured to control the DDH panel to produce the image and to pre-distort a wavefront produced by the DDH panel to compensate for aberrations of the HOE or other integrated optics.
18. The near-eye display device of claim 17, wherein the controller is configured to control the DDH panel to pre-distort the wavefront to compensate for aberrations in a sub-portion of the image based upon eye tracking data.
19. A near-eye display device, comprising: a DLS; a DDH panel configured to receive light from the DLS, form an image from the light, and redirect the light; a combiner comprising a HOE positioned to receive light from the DDH panel and to redirect the light toward an eyebox, the HOE being positioned between the eyebox and a view of an external environment to combine a view of the image formed by the DDH panel and the view of the external environment; and a controller configured to control the DDH panel to produce the image and to pre-distort a wavefront produced by the DDH panel to compensate for optical characteristics of a user's eye.
20. The near-eye display device of claim 19, wherein the controller is configured to control the DDH panel to pre-distort the wavefront to compensate for optical characteristics of the user's eye in a sub-portion of the image based upon eye tracking data.
Description
BACKGROUND
[0001] A near-eye display (NED) device may be implemented as a head-mounted display system that allows users to view images generated by holograms as an immersive experience. Images may appear to be on a single plane in the field-of-view (FOV) of the user, on multiple planes or form a truly three-dimensional object. Images displayed or projected to the user's eye by a NED device may be produced in various manners. For example, some NED devices may utilize (a) a plurality of display panels positioned within a user's FOV, (b) waveguides to deliver to a user's eyes imagery produced elsewhere in the system (c) or a partially reflective, partially transmissive element that relays the image into the user's eye.
SUMMARY
[0002] Examples are disclosed that relate to holographic NED systems. One example provides a NED device comprising a diverging light source (DLS) and a dynamic digital hologram (DDH) panel. The DDH panel is configured to receive light from the DLS and form an image. The NED device also includes a combiner comprising a holographic optical element (HOE) positioned to receive light from the DDH panel and to redirect the light toward an eyebox, the HOE being positioned between the eyebox and a view of an external environment to combine a view of the image formed by the DDH panel and the view of the external environment.
[0003] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] FIG. 1 schematically shows an example holographic display.
[0005] FIG. 2 schematically shows an example holographic display system comprising a relay element.
[0006] FIG. 3 schematically shows another example holographic display system comprising a relay element.
[0007] FIG. 4 shows an example head-mounted display system.
[0008] FIG. 5 shows a block diagram of an example computing system.
DETAILED DESCRIPTION
[0009] A DDH may be utilized in a display device, such as a near-eye display, to reproduce a light field. The term DDH as used herein signifies an optical element which modulates the phase of a wavefront instead of, or in addition to, an amplitude (intensity). Note that a DDH can be either transmissive or reflective and is the element that forms the image. A DDH is often referred to as a Spatial Light Modulator (SLM). The DDH may diffract light over a viewing zone determined by the pixel pitch of the DDH. For example, FIG. 1 schematically shows an example permissible viewing zone 102 and three-dimensional image zone 104 (which extends to infinity) from a holographic display utilizing a DDH 106. The resulting FOV is inversely proportional to the pixel pitch of the device. Therefore, the density of pixels on the DDH is proportional to the maximum FOV of the image, represented here by angle .theta.. When a large FOV is desirable, the overall resolution of the DDH may be relatively high. The required bandwidth of the DDH, here defined as the number of pixels displayed per second, may be very high, requiring the hardware for the DDH to be very complex, with impractical computation requirements and very high data transfer rates when used as a real-time display.
[0010] As one potential solution to such issues, a NED device may utilize an HOE that receives collimated light from a light source and outputs converging light toward a DDH before the light focuses at the pupil of a viewer's eye. The DDH may modulate the phase of the wavefront of the light, such that the light is diffracted to produce a set of rays corresponding to different points in the image. Because most of the diffractive power is in the HOE, this configuration allows the pixel pitch of the DDH to be considerably larger for the same FOV compared to when an HOE is not used. By increasing the pixel pitch, the DDH resolution reduces and the content of the Computer Generated Hologram (CGH) displayed on the DDH is considerably reduced. Thus, such a system may provide a relatively compact, high-resolution NED which may produce a light field and allow arbitrary focus and aberration correction, including aberrations of the viewer's eye.
[0011] However, in such an arrangement, the CGH on the DDH is directly viewed. As such, the FOV of the reconstructed image is limited by the size of the DDH. As DDHs are often implemented on silicon backplanes, they may cost significantly more with increasing size. Therefore, a device with a large FOV may be prohibitively expensive to produce. The use of multiple DDHs may help to address this issue as long as the gaps between the DDHs are smaller than the eye pupil, but also may increase costs. Additionally, such a system may use a liquid crystal on silicon (LCOS) panel as a spatial light modulator, which is opaque and thus prevents augmented reality (AR) or mixed-reality (MR) display without the use of a video feedthrough. Further, even a transparent DDH would diffract and scatter light coming from the outside world, thereby limiting its use as an AR or hMR display system.
[0012] As alternatives, additional optical elements may be used to relay the holographic image to the eye, for example, via a beamsplitting component and a waveguide. However, such optical systems may be limited to producing images at optical infinity, may lead to more constraints on FOV, and may further reduce image quality while driving up cost.
[0013] Accordingly, examples are disclosed herein that relate to a holographic NED comprising a DLS and a DDH, which together serve as a holographic projector (HP) to produce an image via one or more encoded holograms. The NED further includes a HOE configured to relay the image from the HP to a viewer's eye/pupil. This may allow the reduction of the DDH resolution (and thus bandwidth) as achieved by the previous system described above, while additionally providing a transparent image path to allow the holographically projected image to be displayed as superimposed over a view of the real world for AR uses. Further, in the disclosed examples, the physical size of the DDH does not limit the size of the viewing window, permitting a FOV of the real world that is greater than the angular extent of the DDH and the FOV of the virtual world. This reduces the system cost and allows more options in the configuration of the overall system. The FOV of the disclosed examples are instead determined by the size of the HOE, which may be less costly than a DDH. Further, the use of an HOE may allow the display device to be more compact, since it allows the illumination of the display optics in front of the eye at a steep angle, thus reducing an amount of space required between the eye and the display. The disclosed examples also allow for aberration correction, and may utilize hologram computation methods that may allow more fine-grained control of the wave front. Additionally, a dynamic element may be added to the HOE in some examples. Such a dynamic element may be used to steer the light into the user's pupil and/or dim part of the real world.
[0014] HOEs are currently used in combination with scanned beam projectors or conventional image panels, rather than a holographic image-producing panel. Such systems utilizing a conventional scanned beam projector or conventional image panel and a HOE are limited by optical aberrations, in that a wide FOV may be provided, but not without significant aberrations that are difficult to correct. Further, although HPs have been used to correct for aberrations in certain optical configurations, these configurations may utilize optical combiners such as mirrors rather than HOEs. Thus, many existing technologies either include a conventional projector (i.e. a non-HP) and an HOE, or a HP without an HOE.
[0015] In contrast, the examples disclosed herein utilize the combination of a HP and an HOE, while addressing issues that may arise with this combination. As described in more detail below, the examples disclosed herein may allow a wide FOV while providing for aberration correction using encoded holograms, and further may provide better see-through capability and allow smaller form factors than other see-through display device configurations.
[0016] FIG. 2 schematically shows an example optical system 200. FIG. 2 provides a schematic representation of an optical path, and is not intended to imply any particular geometric arrangements of optical components in such an optical system. In optical system 200, DLS 202 and DDH 204 together form a HP 206 configured to form a projected intermediate holographic image 208. The HP unit 206 may include any additional suitable refractive, reflective, diffractive, and/or amplitude-modulating components. The projector 206 may reduce zero or higher holographic orders of light, and may shape, correct aberrations of, and/or re-direct the light exiting the projector unit. The DLS 202 may comprise, for example, one or more laser diodes, a fiber optic light source, and/or any other suitable DLS. In some examples, additional optical components may be optionally included to further increase divergence of the light beam.
[0017] Any suitable phase and/or amplitude modulating devices may be used as the DDH 204, such as a phase or amplitude LCD or LCOS. Color images may be formed using time sequential, spatially multiplexed, or interleaved (Bayer-like) methods.
[0018] The beam 210 output from the HP 206 expands as it travels until it encounters the HOE 212. The HOE 212 redirects the beam to converge at the position of the viewer's eye 214. In some examples, the HOE 212 may provide additional magnification (causing the beam to be converge more rapidly that it was expanding from the projector), may be placed off-axis, and/or may be transparent, allowing an undistorted see-through view to the user for AR image display. Further, in some examples, the HOE 212 may duplicate the incoming projected beam to form replicated exit pupils. For example, a fan-out diffractive optical element (DOE) may be used, with the HP 206 at one conjugate, and form an array of spots at the other conjugate each corresponding to a different pupil position. The arrows shown at 216 represent the overall FOV of the depicted system.
[0019] In other examples, the HOE 212 may have a switchable state, such as those provided by a switchable hologram, to allow the location of the exit pupil(s) to be switched between predetermined positions based on a tracked eye position as detected via an eye tracking device. In further examples, the HOE 212 may be configured to change the position of the exit pupil(s) based upon the optical characteristics of the incoming beam, such as wavelength or incoming angle. For example, a controller 218 may be used to control the HP 206 to change its beam wavelength or incoming angle, thereby causing the HOE 212 to redirect the light differently. In such an example, the HOE 212 may utilize a wavelength or angularly selective Bragg hologram.
[0020] FIG. 3 shows another example optical system 300. In this example, DLS 302 and reflective DDH 304 are arranged off-axis, and illumination of the HOE 306 occurs at a relatively steep angle. Such a configuration may provide a more compact form factor of the NED. In this example, the HOE 306 acts as an off-axis point-to-point reflective HOE to transmit imaged light 308 to the eye 310. A controller 312 may control the DLS 302 and reflective DDH 304 for illumination of the HOE 306. It will be understood that the optical system 300 may have any additional suitable refractive, reflective, diffractive, and/or amplitude-modulating components.
[0021] Aberration correction may be challenging in a system utilizing a HP and a HOE as a combiner for AR display. Thus, in the example of FIG. 3, aberrations may be corrected by computing a CGH for the DDH to pre-distort the wavefront to compensate for any aberrations from the HOE. Further, the DDH also may be configured to control focal depth of the projected image on a pixel-by-pixel basis. Positioning and orientation of the DDH may be selected to provide a suitably even distribution of resolution over the FOV of the NED. For example, the DDH may be tilted such that the optical axes of the DDH and the HOE are aligned.
[0022] As mentioned above, in some examples, multiple exit pupils may be produced. In such examples, the aberrations may be optionally corrected only for an active pupil, e.g. the exit pupil nearest the viewer's eye based on eye tracking data. Where multiple exit pupils are used, the multiple exit pupils may be spaced apart sufficiently, and/or may be appropriately shifted such that only one is positioned to enter the viewer's eye.
[0023] In some examples, the position of the exit pupil(s) may be shifted according to a tracked eye position, e.g. by switching between different light sources incident on the holographic image-producing panel, by mechanically moving a light source to change a location and/or angle at which it is incident on the DDH, and/or by electrically switching between multiple holograms encoded in the HOE. Further, the exit pupil(s) also may be shifted by the HOE based on a property of incoming light. For example, light may be steered to a different exit pupil position based upon a wavelength, polarization, incident angle or other suitable property of the incoming light.
[0024] In some examples, varying aberration correction functions may be applied over portions of the image, while in other examples a same single aberration correction function may be applied over an entire image. Additionally, in some examples, correction of aberrations and/or focus may be performed in a sub-portion of the image, such as a sub-portion in which the viewer is looking or near a location of a tracked fovea position based on eye tracking data, while aberrations and/or focus are left uncorrected, or only partially corrected, in other portions of the image. The DDH 304 also may be used to correct for aberrations in the viewer's eye, such as astigmatism or higher order aberrations that may be difficult to correct with ordinary eyeglasses. In some examples, where a color NED is utilized, aberrations may be corrected individually for each color channel in the NED device.
[0025] Any suitable techniques may be used to calculate the CGH to be displayed on the DDH. As one example, each point in a perceived image may be traced backwards through the HOE to the DDH in the HP unit, and also traced forward from the DLS to the DDH. Based upon these tracings, a suitable phase delay function may be calculated as the total path length and integrated into the DDH. The process may be repeated for each point of the image to generate an arbitrary image with per-pixel depth and with arbitrary aberrations, with consideration of a bandwidth of the DDH. The phase function for generating each image point may be generated by characterizing the aberrations, which are integrated to form the final hologram. As an example, the aberrations may be characterized in terms of Zernike polynomials.
[0026] Further, in some examples, a CGH may be calculated as an integration of multiple sub-CGHs, where each sub-CGH may produce a wavefront that appears as a sub-region of the image. A phase of each sub-CGH may be selected such that the phase has a suitable level of consistency over the image between adjacent sub-CGHs (e.g. within a threshold consistency). In such examples, each sub-CGH may be computed to produce a wavefront based on a selected optical aberration model. The optical aberration model may include model parameters selected by observing the wavefront through the NED. e.g. via a camera, and adjusting the model parameters to reduce a spot size.
[0027] In some examples, to achieve faster computation of the CGH, aberrations may be corrected using a single aberration correction function and a single focal depth applied to the whole image. Such a method may result in less accurate aberration correction than the use of a greater number of correction functions and different focal depths at different pixels, but may be sufficient if the correction is accurate for the portion of the image that the viewer is looking at based upon eye tracking data. Such a method can be computed by convolving the image with a suitable aberration correction phase function. Fast convolution may be performed using fast Fourier transform-based convolution, or separable (e.g. linearly separable) direct convolution. As a more specific example, a CGH may be calculated, among other steps, as an inverse fast Fourier transform (FFT) of the product of FFT of the image multiplied with the FFT of a phase function that corrects for aberrations and focuses light at a depth in the portion of the image that is near the tracked position of the fovea. CGH calculation may further be simplified by using a larger area of the DDH to calculate portions of the image near the tracked position of the fovea, while using a smaller area of the DDH in other portions of the image.
[0028] Phase-only CGH may be optimized and/or calculated via a Gerchberg-Saxton-like algorithm using corresponding forward and reverse transforms and its variants like the Fienup algorithm. Phase-only holograms may also be computed using non-iterative direct encoding methods, such as double phase and error diffusion methods. To further improve CGH calculation speed, the CGH may be computed such that the image is calculated based on the fall-off of human visual acuity with angle. This may be accomplished by reducing the effective window or aperture size of the CGH (e.g. the region of the CGH that affects a single image point) to be suitably small, while achieving the target resolution over the FOV. For a sparse AR display, e.g. where virtual objects do not fill up the majority of the FOV, computation time may be further reduced by reducing the computation effort or not computing regions of the CGH that correspond to empty, non-virtual regions in the image.
[0029] In addition to the combiner being comprised of one or more HOEs, it will be understood that other diffractive, refractive or reflective elements may be employed as a combiner, including engineered metamaterials. Metamaterial layers can be used in a similar manner to HOEs, and may avoid chromatic aberrations that may be found in optically generated holographic optics.
[0030] Yet other examples of optics that may be used as a combiner may include digitally generated (computed) diffractive optics, and optical copies of digitally generated optics. The latter may include Bragg reflective-type holograms, which preferentially reflect different wavelengths of light corresponding to the Red, Green, and Blue wavelengths of the dynamic CGH, with a separate holographic fringe structure for each of the three (or more) wavelengths corresponding to the desired conjugate point for its wavelength, and may be substantially transparent for other wavelengths. It will be understood that the different Bragg structures corresponding to each wavelength may be in physically separate but adjacent layers recorded in a material such as photopolymer, or in a single layer in which the different Bragg structures are multiplexed via recording with multiple lasers or multiple exposures. Another example may include polarization holographic elements which employ a chiral dopant to obtain properties similar to Bragg holograms but with a polarization dependent property.
[0031] As described above, the examples disclosed herein may be incorporated into NED devices with relatively compact form factors, given the ability to utilize a relatively small DDH and compact combiner, and to illuminate the display optics off-axis at a steep angle. FIG. 4 shows an example head-mounted display device 400 having a relatively thin form factor, e.g. an eyeglass configuration. The head-mounted display device 400 includes one or more lenses 401 that form a part of a see-through display 402, such that images may be displayed via lenses 401, e.g. via projection onto lenses 401. The see-through display 402 may take any suitable form, such as that of a combiner comprising an HOE configured to receive a generated image from a HP unit 403 and direct the image towards a wearer's eye. The head-mounted display device 400 further includes one or more outward-facing image sensors 404 configured to acquire images of a background scene and/or physical space being viewed by a user, and may include one or more microphones 406 configured to detect sounds, such as voice commands from a user. The outward-facing image sensors 406 may include one or more depth sensors and/or one or more two-dimensional image sensors.
[0032] The head-mounted display device 400 may further comprise an eye tracking subsystem 408 configured to detect an eye gaze direction/fovea position of each eye of a user for use in hologram computation, as mentioned above. The eye tracking subsystem 408 may be configured to determine gaze directions of each of a user's eyes. For example, in the depicted embodiment, the eye tracking subsystem 408 includes one or more glint sources 410, such as infrared light sources, configured to cause a glint of light to reflect from each eyeball of a user, and one or more image sensors 412, such as inward-facing sensors, configured to capture an image of each eyeball of the user. Changes in the glints from the user's eyeballs and/or a location of a user's pupil as determined from image data gathered via the image sensor(s) 412 may be used to determine an eye gaze direction/fovea position. The eye tracking subsystem 408 may have any suitable number, type, and arrangement of light sources and image sensors.
[0033] The head-mounted display device 400 may further include one or more motion sensors 414 to detect movements of a user's head when the user is wearing the head-mounted display device 400. Motion data may be used, along with optional eye tracking data and/or outward-facing image data, for eye tracking and gaze detection, as well as for image stabilization to help correct for blur in images from the outward-facing image sensor(s) 406. The use of motion data may allow changes in gaze location to be tracked even if image data from outward-facing image sensor(s) 406 cannot be resolved. Likewise, motion sensors 414, as well as microphone(s) 406 and eye tracking subsystem 408 also may be employed as user input devices, such that a user may interact with the head-mounted display device 400 via gestures of the eye, neck and/or head, as well as via verbal commands. It will be understood that the sensors illustrated in FIG. 4 are shown for the purpose of example and are not intended to be limiting in any manner, as any other suitable sensors and/or combination of sensors may be utilized.
[0034] The head-mounted display device 400 further includes a controller 416 having a logic subsystem and a storage subsystem in communication with the various sensors, the holographic projector 403, and the eye tracking subsystem 408. The controller 416 is an example of the controller 218 or the controller 312 described above. The controller 416 may be configured, for example, to receive and analyze inputs from the sensors, to identify movements of a user, to track eye fovea positions and gaze directions, to generate images, and to compute holograms to pre-distort the wavefront for aberration correction, among other tasks.
[0035] The depicted head-mounted display device 400 is shown for the purpose of example, and is not meant to be limiting. The head-mounted display device 400 may include additional and/or alternative sensors, cameras, microphones, input devices, output devices, etc. than those shown without departing from the scope of this disclosure. As examples, other components may include speakers, accelerometers, gyroscopes, magnetometers, temperature sensors, touch sensors, biometric sensors, other image sensors, energy-storage components (e.g. battery), a communication facility, a Global Positioning System receiver, etc. Further, the physical configuration of a display device and its various sensors and subcomponents may take a variety of different forms without departing from the scope of this disclosure.
[0036] In some examples, the head-mounted display device 400 may include a low resolution peripheral display to increase the FOV. As another example, the holographic projector may form the image in a tracked foveal region while an alternate display provides a lower fidelity background image to other regions. A near-eye display device may also include a global or regional dimming or occluding device that allows a transparency of the device to be varied. A light engine for the holographic projector, such as an array of RGB lasers, may be incorporated into a remote unit that may reside in a separate location than the head-mounted display device, such as in a pocket, worn on another part of the body, etc., and connected to the glasses with a fiber optic cable.
[0037] In some embodiments, the methods and processes described herein may be tied to a computing system of one or more computing devices. In particular, such methods and processes may be implemented as a computer-application program or service, an application-programming interface (API), a library, and/or other computer-program product.
[0038] FIG. 5 schematically shows a non-limiting embodiment of a computing system 500 that can enact one or more of the methods and processes described above. For example, computing system 500 is representative of controller 218, 312, or 416. Computing system 500 is shown in simplified form. Computing system 500 may take the form of one or more personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices.
[0039] Computing system 500 includes a logic subsystem 502 and a storage subsystem 504. Computing system 500 may optionally include a display subsystem 506, input subsystem 508, communication subsystem 510, and/or other components not shown in FIG. 5.
[0040] Logic subsystem 502 includes one or more physical devices configured to execute instructions. For example, the logic subsystem 502 may be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result.
[0041] The logic subsystem 502 may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic subsystem 502 may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. Processors of the logic subsystem 502 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and/or distributed processing. Individual components of the logic subsystem 502 optionally may be distributed among two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the logic subsystem 502 may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration.
[0042] Storage subsystem 504 includes one or more physical devices configured to hold instructions executable by the logic subsystem 502 to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage subsystem 504 may be transformed--e.g., to hold different data.
[0043] Storage subsystem 504 may include removable and/or built-in devices. Storage subsystem 504 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storage subsystem 504 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices.
[0044] It will be appreciated that storage subsystem 504 includes one or more physical devices. However, aspects of the instructions described herein alternatively may be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for a finite duration.
[0045] Aspects of logic subsystem 502 and storage subsystem 504 may be integrated together into one or more hardware-logic components. Such hardware-logic components may include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC/ASICs), program- and application-specific standard products (PSSP/ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.
[0046] When included, display subsystem 506 may be used to present a visual representation of data held by storage subsystem 504. This visual representation may take the form of a graphical user interface (GUI). As the herein described methods and processes change the data held by the storage subsystem 504, and thus transform the state of the storage subsystem 504, the state of display subsystem 506 may likewise be transformed to visually represent changes in the underlying data. Display subsystem 506 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with logic subsystem 502 and/or storage subsystem 504 in a shared enclosure, or such display devices may be peripheral display devices.
[0047] When included, input subsystem 508 may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may comprise or interface with selected natural user input (NUI) componentry. Such componentry may be integrated or peripheral, and the transduction and/or processing of input actions may be handled on- or off-board. Example NUI componentry may include a microphone for speech and/or voice recognition; an infrared, color, stereoscopic, and/or depth camera for machine vision and/or gesture recognition; a head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition; as well as electric-field sensing componentry for assessing brain activity.
[0048] When included, communication subsystem 510 may be configured to communicatively couple computing system 500 with one or more other computing devices. Communication subsystem 510 may include wired and/or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem may be configured for communication via a wireless telephone network, or a wired or wireless local- or wide-area network. In some embodiments, the communication subsystem may allow computing system 500 to send and/or receive messages to and/or from other devices via a network such as the Internet.
[0049] Another example provides near-eye display device comprising a DLS, an image-producing DDH panel configured to receive light from the diverging light source and form an image, and a combiner comprising a HOE positioned to receive light from the image-producing DDH panel and to redirect the light toward an eyebox, the HOE being positioned between the eyebox and a view of an external environment to combine a view of the image formed by the DDH panel and the view of the external environment. The near-eye display device may additionally or alternatively include a controller in communication with the DDH panel, the controller being configured to control the DDH panel to control a focal depth of each pixel in the image. The controller may be additionally or alternatively configured to control the DDH panel to correct for aberrations in one or more of the holographic optical element and a viewer's eye. The controller may be additionally or alternatively configured to control the DDH panel to correct for aberrations by calculating a CGH as an integration of sub-holograms. The controller may be additionally or alternatively configured to control the DDH panel to compute the sub-CGH by tracing a bundle of rays for each point in the image from an eye to the DDH panel, and from the DLS to the DDH panel, and to compute an associated phase delay. The controller may be additionally or alternatively configured to control the DDH panel to compute the sub-CGH to produce a wavefront based on an optical aberration model for the optical system. The near-eye display device may additionally or alternatively include a camera, wherein the controller is configured to select model parameters of the optical aberration model based on observing the wavefront via one or more of an eye and the camera, and adjust the model parameters to reduce a spot size. The controller may be additionally or alternatively configured to select the phase of each sub-CGH such that the phase is within a threshold consistency over the image between sub-CGHs. The controller may be additionally or alternatively configured to control the DDH panel to correct for aberrations individually for each color channel in the display device. The controller may be additionally or alternatively configured to control the DDH panel to correct for one or more of focus and aberrations in a portion of the image near a tracked position of a fovea and not in another portion of the image. The controller may be additionally or alternatively configured to control the DDH panel to compute a CGH as an inverse fast Fourier transform of a product of a fast Fourier transform of the image multiplied with a fast Fourier transform of a phase function that corrects for aberrations and focuses light at a depth in the portion of the image near the tracked position of the fovea. The DDH panel may be additionally or alternatively controllable to compute a CGH based on a separable convolution or a fast Fourier transform convolution, and wherein the phase modulating panel is further controllable to compute the hologram by using a larger area of the CGH to calculate portions of the image near the tracked position of the fovea, and a smaller area of the CGH in other portions of the image. The controller may be additionally or alternatively configured to shift an exit pupil by electrically switching between holograms in the holographic optical element. The controller may be additionally or alternatively configured to shift an exit pupil via an HOE sensitive to a property of incoming light and configured to steer light to a different exit pupil position based upon the property. The controller may be additionally or alternatively configured to control the DDH panel to correct for aberrations based on a position of an exit pupil. The DDH panel may be additionally or alternatively positioned and oriented to provide an even distribution of resolution over a field of view of the display device.
[0050] Another example provides for a near-eye display device, comprising a DLS, a DDH panel configured to receive light from the DLS, form an image from the light, and redirect the light, a combiner comprising a HOE positioned to receive light from the DDH panel and to redirect the light toward an eyebox, the HOE being positioned between the eyebox and a view of an external environment to combine a view of the image formed by the DDH panel and the view of the external environment, and a controller configured to control the DDH panel to produce the image and to pre-distort a wavefront produced by the DDH panel to compensate for aberrations of the HOE or other integrated optics. The controller may be additionally or alternatively configured to control the DDH panel to pre-distort the wavefront to compensate for aberrations in a sub-portion of the image based upon eye tracking data.
[0051] Another example provides a near-eye display device, comprising a DLS, a DDH panel configured to receive light from the DLS, form an image from the light, and redirect the light, a combiner comprising a HOE positioned to receive light from the DDH panel and to redirect the light toward an eyebox, the HOE being positioned between the eyebox and a view of an external environment to combine a view of the image formed by the DDH panel and the view of the external environment, and a controller configured to control the DDH panel to produce the image and to pre-distort a wavefront produced by the DDH panel to compensate for optical characteristics of a user's eye. The controller may be additionally or alternatively configured to control the DDH panel to pre-distort the wavefront to compensate for optical characteristics of the user's eye in a sub-portion of the image based upon eye tracking data.
[0052] It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed.
[0053] The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.
