Microsoft Patent | Backlight for viewing three-dimensional images from a display from variable viewing angles
Patent: Backlight for viewing three-dimensional images from a display from variable viewing angles
Drawings: Click to check drawins
Publication Number: 20140098010
Publication Date: 20140410
Applicants: Microsoft Corporation
Assignee: Microsoft Corporation
Abstract
A display system produces three-dimensional images. The display system includes a waveguide, and a light source that injects light into the waveguide. A switchable grating allows individual positions in the grating in a two-dimensional array to be turned on and off. A controller configured to spatially modulate the switchable grating so as to control where light exits the waveguide. An eye tracking module tracks eye position of a viewer. The controller uses the eye position to control switching of the grating.
Claims
1. A display system that produces three-dimensional images, comprising: a waveguide; a light source configured to inject light into the waveguide; a switchable grating; and a controller configured to spatially modulate the switchable grating so as to control where light exits the waveguide.
2. The display system of claim 1, wherein the controller is further configured to modulate an angle of injection of the light into the waveguide.
3. The display system of claim 1, further comprising an eye tracking module providing an output indicative of an eye position of a viewer.
4. The display system of claim 3, wherein the controller is further configured to control the switchable grating according to the eye position of the viewer.
5. The display system of claim 3, wherein the controller is further configured to modulate an angle of injection of light into the waveguide according to the eye position of the viewer.
6. The display system of claim 1, wherein the switchable grating comprises, for each of a plurality of positions in the switchable grating: a reservoir including a fluid; a first conductor and a second conductor; and a gap between the first and second conductor and contiguous with a face of the waveguide. wherein the fluid is in the gap or in the reservoir according to voltage applied to the first and second conductors.
7. The display system of claim 6, wherein the grating and the waveguide are located on a first side of the gap.
8. The display system of claim 6, wherein the grating and the waveguide are on opposite sides of the gap.
9. The display system of claim 6, wherein the controller is further configured to spatially modulate the switchable grating according to a received eye position.
10. The display system of claim 9, wherein the controller is further configured to control a scanning mirror to modulate an angle of injection of the light into the waveguide.
11. A method for controlling a display system to produce three-dimensional images, the display system including a waveguide and a switchable grating, the method comprising: receiving image data to be displayed, the image data including data for left and right channels; controlling the switchable grating according to data to be displayed for each of the left and right channels.
12. The method of claim 11, further comprising modulating an angle of injection of light into the waveguide.
13. The method of claim 11, further comprising receiving data indicative of an eye position of a viewer.
14. The method of claim 13, further comprising controlling the switchable grating according to the eye position of the viewer.
15. The method of claim 13, further comprising modulating an angle of injection of light into the waveguide according to the eye position of the viewer.
16. The method of claim 11, wherein controlling the switchable grating comprises applying a voltage to conductors to draw fluid into a gap adjacent the waveguide.
17. The method of claim 12, wherein modulating the angle comprises controlling a scanning mirror that directs the light into the waveguide.
18. A computer program product, comprising: a computer readable storage medium; computer program instructions stored on the computer readable storage medium which, when processed by a computer processor, instruct the computer processor to perform a process of controlling a display system to produce three-dimensional images, the display system including a waveguide and a switchable grating, the process comprising: receiving image data to be displayed, the image data including data for left and right channels; controlling the switchable grating according to data to be displayed for each of the left and right channels.
19. The computer program product of claim 18, wherein the process further comprises receiving data indicative of an eye position of a viewer.
20. The computer program product of claim 19, further comprising controlling the switchable grating according to the eye position of the viewer.
Description
BACKGROUND
[0001] Three-dimensional images, such as those seen in motion pictures, typically are viewable due to spectacles that are worn by each viewer. Three-dimensional displays produce a left eye and right eye view of a three-dimensional image. The viewer wears spectacles which block light to one eye when the view intended for the other eye is displayed.
SUMMARY
[0002] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is intended neither to identify key features or essential features of the claimed subject matter, nor to be used to limit the scope of the claimed subject matter.
[0003] The use of spectacles can be avoided by providing a display with a waveguide and a switchable grating. The combination of waveguide and switchable grating concentrates rays into the left eye, then the right eye, in rapid alternating succession to avoid flicker.
[0004] In one implementation, light from a laser is injected via a scanner into the edge of a slab waveguide. A spatial array of switchable gratings controllably ejects light from the waveguide. In one implementation, the grating can be made switchable by using an electric field to draw index-matching fluid into a gap between the grating and the waveguide.
[0005] Thus, to provide a three-dimensional display, the angle at which the laser beam is injected into a slab waveguide is controlled, and the diffractive efficiency of gratings across the surface of the waveguide is spatially modulated. The gratings may be switched on and off by using an electric field to draw up an index matching fluid between the grating and the waveguide. The fraction of grating introduced at each pixel may be used to create an image, or the guide may be used as a backlight for a liquid crystal display.
[0006] By using a scanning mirror to set the launch angle of a ray at the edge of the waveguide and by switching on a grating at a desired position for the ray to emerge, one can control the position, angle and intensity of a ray anywhere on the surface of the device. This configuration allows a three-dimensional image to be time-multiplexed.
[0007] In the following description, reference is made to the accompanying drawings which form a part hereof, and in which are shown, by way of illustration, specific example implementations of this technique. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the disclosure.
DESCRIPTION OF THE DRAWINGS
[0008] FIG. 1 is a block diagram of an example operating environment for a three-dimensional display.
[0009] FIG. 2 is a diagram illustrating an example implementation of a display device.
[0010] FIG. 3 is a more detailed schematic diagram illustrating an example implementation of a display device.
[0011] FIG. 4 is a more detailed schematic diagram illustrating another example implementation of a display device.
[0012] FIG. 5 is a diagram of how light is directed at a viewer's eyes.
[0013] FIG. 6 is a diagram of a system incorporating eye tracking
[0014] FIG. 7 is a block diagram of an example computing device in which such a system can be implemented.
[0015] FIG. 8 is a flowchart describing a process for controlling the display system.
DETAILED DESCRIPTION
[0016] The following section provides an example operating environment in which a display system, including a combination of a waveguide and switchable grating, can be implemented.
[0017] Referring to FIG. 1, one or more sources 100 provide image data 102 to a display system 104 for display. The source can be any image generating appliance that generates image data for three-dimensional viewing. As an example, source 100 can include, but is not limited to, an application on a computing device, such as described below in connection with FIG. 7, that is presenting three-dimensional images. As another example, source 100 can include a playback device, such as a DVD player, which plays back video data from a storage medium to a display.
[0018] Image data 102 can include one or more images, including a sequence of images intended to be played back as motion video. Each image has data to be presented to both left and right eyes to provide a three dimensional viewing environment, and can be represented using two-channels of data, namely left and right.
[0019] The display system 104 includes a controller 106 that, given image data 102, controls both how light 114 from a laser 108 is injected into the wave guide 112, and how the grating 110 is switched, so that the images displayed to the user appear to have three-dimensions.
[0020] It should be understood that the controller 106 can be implemented on a computing device (such as described below in connection with FIG. 7) that is connected to a display, or can be incorporated into a display. The display system 104 can take a variety of forms, including but not limited to a display on a computing device, such as a mobile phone or laptop computer display, or a standalone display, such as a desktop display, wall-mounted display or table mounted display, or any other device connected to any of a variety of image generating appliances. A housing for the display can include the controller or the controller can be provided in a separate computing device connected to the display.
[0021] Referring now to FIG. 2, more details of the display device will now be described. The display device includes a waveguide 200, such as a slab waveguide, which can be made of glass or other material on which transistors can be formed. A scanning mirror 202 directs light 204 from a laser 210 into the waveguide 200. A switchable grating 206 controls the reflection of the light 204 to cause the light to exit the waveguide. In one implementation, a grating can be embossed on the waveguide. The grating is structured to diffract light in one direction at any point in time. The switchable grating is a grating comprised on multiple portions 212, each of which is individually addressable at a position along the waveguide and can be either on or off. When the grating is off at a position along the waveguide, the light remains guided through the waveguide at that position. When the grating is on at a position along the waveguide, the light is diffracted out of the waveguide at that position. Thus, the waveguide and the switchable grating combine to provide a linearly addressable device that can control where light is diffracted out of the waveguide. An array of such devices can be used to provide a two-dimensional display.
[0022] An example implementation of the display device will be described in more detail in connection with FIGS. 3-4.
[0023] The switchable grating, in one implementation, uses a technique of moving fluid between plates of a capacitor by applying a voltage to those plates. In another implementation, a technique of moving fluid called "electro-wetting" can be used. An implementation of electrowetting using a dye is described in M. G. Pollack, R. B. Fair and A. D. Shenderov, "Electrowetting-based actuation of liquid droplets for microfluidic applications", Applied Physics Letters, Vol. 77, pp. 1725-6, 2000. Instead of using a dye, a change in refractive index can be brought about by the presence or absence of the fluid which causes light to be diffracted by the grating or not.
[0024] As shown in FIG. 3, such a switchable grating includes, for each addressable position in the grating, a reservoir 300 that contains a fluid 302. On the left, the fluid 302 is in the reservoir 300; on the right, the fluid 302 is in the gap. The fluid is index-matching in that the fluid has a refractive index that matches the refractive index of the waveguide. The fluid can be, for example, a material commonly used is such electro-wetting applications, such as an oil, water, or very high index fluid. In the absence of the fluid, the gap may be filled with air, or an oil or liquid with a different refractive index from that of the fluid . An example fluid is GE/Bayer SF1555 silicone which as a refractive index of 1.47. A gap is provided between a grating 310 and waveguide 312. The grating can be an ultraviolet light curing acrylic which is embossed using a mold on the waveguide (which can be glass) and illuminated with ultraviolet light before the mold is removed. The grating is structured to diffract a guided ray in one direction at any point in time. A scanning mirror 314 injects light into the edge of the waveguide, and controls the angle of injection. Transparent conductors 304 and 306 are positioned on either side of the gap. As an example, the conductors can be made of indium tin oxide, with a thickness of about ten microns and a width of one millimeter.
[0025] The conductors 304 and 306 are connected to a voltage source 308. According to a voltage applied by the voltage source to the transparent conductors, the grating is switched on or off.
[0026] FIG. 3 illustrates one implementation in which a grating is turned on by drawing a fluid into the gap. In particular, when the grating is off, the fluid is retained outside of the gap in the reservoir. The light is then guided down the wave guide. When the grating is on, the fluid is drawn into the gap. Light exits the waveguide, passes through the fluid and reflects off the grating, and is directed out of the display, as indicated at 320.
[0027] In FIG. 4, the switchable grating includes, for each addressable position in the grating, a reservoir 400 that contains a fluid 402. Transparent conductors 404 and 406 are positioned on either side of a gap. A grating 410 and waveguide 412 are positioned on one side of the gap. A scanning mirror 414 injects light into the edge of the waveguide, and controls the angle of injection. The conductors 404 and 406 are connected to a voltage source 408.
[0028] FIG. 4 illustrates another implementation in which a grating is turned on by drawing fluid out of the gap. In this implementation, the fluid is index-matching in that the fluid has a refractive index that matches the refractive index of the grating. In particular, when the grating is off, the fluid is drawn into the gap. The light can exit the waveguide, passing through the grating and the fluid, and reflecting back in to the waveguide. The light is guided down the wave guide. When the grating is on, the fluid is drawn out of the gap. Light exits the waveguide, and reflects off the grating, to be directed out of the display, as indicated at 420.
[0029] In these implementations, the scanning mirror (e.g., 314, 414) allows a launch angle of a ray of light at the edge of the waveguide to be set. The switchable grating allows the position to be set where the ray is to emerge. Thus, the scanning mirror and switchable grating allow control of the position, angle and intensity of a ray of light anywhere on the surface of this device. Controlling these parameters allows a three-dimensional image to be time-multiplexed on this device.
[0030] Accordingly, given image data that includes data for left and right images, the scanning mirror and switchable grating can be controlled so as to present an image. The electrowetting-based switchable grating can allow a change in state of a grating every 1 to 2 milliseconds. To reduce the amount of data used to display the image, it is possible to modulate only rays likely to enter the pupil of each eye of a viewer 504, such as illustrated in FIGS. 5 at 500 and 502. In particular, if rays leave the light-guide in parallel, only rays 500 and 502 from a small region of the waveguide will enter a pupil of a viewer 504 at any instant. If the gratings in this small region are on at that instant, then the viewer will see the corresponding image. The intensity of the ray can be modulated as its angle is scanned, and the image will appear virtual in the sense that rays leaving the surface of the display at any instant will be parallel. Alternatively, the laser can be kept at constant intensity and extent to which each grating is switched on when it is visible to the eye can be varied. Thus, an image can be displayed that appears to be three-dimensional.
[0031] To modulate only those rays likely to be directed at a viewer's eye, the position of the viewer's eye can be determined using eye tracking technology. As shown in FIG. 6, an eye tracking module 600 processes images 606 of the viewer to provide data 602 indicating the eye position of a viewer 604 to the controller 106. Given the knowledge of the eye position, the launch angle, intensity and grating can be controlled so as to direct light 608 to the desired eye position. This control can be provided for each channel (left and right) of data for each eye (left and right). Note that only that part of the image for which three-dimensional simulation is desired is controlled in this manner. The remainder of the image (such as a background), can be displayed in a conventional manner.
[0032] A flowchart describing how the controller can present a sequence of images according to a tracked viewer position will now be described in connection with FIG. 8.
[0033] The controller receives 800 the next image data to be displayed. This image may include left and right channels. Any background image data that is simply displayed without reference to the eye position is displayed 802. The viewer's current position is then obtained 804. Using the viewer's position, the first channel of data for one eye is displayed 806. Then the second channel of data for the other eye is displayed 808. This process can be repeated for multiple viewers, as indicated at 810, and for multiple images, as indicated at 812.
[0034] In another implementation, it can be useful to have a display where the image appears to come neither from the display nor from infinitely far behind the display, but somewhere in between. This kind of display can be provided by using a video hologram, which is described in PCT Publication WO9900993. In particular, a hologram can be synthesized by displaying a series of narrow-angle holograms on a liquid crystal display and illuminating it with rays of light which are parallel but in a slightly different direction for each narrow-angle hologram. By placing an LCD in front of the waveguide described herein, and displaying a narrow angle hologram on the LCD, so as to vary the distance to the point from which any part of the image appears to originate, such a display can be created.
[0035] Having now described an example implementation, a computing environment in which such a system is designed to operate will now be described. Such a computing environment can be used to implement, the controller for the display, and/or the eyetracking module, and/or the source of the image data.
[0036] The following description is intended to provide a brief, general description of a suitable computing environment in which this system can be implemented. The system can be implemented with numerous general purpose or special purpose computing hardware configurations. Examples of well known computing devices that may be suitable include, but are not limited to, personal computers, server computers, hand-held or laptop devices (for example, media players, notebook computers, cellular phones, personal data assistants, voice recorders), multiprocessor systems, microprocessor-based systems, set top boxes, game consoles, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
[0037] FIG. 7 illustrates an example of a suitable computing system environment. The computing system environment is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of such a computing environment. Neither should the computing environment be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the example operating environment.
[0038] With reference to FIG. 7, an example computing environment includes a computing machine, such as computing machine 700. In its most basic configuration, computing machine 700 typically includes at least one processing unit 702 and memory 704. The computing device may include multiple processing units and/or additional co-processing units such as graphics processing unit 720. Depending on the exact configuration and type of computing device, memory 704 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. This most basic configuration is illustrated in FIG. 7 by dashed line 706. Additionally, computing machine 700 may also have additional features/functionality. For example, computing machine 700 may also include additional storage (removable and/or non-removable) including, but not limited to, magnetic or optical disks or tape. Such additional storage is illustrated in FIG. 7 by removable storage 708 and non-removable storage 710. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer program instructions, data structures, program modules or other data. Memory 704, removable storage 708 and non-removable storage 710 are all examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by computing machine 700. Any such computer storage media may be part of computing machine 700.
[0039] Computing machine 700 may also contain communications connection(s) 712 that allow the device to communicate with other devices. Communications connection(s) 712 is an example of communication media. Communication media typically carries computer program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal, thereby changing the configuration or state of the receiving device of the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.
[0040] Computing machine 700 may have various input device(s) 714 such as a keyboard, mouse, pen, camera, touch input device, and so on. Output device(s) 716 such as a display, speakers, a printer, and so on may also be included. All of these devices are well known in the art and need not be discussed at length here.
[0041] The input and output devices can be part of a natural user interface (NUI). NUI may be defined as any interface technology that enables a user to interact with a device in a "natural" manner, free from artificial constraints imposed by input devices such as mice, keyboards, remote controls, and the like.
[0042] Examples of NUI methods include those relying on speech recognition, touch and stylus recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, and machine intelligence. Example categories of NUI technologies include, but are not limited to, touch sensitive displays, voice and speech recognition, intention and goal understanding, motion gesture detection using depth cameras (such as stereoscopic camera systems, infrared camera systems, RGB camera systems and combinations of these), motion gesture detection using accelerometers, gyroscopes, facial recognition, 3D displays, head, eye , and gaze tracking, immersive augmented reality and virtual reality systems, all of which provide a more natural interface, as well as technologies for sensing brain activity using electric field sensing electrodes (EEG and related methods).
[0043] Such a display system may be implemented in the general context of software, including computer-executable instructions and/or computer-interpreted instructions, such as program modules, being processed by a computing machine. Generally, program modules include routines, programs, objects, components, data structures, and so on, that, when processed by a processing unit, instruct the processing unit to perform particular tasks or implement particular abstract data types. This system may be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
[0044] Alternatively, or in addition, the functionally described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
[0045] The terms "article of manufacture", "process", "machine" and "composition of matter" in the preambles of the appended claims are intended to limit the claims to subject matter deemed to fall within the scope of patentable subject matter defined by the use of these terms in 35 U.S.C. .sctn.101.
[0046] Any or all of the aforementioned alternate embodiments described herein may be used in any combination desired to form additional hybrid embodiments. It should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific implementations described above. The specific implementations described above are disclosed as examples only.
