Samsung Patent | Hologram profile optimization method, hologram profile generation device, and holographic display device to which hologram profile optimization method is applied

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certain embodiments of the disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a schematic diagram of a holographic display device using a 2-f system;

FIG. 2 is a schematic diagram of a holographic display device, according to an embodiment;

FIGS. 3 and 4 are diagrams for describing a time multiplexing method;

FIG. 5 is a diagram for describing a viewing window formed by a holographic display device including a binary phase spatial light modulator;

FIG. 6 is a block diagram illustrating a hardware configuration of a hologram profile generation device, according to an embodiment;

FIG. 7 is a schematic diagram illustrating the holographic display device of FIG. 2 to explain a hologram profile optimization method;

FIG. 8 is a flowchart of a hologram profile optimization method;

FIGS. 9A and 9B are a sign function and a derivative of the sign function, respectively, according to embodiments;

FIG. 10A and FIG. 10B are a sign function and a derivative of the sign function according to embodiments;

FIG. 11A and FIG. 11B are a sign function and a derivative of the sign function according to embodiments; and

FIGS. 12 and 13 are diagrams for explaining the effects of a hologram profile optimization method and a time multiplexing method.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. In this regard, embodiments may have different forms and should not be construed as being limited to the descriptions set forth herein. Accordingly, embodiments are merely described below, by referring to the figures, to explain aspects. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. Expressions, such as “at least one of,” when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list.

Hereinafter, a retina projection display device will be described in detail with reference to the accompanying drawings. Like reference numerals in the drawings denote like components, and sizes of components in the drawings may be exaggerated for convenience of explanation.

FIG. 1 is a schematic diagram of a holographic display device 1 using a 2-f system.

Referring to FIG. 1, the holographic display device 1 using a 2-f system may include a light source unit LU, a spatial light modulator SLM, a relay lens RL, a Fourier filter FT, and a processor PR. In this case, the relay lens RL and the Fourier filter FT may constitute an optical noise filtering, and may be referred to as the 2-f system. The spatial light modulator SLM may receive a hologram profile from the processor PR, receive light from the light source unit LU including a light source LS and a collimator lens CL, and reproduce a holographic image. When the holographic image output from the spatial light modulator SLM passes through the relay lens RL, a Fourier plane may be formed on a path on an emission surface side of the relay lens RL. The holographic image may be separated into an area in which noise appears on the Fourier plane, and an area in which a signal appears. In this case, the noise may include direct current (DC) term noise, high-order diffraction term noise, and conjugate image noise, and when these pieces of noise are removed from the holographic image, only the signal image IMG may be obtained. The Fourier filter FT may remove noise by covering a region where there is noise on the Fourier surface.

Because the 2-f system includes the relay lens RL for noise removal, an additional optical path that is twice a focal length of the relay lens RL may be required. Accordingly, in the holographic display device 1 designed by using this feature, the 2-f system may occupy a large portion of the volume of the holographic display device 1, and thus, the size of the entire system may be increased.

FIG. 2 is a schematic diagram of a holographic display device 10, according to an example embodiment. FIGS. 3 and 4 are diagrams for describing a time multiplexing method.

Referring to FIG. 2, the holographic display device 10 according to an example embodiment of the disclosure may include the light source unit LU, the spatial light modulator SLM, and the processor PR. The holographic display device 10 illustrated in FIG. 2 may remove noise by using a temporal multiplexing method and the hologram profile optimization method to be described below, instead of removing noise by using an optical noise filtering system (that is, the 2-f system). In other words, because the 2-f system is omitted from the holographic display device 10 of the disclosure illustrated in FIG. 2, the size of the entire system may be reduced.

The light source unit LU according to an example embodiment may include the light source LS and the collimator lens CL. The light source LS may include a laser diode to provide light having high coherence to the spatial light modulator SLM. However, it may be also possible to use a light-emitting diode as the light source LS because light provided by the light source LS may be sufficiently diffracted and modulated by the spatial light modulator SLM when the light provided by the light source LS has a level of coherence equal to or higher than a certain level of spatial coherence. In addition, the light source LS may have a light source array of red color, green color, and blue color, and may also implement a color hologram image by red/green/blue (RGB) time division driving. For example, the light source LS may include a plurality of laser diodes or an array of light-emitting diodes. In addition, the light source LS may use any other light source as long as the light source emits light having spatial coherence in addition to the laser diodes and the light-emitting diodes.

The light source unit LU may illuminate collimated parallel light. For example, the light source unit LU may collimate light emitted from the light source LS into parallel light by using the collimator lens CL.

The spatial light modulator SLM may form a hologram pattern, on a light modulation surface, according to the hologram profile provided by the processor PR. Light incident on the spatial light modulator SLM may be converted into the holographic image by the holographic pattern. In an embodiment of FIG. 2, although the spatial light modulator SLM is illustrated as a transmissive spatial light modulator, the spatial light modulator SLM may also be implemented as a reflective spatial light modulator.

The spatial light modulator SLM according to an example embodiment may be a binary phase spatial light modulator SLM, which is implemented such that a phase signal of each pixel is represented only by two types of signals, that is, 0 and 1. The binary phase spatial light modulator SLM may not express a complex number, and may express only real numbers by using brightness adjustment of a light source corresponding to each pixel.

For example, the binary phase spatial light modulator SLM may be implemented as a digital micromirror device (DMD), a liquid crystal on silicon (LCoS), a ferroelectric liquid crystal on silicon (FLCoS), etc.

By controlling the intensity of a light source in each pixel by using the light source LS and the binary phase spatial light modulator SLM, the holographic display device 10 may not output light for a pixel having a binary value of 0 in the hologram profile, but may output a holographic image by forming a holographic pattern in a form of outputting light for a pixel having a binary value of 1 in the hologram profile.

Due to this operation, the binary phase spatial optical modulator SLM may have an advantage of a fast driving speed because an amount of holographic profile processing computations is significantly reduced, but the binary phase spatial optical modulator SLM may have a disadvantage of low reproducibility of the holographic image, compared to the target image (or, the original image). Accordingly, the holographic display device 10 implemented as the binary phase spatial light modulator SLM may be driven by applying the time multiplexing method thereto.

Referring to FIGS. 2 and 3, the processor PR may perform a plurality of random phase modulations on one input hologram profile. In this case, the hologram profile may correspond to a hologram profile (or a second hologram profile) to be updated by a hologram profile optimization method to be described below.

The binary phase spatial light modulator SLM may diffract light according to the modulated hologram profile. Light emitted by the binary phase spatial light modulator SLM may be randomly scattered according to a random phase. A portion of the randomly scattered light may not pass through the pupil PU of a user's eye E.

Accordingly, some area of the holographic image may look dark as a plurality of holographic images illustrated on the right side of FIG. 4, or speckle noise recognized as small dots may be generated in the holographic image.

The time multiplexing method may include an operation of generating a plurality of holographic images by performing a plurality of random phase modulations on one hologram profile (or the second hologram profile), and an operation of polymerizing the generated plurality of holographic images into one holographic image.

Because the speckle is randomly generated by random phase modulation, positions of a speckle included in a first holographic image, a speckle included in a second holographic image, a speckle included in a third holographic image, through an n^th holographic image may be different. Accordingly, when one holographic image is generated by polymerizing the first through n^th holographic images, distribution of the speckles included in the polymerized holographic image may be uniform. Due to this reason, the user may not recognize the speckle included in the polymerized holographic image as noise.

According to an example embodiment, the processor PR may perform random phase modulation on the hologram profile for each frame of the binary phase spatial light modulator SLM driven at a particular driving frequency. For example, when the driving frequency of the holographic display device 10 is about 60 Hz, the processor PR may perform ten random phase modulations about every 1/60 sec. In other words, the driving frequency of the binary phase spatial light modulator SLM may be about 600 Hz.

FIG. 5 is a diagram for describing a viewing window formed by the holographic display device 1 including the binary phase spatial light modulator SLM.

Referring to FIG. 5, when a hologram profile for reproducing a holographic image is represented by the binary phase spatial light modulator SLM, a process of converting complex information into intensity of light may be essential, and in this case, information loss may inevitably occur.

Due to this reason, as illustrated in FIG. 5, DC term noise, high-order diffraction term noise, and conjugate image noise as well as signals to be displayed in the viewing window may be generated. Accordingly, an area, in which the user may actually normally view, may be a very narrow area excluding noise in the viewing window.

The holographic display device 1 illustrated in FIG. 1 may remove these pieces of noise by using the optical noise filtering system (that is, the 2-f system). On the other hand, because the holographic display device 10 of the disclosure excludes the optical noise filtering system illustrated in FIG. 2, these pieces of noise may be removed by providing an optimized hologram profile (or, the second hologram profile), that takes into consideration noise components, to the binary phase spatial light modulator SLM.

The optimized hologram profile (or, the second hologram profile) may be calculated by using the hologram profile optimization method. The hologram profile optimization method may encode an initial hologram profile into a binary hologram profile, convert the encoded binary hologram profile to spatial frequency information by using a Fourier transform, tile the converted spatial frequency information to model high-order diffraction noise generated by a diffraction grating, adjust the spatial frequency information to have a physically proper energy distribution, calculate a difference between intensity of a holographic image restored after restoring intensity of the holographic image on a display surface and intensity of a target image, and calculate an optimized hologram profile capable of minimizing the difference by using a gradient descent method.

According to an example embodiment, the hologram profile optimization method may be performed by the processor PR of the holographic display device 10 of the disclosure illustrated in FIG. 2. In this case, the processor PR may calculate the optimized hologram profile (or, the second hologram profile) considering the specification of the holographic display device 10. For example, the specification of the holographic display device 10 may include an interval between unit pixels of the spatial light modulator SLM, a center wavelength of light emitted by the light source LS, a propagation distance between the spatial light modulator SLM and the display surface, etc.

However, the disclosure is not limited thereto, and the hologram profile optimization method may also be performed by a hologram profile generation device to be described below. In this case, the hologram profile generation device may calculate the optimized hologram profile (or, the second hologram profile) considering the specification of the holographic display device to provide the hologram profile.

Hereinafter, the hologram profile optimization method is described in more detail with reference to FIGS. 6 through 11B.

FIG. 6 is a block diagram illustrating a hardware configuration of a hologram profile generation device 100, according to an example embodiment. FIG. 7 is a schematic diagram illustrating the holographic display device 10 of FIG. 2 to explain the hologram profile optimization method. FIG. 8 is a flowchart of a hologram profile optimization method. FIGS. 9A and 9B are a sign function and a derivative function, respectively, according to example embodiments. FIG. 10A and FIG. 10B are a sign function and a derivative function according to other embodiments, respectively. FIG. 11A and FIG. 11B are a sign function and a derivative function according to other embodiments, respectively.

Referring to FIG. 6, the hologram profile generation device 100 may include a processor 112, a memory 114, and a user interface 116. Only components related to embodiments are illustrated in the hologram profile generation device 100 illustrated in FIG. 6. Accordingly, it should be obvious to those of skill in the art that the hologram profile generation device 100 may further include other general-purpose components in addition to the components illustrated in FIG. 6.

The processor 112 may correspond to a processor provided in various types of computing devices, such as a personal computer (PC), a server device, a television (TV), a mobile device (smartphone, tablet device, or the like), an embedded device, an autonomous vehicle, a wearable device, an augmented reality (AR) device, an Internet of Things (IoT) device, etc. For example, the processor 112 may correspond to a processor, such as a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), a neural processing unit (NPU), or the like, but is not limited thereto.

The processor 112 may perform overall functions for controlling the hologram profile generation device 100 provided with the processor 112. The processor 112 may overall control the hologram profile generation device 100 by executing programs stored in the memory 114. For example, when the hologram profile generation device 100 is provided in the holographic display device (refer to 10 of FIG. 2), the processor 112 may control display of the holographic image by using the light source unit (refer to LU in FIG. 2) and the spatial light modulator SLM by controlling an image processing of the hologram profile generation device 100.

The processor 112 may include, for example, a control unit for overall controlling hologram profile processing operations, and a plurality of processing cores for performing the Fourier transform operation (that is, a fast Fourier transform (FFT) operation). Because the Fourier transform operation such as the FFT operation corresponds to operations that account for a large proportion of hologram profile processing operations, the processor 112 may accelerate hologram profile processing by using parallel processing of a plurality of processing cores. Each of the plurality of processing cores may be implemented as a circuit logic, in which, for example, a shift register, a multiplier, or the like are combined for performing the FFT operation. The number of processing cores in the processor 112 may be variously changed considering various factors, such as performance of the processor 112, resolution of the spatial light modulator SLM, and resolution of the holographic image. A plurality of FFT cores may perform FFT operations in parallel for performing a double stage Fourier transform (primary Fourier transform and secondary Fourier transform).

The memory 114 may be hardware for storing various types of data processed by the processor 112, and for example, the memory 114 may store hologram profile data processed by the processor 112 and hologram profile data to be processed.

Furthermore, the memory 114 may store various applications to be driven by the processor 112, for example, a hologram profile generation application, a web browsing application, a video application, etc.

The memory 114 may include at least one of a volatile memory and a nonvolatile memory. The non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), a flash memory, etc. The volatile memory may include dynamic RAM (DRAM), static RAM (SRAM), synchronous DRAM (SDRAM), phase-change RAM (PRAM), magnetic RAM (MRAM), resistive RAM (RRAM), ferroelectric RAM (FeRAM), etc. In an embodiment, the memory 114 may be a hard disk drive (HDD), a solid state drive (SSD), a compact flash (CF) memory, a secure digital (SD) memory, a micro secure digital Secure digital) memory, an extreme digital (xD) memory, or a memory stick.

The user interface 116 may include a device that receives a user's command or displays a usage state of the hologram profile generation device 100. For example, the user interface 116 may include a physical button, a touch screen, etc. In addition, the user interface 116 may include a port or the like for exchanging input/output with an external electronic device.

Referring to FIG. 7, the binary phase spatial light modulator SLM may form a hologram pattern on a light modulation surface according to a hologram profile provided by the processor (refer to PR in FIG. 2). Light incident on the binary spatial light modulator SLM may be converted into the holographic image by the hologram pattern. In this case, the holographic image may be displayed on the display surface apart from the light modulation surface of the binary phase spatial light modulator SLM by a first distance d.

Referring to FIG. 8, the hologram profile optimization method according to an example embodiment may include setting an initial hologram profile (or, the first hologram profile) as a variable to be optimized (S10), determining whether an optimization cycle is equal to a preset threshold (S20), encoding the initial hologram profile (or, the first hologram profile) into a binary hologram profile h (S30), calculating a field value z on a display surface with respect to the encoded binary hologram profile h (S40), calculating intensity I of a holographic image on the display surface (S50), calculating a loss function value L based on a difference between intensity I of the restored holographic image and intensity of a target image Î (S60), and updating the initial hologram profile (or, the first hologram profile) to a final hologram profile (or, the second hologram profile) based on the loss function value L (S70).

In operation S10 of setting the initial hologram profile (or, the first hologram profile) as a variable to be optimized, the processor 112 may generate an initial amplitude Av and an initial phase φv of the initial hologram profile (or the first hologram profile) and set them as variables to be optimized.

In operation S20 of determining whether the optimization cycle is equal to a preset threshold value, the processor 112 may determine whether a series of processes (that is, an optimization cycle) for updating the hologram profile reaches the preset threshold value. In other words, the optimization cycle is performed a predetermined number of times equal to the preset threshold value. For example, the threshold may be about 500. When the optimization cycle is less than the preset threshold value, the processor 112 may perform the optimization cycle once more. On the other hand, when the optimization cycle is equal to the preset threshold value, the processor 112 may stop the optimization cycle, and set the calculated variable to a final optimized hologram file (or, the second hologram profile).

In operation S30 of encoding the initial hologram profile (or, the first hologram profile) to the binary hologram profile h, the processor 112 may encode the initial hologram profile (or, the first hologram profile) to the binary hologram profile h by using Formula 1 below.

h=sign(A_v cos(φ_v)+eps)  [Formula 1]