HTC Patent | Head mounted display device

Patent: Head mounted display device

Drawings: Click to check drawins

Publication Number: 20220308662

Publication Date: 20220929

Applicants: Htc

Assignee: Htc Corporation

Abstract

A head mounted display device, including a frame, a mask, at least one infrared (IR) transmitter, and at least one image capture device, is provided. The mask has a first light reflection layer on a first side. The IR transmitter is disposed in the frame and is used to emit an emitting light beam toward the first light reflection layer. The first light reflection layer reflects the emitting light beam to send a reflective light beam toward a target area. The image capture device is disposed in the frame and is used to capture a target area reflective image of the target area.

Claims

1. A head mounted display device, comprising: a frame; a mask, having a first light reflection layer on a first side thereof; at least one infrared transmitter, disposed in the frame and used to emit at least one emitting light beam toward the first light reflection layer, wherein the first light reflection layer reflects the at least one emitting light beam to send at least one reflective light beam toward a target area; and at least one image capture device, disposed in the frame and used to capture at least one target area reflective image of the target area.

2. The head mounted display device according to claim 1, wherein the at least one reflective light beam generates at least one light spot in the at least one target area reflective image.

3. The head mounted display device according to claim 1, wherein the at least one image capture device captures the target area reflective image from the first light reflection layer on the mask.

4. The head mounted display device according to claim 1, wherein the target area comprises a first region corresponding to a first eye of a user and a second region corresponding to a second eye of the user, the head mounted display device further comprising: a fisheye lens, disposed in front of a lens of the at least one image capture device, so that the at least one image capture device simultaneously captures a first target area reflective image corresponding to the first eye of the user and a second target area reflective image corresponding to the second eye of the user.

5. The head mounted display device according to claim 4, wherein the target area comprises the first region corresponding to the first eye of the user and the second region corresponding to the second eye of the user; the at least one image capture device comprises a first image capture device and a second image capture device; the first image capture device is used to simultaneously capture the first target area reflective image corresponding to the first eye of the user in the first region and the second target area reflective image corresponding to the second eye of the user in the second region; and the second image capture device is used to simultaneously capture a third target area reflective image corresponding to the first eye of the user in the first region and a fourth target area reflective image corresponding to the second eye of the user in the second region.

6. The head mounted display device according to claim 5, wherein the at least one image capture device is further used to capture an auxiliary image of an auxiliary target area, wherein the auxiliary target area corresponds to lips of the user.

7. The head mounted display device according to claim 6, wherein the first target area reflective image and the second target area reflective image are infrared light images, and the auxiliary image is a visible light image.

8. The head mounted display device according to claim 7, wherein the at least one image capture device has a visible light sensing pixel and an infrared light sensing pixel.

9. The head mounted display device according to claim 6, further comprising: a light reflection element, disposed in the frame, wherein the at least one image capture device captures the auxiliary image through the light reflection element.

10. The head mounted display device according to claim 1, further comprising: a light reflection element, disposed in the frame, wherein the at least one image capture device captures the at least one target area reflective image through the light reflection element.

11. The head mounted display device according to claim 1, further comprising: at least one lens set, disposed in front of a lens of the at least one image capture device.

12. The head mounted display device according to claim 11, wherein the at least one lens set has an adjustable focal length.

13. The head mounted display device according to claim 1, wherein the at least one image capture device is further used to capture a reference object image outside a second side of the mask.

14. The head mounted display device according to claim 1, further comprising: a fingerprint identification indicator, disposed in the frame and used to define a fingerprint identification area on the mask, wherein the at least one image capture device captures a fingerprint image on the fingerprint identification area as a basis for a fingerprint identification action.

15. The head mounted display device according to claim 1, further comprising: a first controller, coupled to the at least one infrared transmitter and the at least one image capture device, and driving the at least one infrared transmitter to send the at least one emitting light beam to execute an eyeball tracking action on the target area according to the at least one light spot on the at least one target area reflective image.

16. The head mounted display device according to claim 15, further comprising: a display, disposed between the target area and the mask and coupled to the first controller; and the display being used to project a display image to the target area.

17. The head mounted display device according to claim 16, further comprising: a display driver, coupled to the display, wherein the display driver receives synchronization information transmitted by the at least one image capture device and drives the display to generate the display image according to the synchronization information.

18. The head mounted display device according to claim 16, further comprising: a second controller, coupled to the display driver and used to provide a virtual display image.

19. The head mounted display device according to claim 15, wherein the first controller is further used to perform an iris recognition action according to the at least one target area reflective image.

20. The head mounted display device according to claim 15, wherein the display has a second light reflection layer on a surface facing the target area, and the at least one image capture device captures the target area reflective image from the second light reflection layer on the display.

Description

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the priority benefit of U.S. Provisional Application No. 63/166,243, filed on Mar. 26, 2021. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.

BACKGROUND

Technical Field

[0002] The disclosure relates to a head mounted display device, and more particularly relates to a head mounted display device that captures an eyeball image through the mechanism of light reflection.

Description of Related Art

[0003] Among head mounted display products for providing virtual reality and augmented reality that are currently on the market, cameras for executing eyeball tracking actions generally adopt an exposed design. In other words, the cameras for executing eyeball tracking actions are all disposed around the display. As a result, when the camera needs to capture an image of the eyeballs of the user, there is often a considerable angle difference between the lens of the camera and the eyeballs of the user, which affects the performance of the head mounted display device.

SUMMARY

[0004] The disclosure provides a head mounted display device that captures an eyeball image through the mechanism of light reflection.

[0005] The head mounted display device of the disclosure includes a frame, a mask, at least one infrared transmitter, and at least one image capture device. The mask has a first light reflection layer on a first side. The infrared transmitter is disposed in the frame and is used to emit an emitting light beam toward the first light reflection layer. The first light reflection layer reflects the emitting light beam to send a reflective light beam toward a target area. The image capture device is disposed in the frame and is used to capture a target area reflective image of the target area.

[0006] Based on the above, the disclosure reflects the emitting light beam generated by the infrared transmitter to the target area through the light reflection layer on the mask. The image capture device of the disclosure is used to capture the target area reflective image of the target area. In this way, the infrared emitting light beam may be projected to the target area frontally, and the image capture device may capture a frontal eyeball image of the user, which can improve the accuracy of an eyeball tracking action.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 is a schematic view of a head mounted display device according to an embodiment of the disclosure.

[0008] FIG. 2A is a top view of a head mounted display device according to another embodiment of the disclosure.

[0009] FIG. 2B is a side view of a head mounted display device 200 according to the embodiment in FIG. 2A of the disclosure.

[0010] FIG. 3 is a schematic view of a head mounted display device according to another embodiment of the disclosure.

[0011] FIG. 4 is a schematic view of a head mounted display device according to another embodiment of the disclosure.

[0012] FIG. 5 is a schematic view of a structure of sensing pixels of an image capture device according to an embodiment of the disclosure.

[0013] FIG. 6A and FIG. 6B are respectively schematic views of a head mounted display device according to different embodiments of the disclosure.

[0014] FIG. 7A and FIG. 7B are respectively schematic views of a head mounted display device according to different embodiments of the disclosure.

[0015] FIG. 8A and FIG. 8B are respectively schematic views of head mounted display devices according to different embodiments of the disclosure.

[0016] FIG. 9 is a schematic view of a head mounted display device according to another embodiment of the disclosure.

[0017] FIG. 10A to FIG. 10C are respectively schematic views of various different control mechanisms of head mounted display devices according to an embodiment of the disclosure.

DETAILED DESCRIPTION OF DISCLOSED EMBODIMENTS

[0018] Please refer to FIG. 1. FIG. 1 is a schematic view of a head mounted display device according to an embodiment of the disclosure. A head mounted display device 100 includes a frame 110, a mask 120, infrared transmitters 131 and 132, an image capture device 140, and a display 150. The frame 110 is combined with the mask 120. The mask 120 is disposed in front of a target area TG. The target area TG corresponds to positions of the eyeballs of the user of the head mounted display device 100.

[0019] The mask 120 has a light reflection layer 121 on a first side thereof. In the embodiment, the first side of the mask 120 is the side facing the target area TG. In addition, the infrared transmitters 131 and 132 are disposed in the frame 110, and the infrared transmitters 131 and 132 are used to respectively emit emitting light beams IR1 and IR2 toward the light reflection layer 121. The light reflection layer 121 also reflects the emitting light beams IRl and IR2 to respectively send reflective light beams RIR1 and RIR2 toward the target area TG. The number of the infrared transmitters 131 and 132 in the embodiment is two, which is only an example for illustration. In other embodiments of the disclosure, the number of infrared transmitters may be one or more, which is not particularly limited.

[0020] In addition, the image capture device 140 is disposed in the frame 110. The image capture device 140 is used to capture a target area reflective image RTMG of the target area TG. In the embodiment, a target area image TMG of the target area TG may be transmitted to the light reflection layer 121. The light reflection layer 121 is used to reflect the target area image TMG to generate the target area reflective image RTMG, and the target area reflective image RTMG is projected into the image capture device 140. It is worth noting that in the embodiment, when the reflective light beams RIR1 and RIR2 are projected to the target area TG, in the target area reflective image RTMG captured by the image capture device 140, there may be light spots generated by the reflective light beams RIR1 and RIR2.

[0021] From the above description, it can be known that the head mounted display device 100 of the disclosure may analyze the light spots generated on the eyeballs of the user through the target area reflective image RTMG obtained by the image capture device 140, so as to execute an eyeball tracking action. In the embodiment, through the mechanism of light reflection, incidence angles of the reflective light beams RIR1 and RIR2 projected to the target area TG may be effectively reduced, and an included angle between an image capture direction of the image capture device 140 and a normal vector of the eyeballs may also be effectively reduced.

[0022] In other words, the image capture device 140 may capture a frontal eyeball image of the user, which can effectively improve the accuracy of eyeball tracking.

[0023] Furthermore, the display 150 in the embodiment is disposed between the target area TG and the mask 120, and is combined with the frame 110. The display 150 may be a transmissive display, the mask 120 may be a transparent mask, and a visible light VL may be transmitted to the target area TG through the mask 120 and the display 150. The display 150 may be used to provide virtual reality or augmented reality display images.

[0024] Please refer to FIG. 2A. FIG. 2A is a top view of a head mounted display device according to another embodiment of the disclosure. A head mounted display device 200 includes a frame 210, a mask 220, an infrared transmitter 230, an image capture device 240, and displays 251 and 252. In the embodiment, a target area TG may be divided into a first region TG1 corresponding to a first eye of the user and a second region TG2 corresponding to a second eye of the user. The displays 251 and 252 are located between the first region TG1 and the second region TG2 and the mask 220. The image capture device 240 may be disposed in the frame 210 and located between the displays 251 and 252.

[0025] In the embodiment, the mask 220 has a light reflection layer 221 on the side facing the target area TG. The light reflection layer 221 may be a light reflection coating or composed of a light reflection material coated on the mask 220. There may be a fisheye lens 141 on the lens of the image capture device 240. The fisheye lens 241 enables the image capture device 240 to simultaneously capture a first target area reflective image corresponding to the first eye of the user and a second target area reflective image corresponding to the second eye of the user. In the embodiment, the head mounted display device 200 may obtain the images of both eyes of the user in the target area TG through disposing the single image capture device 240.

[0026] Furthermore, in the embodiment, the number of the infrared transmitter 230 is multiple, and the infrared transmitters 230 may be evenly distributed in the frame 210. Part of the infrared transmitters 230 are used to project emitting light beams to the light reflection layer 221, and generate and transmit reflective light beams to the first eye of the user through the light reflection layer 221. The other part of the infrared transmitters 230 are used to project emitting light beams to the light reflection layer 221, and generate and transmit reflective light beams to the second eye of the user through the light reflection layer 221.

[0027] Please refer to FIG. 2B below. FIG. 2B is a side view of the head mounted display device 200 of the embodiment in FIG. 2A of the disclosure. The infrared transmitter 230 is used to generate the emitting light beams IRI and IR2. The light reflection layer 221 on the mask 220 reflects the emitting light beams IRI and IR2 to generate the reflective light beams RIR1 and RIR2, and transmits the reflective light beams RIR1 and RIR2 to the first region TG1.

[0028] The image capture device 240 captures the target area reflective image of the eyeballs of the user in the first region TG1 through the fisheye lens 241 and the light reflection layer 221 on the mask 220.

[0029] Please refer to FIG. 3 below. FIG. 3 is a schematic view of a head mounted display device according to another embodiment of the disclosure. A head mounted display device 300 includes a frame 310, a mask 320, an infrared transmitter 330, image capture devices 341 and 342, and displays 351 and 352. Different from the embodiment in FIG. 2A, the head mounted display device 300 of the embodiment has two image capture devices 341 and 342. The image capture devices 341 and 342 are disposed on two opposite sides of the frame 310. The image capture device 341 is used to simultaneously capture a first target area reflective image corresponding to a first eye of the user in a first region TG1 and a second target area reflective image corresponding to a second eye of the user in a second region TG2, and the image capture device 342 is used to simultaneously capture a third target area reflective image corresponding to the first eye of the user in the first region TG1 and a fourth target area reflective image corresponding to the second eye of the user in the second region TG2. The first target area reflective image, the second target area reflective image, the third target area reflective image, and the fourth target area reflective image may be used as the basis for an eyeball tracking action.

[0030] Please refer to FIG. 4. FIG. 4 is a schematic view of a head mounted display device according to another embodiment of the disclosure. A head mounted display device 400 includes a frame 410, a mask 420, infrared transmitters 431 and 432, an image capture device 440, and a display 450. In the embodiment, the frame 410 is combined with the mask 420. The mask 420 is disposed in front of a target area TG. The target area TG corresponds to positions of the eyeballs of the user of the head mounted display device 400. The mask 420 has a light reflection layer 421 on a first side thereof. In the embodiment, the first side of the mask 420 is the side facing the target area TG. In addition, the infrared transmitters 431 and 432 are disposed in the frame 410, and the infrared transmitters 431 and 432 are used to respectively emit emitting light beams toward the light reflection layer 421. The light reflection layer 421 reflects the emitting light beams toward the target area TG to respectively send corresponding reflective light beams. The image capture device 440 captures a target area reflective image corresponding to the target area through the light reflection layer 421.

[0031] In the embodiment, different from the foregoing embodiment, the image capture device 440 may be used to capture an auxiliary image on an auxiliary target area ATG. The auxiliary target area ATG may correspond to a lip part of the user, and the auxiliary image may be a lip image of the user. The lip image of the user is captured through the image capture device 440.

[0032] The head mounted display device 400 may not only execute an eyeball tracking action, but also execute a lip tracking action of the user, which can effectively improve the performance of the head mounted display device 400.

[0033] It is worth mentioning that in the embodiment, the lens of the image capture device 440 may directly face the auxiliary target area ATG and directly capture the auxiliary image. The target area reflective image is an infrared light image, and the auxiliary image is a visible light image.

[0034] In addition, please refer to FIG. 5. FIG. 5 is a schematic view of a structure of sensing pixels of an image capture device according to an embodiment of the disclosure. In order to simultaneously capture an infrared light image and a visible light image, the image capture device may have multiple visible light sensing pixels PX1 to PX3 and an infrared light sensing pixel PX4. The visible light sensing pixels PX1 to PX3 are respectively used to sense red (R), green (G), and blue (B) visible light images, and the infrared light sensing pixel PX4 is used to sense the infrared (IR) light images. The visible light sensing pixels PX1 to PX3 and the infrared light sensing pixel PX4 may be arranged in an array, but there is no specific arrangement sequence.

[0035] Please refer to FIG. 6A and FIG. 6B below. FIG. 6A and FIG. 6B are respectively schematic views of a head mounted display device according to different embodiments of the disclosure. In FIG. 6A, a head mounted display device 600 includes a frame 610, a mask 620, infrared transmitters 631 and 632, an image capture device 640, a display 650, and a light reflection element 660. The infrared transmitters 631 and 632, the image capture device 640, and the light reflection element 660 are all disposed in the frame 610. In the embodiment, the image capture device 640 may capture a target area reflective image on the light reflection layer 621 of the mask 620 through the light reflection element 660. Through the configuration of the light reflection element 660, the convenience of the image capture device 640 in terms of configuration can be improved, which facilitates the design of the head mounted display device 600 in terms of mechanical.

[0036] The image capture device 640 is a camera that can capture an infrared light image and a visible light image. The mask 620 may be a transparent mask, and the display 650 may be a transmissive display. A visible light VL may be transmitted to a target area TG through the mask 620 and the display 650.

[0037] It is worth mentioning that the light reflection surface of the light reflection element 660 may be adjusted. When the light reflection surface of the light reflection element 660 is at a first angle, the image capture device 640 may capture an image of the eyeballs of the user in the target area TG. In addition, when the light reflection surface of the light reflection element 660 rotates to a second angle, as shown in FIG. 6B, the image capture device 640 is used to capture a lip image of the user in an auxiliary target area ATG.

[0038] It can be seen from FIG. 6A and 6B that through changing the light reflection surface of the light reflection element 660 between the first angle and the second angle, the head mounted display device 600 may respectively work in an eyeball tracking mode and a lip tracking mode, which effectively improves the performance of the head mounted display device 600.

[0039] Please refer to FIG. 7A and FIG. 7B below. FIG. 7A and FIG. 7B are respectively schematic views of a head mounted display device according to different embodiments of the disclosure. In FIG. 7A, a head mounted display device 700 includes a frame 710, a mask 720, infrared transmitters 731 and 732, an image capture device 740, a display 750, and a lens set 741. Different from the foregoing embodiment, the lens set 741 is a lens set with an adjustable focal length and is disposed in front of the lens of the image capture device 740. When the lens set 741 is at a first focal length, the image capture device 740 may capture a target area image on a target area TG of a first field of view RG1. When the lens set 741 is adjusted to a second focal length, as shown in FIG. 7B, the image capture device 740 may capture a target area image on the target area TG of a second field of view RG2. The first focal length is greater than the second focal length, and the first field of view RG1 may be greater than the second field of view RG2.

[0040] It is worth mentioning that when the lens set 741 is adjusted to the second focal length, the image capture device 740 may capture a small range image of the eyeballs of the user, and the head mounted display device 700 may execute an iris recognition action of the user according to the small range image. In other words, through the configuration of the lens set 741, the performance of the head mounted display device 700 can be further improved.

[0041] Next, please refer to FIG. 8A and FIG. 8B. FIG. 8A and FIG. 8B are respectively schematic views of head mounted display devices according to different embodiments of the disclosure. In FIG. 8A, a head mounted display device 801 includes a frame 810, a mask 820, infrared transmitters 831 and 832, an image capture device 840, and a display 850. The mask 820 has a light reflection layer 821 on a first side.

[0042] Different from the foregoing embodiment, the image capture device 840 of the embodiment of the disclosure may be further used to capture a reference object image of a reference object EXTO outside a second side of the mask 820. The first side and the second side of the mask 820 are opposite to each other. The first side of the mask 820 is located on the inner side of the head mounted display device 800, and the second side of the mask 820 is located on the outer side of the head mounted display device 800.

[0043] The reference object EXTO of the embodiment may be a hand of the user or may also be a tool, which may be any tool such as a pen and a lamp, held by the user. The head mounted display device 801 may execute a corresponding action through the reference object image of the reference object EXTO captured by the image capture device 840. Simply put, the user may use a gesture to issue an operation command to the head mounted display device 801, and the head mounted display device 801 may analyze the command to be executed according to the reference object image.

[0044] Here, the head mounted display device 801 may directly capture the reference object image of the reference object EXTO without the mechanism of light reflection.

[0045] In FIG. 8B, compared to the head mounted display device 801, a head mounted display device 802 further includes a fingerprint identification indicator 833. The fingerprint identification indicator 833 is disposed in the frame 810. The fingerprint identification indicator 833 is used to define a fingerprint identification area IDZ on the mask 820. The fingerprint identification indicator 833 may be an infrared transmitter, wherein the wavelength of infrared light sent by the fingerprint identification indicator 833 may be different from the wavelengths of emitting light beams sent by the infrared transmitters 831 and 832.

[0046] In addition, the fingerprint identification area IDZ is used by the user to perform a fingerprint pressing action. Simultaneously, the image capture device 840 may capture a fingerprint image of the user on the fingerprint identification area IDZ and execute a fingerprint identification action according to the fingerprint image.

[0047] Please refer to FIG. 9. FIG. 9 is a schematic view of a head mounted display device according to another embodiment of the disclosure. A head mounted display device 900 includes a frame 910, a mask 920, infrared transmitters 931 and 932, an image capture device 940, and a display 950. The mask 920 has a light reflection layer 921 for reflecting infrared light beams generated by the infrared transmitters 931 and 932 on a first side. Different from the foregoing embodiment, the display 950 of the embodiment has another light reflection layer 951 on the surface facing a target area. The lens of the image capture device 940 faces the light reflection layer 951 and is used to capture a target area reflective image of the target area on the light reflection layer 951.

[0048] Please refer to FIG. 10A to FIG. 10C. FIG. 10A to FIG. 10C are respectively schematic views of various different control mechanisms of head mounted display devices according to an embodiment of the disclosure. In FIG. 10A, a head mounted display device 1001 includes a controller 1011, a memory 1021, an infrared light emitting diode (IR LED) 1031, an infrared (IR) camera 1041, and a liquid crystal lens element 1051, which may correspond to the embodiment of FIG. 7A and FIG. 7B of the disclosure. The infrared light emitting diode 1031 corresponds to the infrared transmitters 731 and 732, the infrared camera 1041 corresponds to the image capture device 740, and the liquid crystal lens element 1051 corresponds to the zoom lens element 741.

[0049] The controller 1011 is coupled to the memory 1021, the infrared light emitting diode 1031, the infrared camera 1041, and the liquid crystal lens element 1051. The controller 1011 may send a control signal that is an electrical signal to control the field of view of the infrared camera 1041. The controller 1011 may generate a first lock key according to a random number generation mechanism and determine the number and the position of the infrared light emitting diode 1031 to be lit through the first lock key. Simultaneously, the infrared camera 1041 may capture a first eyeball image on a target area TG. Then, the controller 1011 generates a second lock key according to the random number generation mechanism and determine the number and the position of the infrared light emitting diode 1031 to be lit through the second lock key, so that the infrared camera 1041 may capture a second eyeball image of the target area TG. Then, the controller 1011 may compare the first eyeball image and the second eyeball image according to a preset database and execute an iris recognition action for the user.

[0050] The memory 1021 in the embodiment may be used as a temporary storage area for data required during an iris recognition process or may be used to store a preset database.

[0051] In addition, in FIG. 10B, a head mounted display device 1002 includes a controller 1012, a display driver 1032, a display 1042, a controller 1022, an infrared light emitting diode (IR LED) 1052, and an image capture device 1062, which may correspond to the embodiment in

[0052] FIG. 8A of the disclosure. The infrared light emitting diode 1031 corresponds to the infrared transmitters 831 and 832. The controller 1012 is coupled to the display driver 1032, the infrared light emitting diode 1052, and the image capture device 1062. The display driver 1032 is coupled to the display 1042, the controller 1022, and the image capture device 1062.

[0053] In the embodiment, the head mounted display device 1002 may be used to execute a three-dimensional virtual object display action of eyeball tracking and space tracking. The controller 1012 may load position correction information related to the image capture device 1062 and the display 1042. The display 1042 may be used to display a three-dimensional image. Then, the image capture device 1062 may capture an eyeball image of the user and a hand image the user. Then, the controller 1012 analyzes and obtains corresponding positions of the eyeballs and a hand in a three-dimensional space. Then, the controller 1012 calculates sizes corresponding to virtual object images in the space according to the corresponding positions of the eyeballs and the hand. The virtual object images are divided into two-dimensional images of the left and right eyes. The two-dimensional images are combined with other virtual display images (such as a status bar), then output to the displays 1042 respectively corresponding to different eyes for the user to view, and then synthesized by the brain of the user to generate a three-dimensional virtual image.

[0054] In the embodiment, the display driver 1032 receives synchronization information Sync transmitted by the image capture device 1062 and drives the display 1042 according to the synchronization information Sync to generate a display image. A virtual display image may be provided by the controller 1022. The display driver 1032 may execute a combining action of the two-dimensional images and the virtual display image.

[0055] In FIG. 10C, a head mounted display device 1003 includes a controller 1013, a memory 1023, an infrared light emitting diode (IR LED) 1033, and an image capture device 1043, which may correspond to the embodiment in FIG. 6B of the disclosure. The infrared light emitting diode 1033 corresponds to the infrared transmitters 631 and 632. The controller 1013 is coupled to the memory 1023, the infrared light emitting diode (IR LED) 1033, and the image capture device 1043.

[0056] In terms of action execution, the controller 1013 may first load position correction information of the image capture device 1043. Then, the image capture device 1043 may capture an eyeball image and a lip image of the user. The controller 1013 analyzes a gaze point of the user and analyzes lip information. The controller 1013 may update avatar information of the user according to the gaze point of the user and the lip information. The controller 1013 may store multiple continuous lip information in a queue according to time sequence and obtain lip language information expressed by the user through analyzing the lip information.

[0057] The image capture device 1043 may continuously execute actions of capturing the eyeball image and the lip image of the user to achieve real-time monitoring of head movement and related information such as the lip language information of the user.

[0058] In summary, in the head mounted display device of the disclosure, the image capture device captures the target area reflective image of the target area by using the mechanism of light reflection, and the infrared transmitter transmits the infrared light beam to the eyeballs of the user also by using the mechanism of light reflection. In this way, the infrared emitting light beam may be projected to the target area frontally, and the image capture device may capture the frontal eyeball image of the user, which can improve the accuracy of the eyeball tracking action.