Facebook Patent | Virtual path generation in a virtual environment that is based on a physical environment

Patent: Virtual path generation in a virtual environment that is based on a physical environment

Drawings: Click to check drawins

Publication Number: 20210232210

Publication Date: 20210729

Applicant: Facebook

Abstract

In one embodiment, a method includes segmenting a layout of a physical space surrounding a user into physical segments; generating, based on the physical segments, virtual paths for a virtual environment through which the user can navigate by traveling the physical segments; displaying a particular virtual path based on a location of the user in the virtual environment; determining that a forward direction of travel of the user is proximate to a boundary condition of a particular physical segment corresponding to the particular virtual path; notifying the user that a physical rotation of the user is needed in order for the user to travel beyond a point in the particular virtual path; and detecting that the physical rotation of the user is complete, and in response, updating the display to show the particular virtual path and allowing the user to travel beyond the point in the particular virtual path.

Claims

1. A computer-implemented method, comprising: segmenting a layout of a physical space surrounding a user into physical segments; generating, based on the physical segments, virtual paths for a virtual environment through which the user can navigate by traveling the physical segments; displaying a particular virtual path of the virtual paths based on a location of the user in the virtual environment; determining that a forward direction of travel of the user is proximate to a boundary condition of a particular physical segment of the physical segments, the particular physical segment corresponding to the particular virtual path; in response to the determining, notifying the user that a physical rotation of the user is needed in order for the user to travel beyond a point in the particular virtual path; and detecting that the physical rotation of the user is complete, and in response, updating the display to show the particular virtual path and allowing the user to travel beyond the point in the particular virtual path.

2. The computer-implemented method of claim 1, wherein notifying the user that the physical rotation of the user is needed for the user to travel beyond the point in the particular virtual path further includes displaying a graphical user interface (GUI) element within the particular virtual path that indicates that the physical rotation of the user is needed.

3. The computer-implemented method of claim 1, wherein notifying the user that the physical rotation of the user is needed for the user to travel beyond the point in the particular virtual path further includes providing an audio notification that indicates that the physical rotation of the user is needed.

4. The computer-implemented method of claim 1, further comprising: during detection of the physical rotation of the user, updating a graphical user interface (GUI) that displays the virtual environment such that the GUI correlates with the physical rotation of the user.

5. The computer-implemented method of claim 4, wherein updating the display to show the particular virtual path further includes updating the GUI to display the particular virtual path after detecting that the physical rotation of the user is complete.

6. The computer-implemented method of claim 4, wherein updating the GUI such that the GUI correlates with the physical rotation of the user further comprises rotating a virtual scene of the GUI that correlates with the physical rotation of the user.

7. The computer-implemented method of claim 6, wherein rotating the virtual scene of the GUI includes rotating the virtual scene of the GUI approximately 90 degrees.

8. The computer-implemented method of claim 6, after detecting the physical rotation of the user is complete, rotating the virtual scene of the GUI in a direction opposite to the rotation of the virtual scene of the GUI that correlates with the physical rotation of the user.

9. The computer-implemented method of claim 8, wherein the virtual scene of the GUI is rotated in the direction of opposite to the rotation of the virtual scene of the GUI that correlates with the physical ration of the user for approximately 0.3 seconds.

10. The computer-implemented method of claim 1, further comprising: during detection of the physical rotation of the user, updating a graphical user interface (GUI) that displays the virtual environment such that movement of the GUI is frozen.

11. The computer-implemented method of claim 1, wherein determining that the forward direction of travel of the user is proximate to the boundary condition of a particular physical segment of the physical segments further includes determining that the user is within a threshold distance of a physical constraint of the particular physical segment.

12. The computer-implemented method of claim 1, wherein generating the virtual paths includes generating the particular virtual path based on a first and a second physical segment of the physical segments, where the first and the second physical segments share a common point.

13. A computer-implemented method, comprising: segmenting a layout of a physical space surrounding a user into physical segments; generating, based on the physical segments, virtual paths for a virtual environment through which the user can navigate by traveling the physical segments; displaying a particular virtual path of the virtual paths based on a location of the user in the virtual environment; determining that a forward direction of travel of the user is proximate to a boundary condition of a particular physical segment of the physical segments, the particular physical segment corresponding to the particular virtual path; in response to the determining, notifying the user that a physical rotation of the user is needed in order for the user to travel beyond a point in the particular virtual path; and detecting that the physical rotation of the user is complete, and in response, updating the display to show a different virtual path and allowing the user to travel along the different virtual path.

14. The computer-implemented method of claim 13, further comprising: during detection of the physical rotation of the user, updating a graphical user interface (GUI) that displays the virtual environment such that the GUI corelates with the physical rotation of the user.

15. The computer-implemented method of claim 14, wherein updating the GUI such that the GUI correlates with the physical rotation of the user further comprises rotating a virtual scene of the GUI that correlates with the physical rotation of the user.

16. The computer-implemented method of claim 15, wherein rotating the virtual scene of the GUI includes rotating the virtual scene of the GUI approximately 180 degrees.

17. The computer-implemented method of claim 13, wherein the different virtual path is opposite the particular virtual path.

18. A system comprising a processor having access to memory media storing instructions executable by the processor to: segmenting a layout of a physical space surrounding a user into physical segments; generating, based on the physical segments, virtual paths for a virtual environment through which the user can navigate by traveling the physical segments; displaying a particular virtual path of the virtual paths based on a location of the user in the virtual environment; determining that a forward direction of travel of the user is proximate to a boundary condition of a particular physical segment of the physical segments, the particular physical segment corresponding to the particular virtual path; in response to the determining, notifying the user that a physical rotation of the user is needed in order for the user to travel beyond a point in the particular virtual path; and detecting that the physical rotation of the user is complete, and in response, updating the display to show the particular virtual path and allowing the user to travel beyond the point in the particular virtual path.

19. The system of claim 18, wherein notifying the user that the physical rotation of the user is needed for the user to travel beyond the point in the particular virtual path further includes displaying a graphical user interface (GUI) element within the particular virtual path that indicates that the physical rotation of the user is needed.

20. The system of claim 18, wherein notifying the user that the physical rotation of the user is needed for the user to travel beyond the point in the particular virtual path further includes providing an audio notification that indicates that the physical rotation of the user is needed.

Description

TECHNICAL FIELD

[0001] This disclosure generally relates to generation virtual paths in a virtual environment that is based on a physical environment.

BACKGROUND

[0002] Traditional virtual reality (VR) computing systems are designed to be used within a confined space. Due to the limited physical space for walking within the confined space, content designers (e.g., game designers) may not use the user’s physical relocation as the mechanism for navigating a virtual environment. While it is possible to simulate walking in a virtual reality environment by the press of a button, doing so would likely cause motion sickness due to the contradiction between the visually perceived movement and the vestibular system’s sense of bodily movement. Due to these limitations, traditional VR content designers make “teleportation” the mechanism for navigating a virtual environment. For example, the user may point at a location in a virtual environment, press a button, and teleport instantaneously to that location without being shown the transitional movement. While the teleportation mechanism provides a viable solution to the problem of VR navigation, it does not provide a realistic, immersive experience (e.g., as the journey to each destination is skipped).

SUMMARY OF PARTICULAR EMBODIMENTS

[0003] This application discusses a solution to the virtual reality (VR) navigation problem by enabling VR content developers to procedurally generate navigation paths based on the physical room layout available to the user. In addition, a new user interface is inserted into certain junctions within the procedurally-generated path to cause the user to physically turn in the real world to avail the user with walking space, despite the limitations of the physical room layout. At such a junction, the user may be asked to perform a turning motion in order to continue. The degree of turn required by the turning motion would depend on the physical layout of the user’s space. To address the potential mismatch between the degree of turn made by the user and the direction of travel down the procedurally-generated path in VR, the VR content may quickly rotate the VR environment so that the user would be facing the intended travel direction in the virtual environment. For example, when the user reaches the end of a physical corridor, the VR content may position a UI device that needs to be rotated in order for the user to continue. If the physical corridor is a dead end, the user may have to turn the UI device 180 degree about the user’s axis in order to unlock the path. In the real world, after doing so, the user would be facing the direction from which he came. Visually in VR, the user will see the scene rotate quickly so that he would be facing the intended direction of travel.

[0004] The embodiments disclosed herein are only examples, and the scope of this disclosure is not limited to them. Particular embodiments may include all, some, or none of the components, elements, features, functions, operations, or steps of the embodiments disclosed herein. Embodiments according to the invention are in particular disclosed in the attached claims directed to a method, a storage medium, a system and a computer program product, wherein any feature mentioned in one claim category, e.g. method, can be claimed in another claim category, e.g. system, as well. The dependencies or references back in the attached claims are chosen for formal reasons only. However any subject matter resulting from a deliberate reference back to any previous claims (in particular multiple dependencies) can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims. The subject-matter which can be claimed comprises not only the combinations of features as set out in the attached claims but also any other combination of features in the claims, wherein each feature mentioned in the claims can be combined with any other feature or combination of other features in the claims. Furthermore, any of the embodiments and features described or depicted herein can be claimed in a separate claim and/or in any combination with any embodiment or feature described or depicted herein or with any of the features of the attached claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] FIG. 1 illustrates an example environment for generation of virtual paths in a virtual environment based on a physical environment.

[0006] FIG. 2 illustrates an example physical environment.

[0007] FIG. 3A, 3B, 3C illustrate example virtual paths.

[0008] FIG. 4 illustrates connected virtual paths of a virtual environment.

[0009] FIG. 5 illustrates an example method for generation of virtual paths in a virtual environment based on a physical environment.

[0010] FIG. 6 illustrates an example computer system.

[0011] FIG. 7 illustrates an example artificial reality system.

DESCRIPTION OF EXAMPLE EMBODIMENTS

[0012] FIG. 1 illustrates an environment 100 for generation of virtual paths in a virtual environment based on a physical environment. The environment 100 can include a physical space 102 that can include a user 104 interacting with a computing system 106. In some examples, the computing system 106 can include a virtual reality computing system, an augmented reality computing system, or a mixed-reality computing system. The computing system 106 can include a virtual path computing module 108.

[0013] The virtual path manager computing module 108 can receive physical space data 112 identifying a layout of the physical space 102 surrounding the user 104. For example, the computing system 106 can include a wearable computing device (such as a headset wearable computing device). The user 104 can travel (walk) about the physical space 102 such that the computing system 106 can identify/recognize the physical space 102 (e.g., utilizing a camera sensing module) to generate the physical space data 112 (e.g., a topology of the physical space 102).

[0014] The virtual path manager computing module 108 can segment the layout of the physical space 102 surrounding the user 104 into one or more physical segments 114. Referring to FIG. 2, an environment 200 is illustrated. For example, the environment 200 can include a multiple (physical) rooms 204a, 204b, 204c, 204d, 204e, 204f (collectively referred to as rooms 204) and hallways 206a, 206b, 206c (collectively referred to as hallways 206) connecting one or more of the rooms 204. To that end, as the user 104 walks about the environment 200, the virtual path manager computing module 108 can identify the layout of the physical space 208, similar to the physical space 102, included by the environment 200. Referring to FIGS. 3A, 3B, 3C, the virtual path manger computing module 108 can segment the layout of the physical space 208 into physical segments 210a, 210b, 210c (collectively referred to as physical segments 210). In some examples, a portion of one or more of the segments 210 can overlap with another segment 210 (e.g., the segment 210b overlaps with the segment 210a).

[0015] Referring back to FIG. 1, the virtual path manager computing module 108 can generate, based on the physical segments 114, virtual paths 116 of a virtual environment 118 through which the user 104 can navigate by traveling the physical segments 114. Specifically, the virtual path manager computing module 108 can generate the virtual environment 118 for display on a graphical user interface (GUI) 119. Referring to FIG. 4, virtual paths of a virtual environment 400 is shown. Specifically, based on the physical segments 210 of FIGS. 3A, 3B, 3C, the virtual path manager computing module 108 can generate the virtual paths 402a, 402b, 402c, 402d, 402e, 402f, 402g, 402h (collectively referred to as virtual paths 402). The virtual paths 402c, 402d, 402e, 402f can be based on the physical segment 210a; the virtual path 402a can be based on the physical segment 210b; and the virtual paths 402b, 402g, and 402h can be based on the physical segment 210c. In some examples, the virtual paths 402 can be based on two of the physical segments 210, with the two physical segments 210 sharing a common point (or intersection).

[0016] Referring back to FIG. 1, the virtual path manager computing module 108 can display a particular virtual path 116 based on a location of the user 104 in the virtual environment 118. Specifically, the virtual path manager computing module 108 can provide for display on the GUI 119 a particular virtual path 116. For example, the display device computing module 120 can provide for display on the GUI the virtual path 402c. The virtual path manager computing module 108 can determine that a forward direction of travel of the user 104 is proximate to a boundary condition of a particular physical segment 114 that corresponds to the particular virtual path 116, i.e., based on received location data 130. For example, the virtual path manager computing module 108 can determine that a forward direction of travel of the user 104 is proximate to a boundary condition of the physical segment 210a that corresponds to the particular virtual path 402c that user is currently located in. Specifically, the user 104 can be moving toward point 410 in the virtual path 402c that corresponds with moving towards point 250 of the corresponding physical segment 210a, as shown in FIG. 3A, based on the location data 130. Point 250 of the physical segment 210a can correspond to a boundary condition of the physical segment 210a, e.g., a wall of the physical space 208.

[0017] In some examples, determining that that the forward direction of travel of the user 104 is proximate to the boundary condition of the particular physical segment 114 includes the virtual path manager computing module 108 determining that the user 104 is within a threshold distance of physical contact with the particular physical segment 114. For example, the virtual path manager computing module 108 can determine that, when the user is at point 250 of the physical segment 210a that corresponds to the virtual path 402c, the user 104 is within a threshold distance of a physical constraint (e.g., a physical wall) of the physical segment 210a.

[0018] The virtual path manager computing module 108, in response to determining that the forward direction of travel of the user 104 is proximate to the boundary condition of the particular physical segment 114, notifies the user 104 that a physical rotation of the user 104 is needed in order for the user 104 to travel beyond a point in the particular virtual path 116. For example, as the user 104 physically approaches point 250 within the physical segment 210a along the direction D1, the virtual path manager computing module 108 provides a notification 122 to the user 104 that a physical rotation of the user 104 is needed in order for the user 104 to travel beyond the point 410 in the virtual path 402c.

[0019] In some examples, in response to determining that the forward direction of travel of the user 104 is proximate to the boundary condition of the particular physical segment 114, the virtual path manager computing module 108 can provide for display on the GUI 119 a GUI element within the particular virtual path 116 that indicates that the physical rotation is needed. In some examples, in response to determining that that the forward direction of travel of the user 104 is proximate to the boundary condition of the particular physical segment 114, the virtual path manager computing module 108 can provide an audio notification that indicates that the physical rotation is needed. In some examples, the virtual path manager computing module 108 can generate the GUI element in response to determining that the forward direction of travel of the user 104 is proximate to the boundary condition of the particular physical segment 114. In some examples, the virtual path manager computing module 108 can generate the GUI element prior to determining that the forward direction of travel of the user 104 is proximate to the boundary condition of the particular physical segment 114. That is, the location within the virtual environment 118 of the GUI elements can be predetermined when initially generating the virtual environment 118.

[0020] The virtual path manager computing module 108 can detect that physical rotation of the user 104 is complete. Specifically, the virtual path computing module 108 can receive rotation data 132 indicating a physical rotation status of the user 104. For example, the rotational data 132 can indicate that the user 104 has rotated at point 250 of the physical segment 210a as shown in FIG. 3A such that the user 104 is facing towards the point 252 of the physical segment 210 (and away from the boundary condition at point 250) along the direction D2. The virtual path manager computing module 108, in response to detecting that physical rotation of the user 104 is complete, can update the GUI 119 to show the virtual path 402d and allow the user 104 to travel beyond the point 410. That is, the user 104 is allowed to travel beyond the point 410 along the virtual path 402d that corresponds to physical segment 210a and towards point 252 of the physical segment 210a.

[0021] In some examples, the virtual path manager computing module 108, in response to detecting that physical rotation of the user 104 is complete, can update the GUI 119 to show the virtual path 402c and allow the user 104 to travel beyond the point 410, e.g., towards point 412. That is, the user 104 is allowed to travel beyond the point 410 and towards point 412 along the virtual path 402c that corresponds to physical segment 210a and towards point 252 of the physical segment 210a.

[0022] In some examples, the virtual path manager computing module 108, in response to detecting that physical rotation of the user 104 is complete, can update the GUI 119 to show a different virtual path 116 that is opposite the particular virtual path 116.

[0023] In some examples, the notification 122 can indicate to the user 104 to physically rotate 90 degrees. That is, after physical rotation of the user 104, the rotational data 132 can indicate that the user 104 has rotated at point 250 of the physical segment 210a as shown in FIG. 3A such that the user 104 is facing along the direction D3. The virtual path computing module 108 can determine that a further physical rotation of the user 104 is needed in order for the user 104 to travel beyond a point in the particular virtual path 116. The virtual path computing module 108 can provide a further notification to the user 104 that a further physical rotation of the user 104 is needed for the user 104 to travel beyond the point 410 in the virtual path. The virtual path manager computing module 108 can detect that further physical rotation of the user 104 is complete. For example, the rotational data 132 can indicate that the user 104 has rotated at point 250 of the physical segment 210a as shown in FIG. 3A such that the user 104 is facing towards the point 252 of the physical segment 210 along the direction D2. The virtual path manager computing module 108, in response to detecting that physical rotation of the user 104 is complete, can update the GUI 119 to show the virtual path 402d and allow the user 104 to travel beyond the point 410. That is, the user 104 is allowed to travel beyond the point 410 along virtual path 402d that corresponds to physical segment 210a and towards point 252 of physical segment 210a.

[0024] In some examples, the notification 122 can indicate to the user 104 to physically rotate 180 degrees. That is, the virtual path manager computing module 108 can detect that physical rotation of the user 104 is complete such that the rotational data 132 can indicate that the user 104 has rotated at point 250 of the physical segment 210a as shown in FIG. 3A such that the user 104 is facing towards the point 252 of the physical segment 210 (and away from the boundary condition at point 250) along the direction D2.

[0025] In some examples, during detection of the physical rotation of the user 104, the virtual path manager computing module 108 updates the GUI 119 to display the virtual environment 118 such that the GUI 119 correlates with the physical rotation of the user 104. Specifically, the virtual path manager computing module 108 can update the GUI 119 such that a virtual scene of the virtual environment 118 is rotated that correlates with the physical rotation of the user 104. That is, the virtual environment 119 can rotate correspondingly with the physical rotation of the user 104. For example, the rotation of the virtual scene of the virtual environment 118 can include rotation of the virtual scene by 90 degrees. For example, the rotation of the virtual scene of the virtual environment 118 can include rotation of the virtual scene by 180 degrees.

[0026] In some examples, during detection of the physical rotation of the user 104, the virtual path manager computing module 108 updates the GUI 119 such that movement of the virtual environment 118 is frozen.

[0027] In some examples, after the virtual path manager computing module 108 detects that physical rotation of the user 104 is compete, the virtual path manager computing module 108 can rotate the virtual environment 118 in a direction opposite to the rotation of the virtual environment 118 that correlates with the physical rotation of the user 104. That is, the virtual environment 118 can rotate correspondingly with the physical rotation of the user 104, and after the physical rotation of the user 104 is complete, the virtual environment 118 can rotate in a direction opposite to the initial rotation (e.g., rotate backwards). In some examples, the virtual path manager computing module 108, after the physical rotation of the user 104 is complete, rotates the virtual environment 118 in the direction opposite to the initial rotation such that the GUI 119 displays an orientation of the virtual environment 118 about the particular virtual path 116 similar to that prior to rotation of the virtual environment 118. In some examples, the virtual path manager computing module 108, after the physical rotation of the user 104 is complete, rotates the virtual environment 118 in the direction opposite to the initial rotation for approximately 0.3 seconds. In some examples, the virtual path manager computing module 108, after the physical rotation of the user 104 is complete, rotates the virtual environment 118 in the direction opposite to the initial rotation for less than 0.3 seconds.

[0028] FIG. 5 illustrates an example method 500 for generation of virtual paths in a virtual environment based on a physical environment. The method may begin at step 510, wherein the virtual path manager computing module 108 segments a layout of the physical space 102 surrounding the user 104 into physical segments 114. At step 520, the virtual path manager computing module 108 generates, based on the physical segments 114, virtual paths 116 through which the user 104 can navigate by traveling the physical segments 114. At 530, the virtual path manager computing module 108 displays a particular virtual path 116 based on a location of the user 104 in the virtual environment 118. At step 540, the virtual path manager computing module 108 determines that a forward direction of travel of the user 104 is proximate to a boundary condition of a particular physical segment 114 that corresponds to a particular virtual path 116. At 550, the virtual path manager computing module 108, in response to the determining, provides a notification 122 to the user 104 that a physical rotation of the user 104 is needed in order for the user 104 to travel beyond a point in the particular virtual path 116. At step 560, the virtual path manager computing module 108 detects that the physical rotation of the user 104 is complete (via the rotation data 132), and in response, updates the display (e.g., the GUI 119) to show the particular virtual path 116 and allows the user 104 to travel beyond the point in the particular virtual path 116. Although this disclosure describes and illustrates particular steps of the method of FIG. 5 as occurring in a particular order, this disclosure contemplates any suitable steps of the method of FIG. 5 occurring in any suitable order. Moreover, although this disclosure describes and illustrates an example method for generation of virtual paths in a virtual environment based on a physical environment including the particular steps of the method of FIG. 5, this disclosure contemplates any suitable method for generation of virtual paths in a virtual environment based on a physical environment including any suitable steps, which may include all, some, or none of the steps of the method of FIG. 5, where appropriate. Furthermore, although this disclosure describes and illustrates particular components, devices, or systems carrying out particular steps of the method of FIG. 5, this disclosure contemplates any suitable combination of any suitable components, devices, or systems carrying out any suitable steps of the method of FIG. 5.

[0029] FIG. 6 illustrates an example computer system 600. In particular embodiments, one or more computer systems 600 perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems 600 provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems 600 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems 600. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate.

[0030] This disclosure contemplates any suitable number of computer systems 600. This disclosure contemplates computer system 600 taking any suitable physical form. As example and not by way of limitation, computer system 600 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented/virtual reality device, or a combination of two or more of these. Where appropriate, computer system 600 may include one or more computer systems 600; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 600 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example and not by way of limitation, one or more computer systems 600 may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems 600 may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

[0031] In particular embodiments, computer system 600 includes a processor 602, memory 604, storage 606, an input/output (I/O) interface 608, a communication interface 610, and a bus 612. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0032] In particular embodiments, processor 602 includes hardware for executing instructions, such as those making up a computer program. As an example and not by way of limitation, to execute instructions, processor 602 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 604, or storage 606; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 604, or storage 606. In particular embodiments, processor 602 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor 602 including any suitable number of any suitable internal caches, where appropriate. As an example and not by way of limitation, processor 602 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory 604 or storage 606, and the instruction caches may speed up retrieval of those instructions by processor 602. Data in the data caches may be copies of data in memory 604 or storage 606 for instructions executing at processor 602 to operate on; the results of previous instructions executed at processor 602 for access by subsequent instructions executing at processor 602 or for writing to memory 604 or storage 606; or other suitable data. The data caches may speed up read or write operations by processor 602. The TLBs may speed up virtual-address translation for processor 602. In particular embodiments, processor 602 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 602 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor 602 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 602. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0033] In particular embodiments, memory 604 includes main memory for storing instructions for processor 602 to execute or data for processor 602 to operate on. As an example and not by way of limitation, computer system 600 may load instructions from storage 606 or another source (such as, for example, another computer system 600) to memory 604. Processor 602 may then load the instructions from memory 604 to an internal register or internal cache. To execute the instructions, processor 602 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor 602 may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor 602 may then write one or more of those results to memory 604. In particular embodiments, processor 602 executes only instructions in one or more internal registers or internal caches or in memory 604 (as opposed to storage 606 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 604 (as opposed to storage 606 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple processor 602 to memory 604. Bus 612 may include one or more memory buses, as described below. In particular embodiments, one or more memory management units (MMUs) reside between processor 602 and memory 604 and facilitate accesses to memory 604 requested by processor 602. In particular embodiments, memory 604 includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory 604 may include one or more memories 604, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

[0034] In particular embodiments, storage 606 includes mass storage for data or instructions. As an example and not by way of limitation, storage 606 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage 606 may include removable or non-removable (or fixed) media, where appropriate. Storage 606 may be internal or external to computer system 600, where appropriate. In particular embodiments, storage 606 is non-volatile, solid-state memory. In particular embodiments, storage 606 includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage 606 taking any suitable physical form. Storage 606 may include one or more storage control units facilitating communication between processor 602 and storage 606, where appropriate. Where appropriate, storage 606 may include one or more storages 606. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

[0035] In particular embodiments, I/O interface 608 includes hardware, software, or both, providing one or more interfaces for communication between computer system 600 and one or more I/O devices. Computer system 600 may include one or more of these I/O devices, where appropriate. One or more of these I/O devices may enable communication between a person and computer system 600. As an example and not by way of limitation, an I/O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these. An I/O device may include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces 608 for them. Where appropriate, I/O interface 608 may include one or more device or software drivers enabling processor 602 to drive one or more of these I/O devices. I/O interface 608 may include one or more I/O interfaces 608, where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface.

[0036] In particular embodiments, communication interface 610 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 600 and one or more other computer systems 600 or one or more networks. As an example and not by way of limitation, communication interface 610 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface 610 for it. As an example and not by way of limitation, computer system 600 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system 600 may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these. Computer system 600 may include any suitable communication interface 610 for any of these networks, where appropriate. Communication interface 610 may include one or more communication interfaces 610, where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface.

[0037] In particular embodiments, bus 612 includes hardware, software, or both coupling components of computer system 600 to each other. As an example and not by way of limitation, bus 612 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus 612 may include one or more buses 612, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

[0038] Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (such, as for example, field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.

[0039] FIG. 7 illustrates an example artificial reality system 700A. In particular embodiments, the artificial reality system 700A may comprise a headset 704, a controller 706, and a computing system 708, etc. A user 702 may wear the headset 704 that could display visual artificial reality content to the user 702. The headset 704 may include an audio device that could provide audio artificial reality content to the user 702. The headset 704 may include one or more cameras which can capture images and videos of environments. The headset 704 may include an eye tracking system to determine the vergence distance of the user 702. The headset 704 may be referred as a head-mounted display (HDM). The controller 706 may comprise a trackpad and one or more buttons. The controller 706 may receive inputs from the user 702 and relay the inputs to the computing system 708. The controller 706 may also provide haptic feedback to the user 702. The computing system 708 may be connected to the headset 704 and the controller 706 through cables or wireless connections. The computing system 708 may control the headset 704 and the controller 706 to provide the artificial reality content to and receive inputs from the user 702. The computing system 708 may be a standalone host computer system, an on-board computer system integrated with the headset 704, a mobile device, or any other hardware platform capable of providing artificial reality content to and receiving inputs from the user 702.

[0040] Herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A and B” means “A and B, jointly or severally,” unless expressly indicated otherwise or indicated otherwise by context.

[0041] The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, feature, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative. Additionally, although this disclosure describes or illustrates particular embodiments as providing particular advantages, particular embodiments may provide none, some, or all of these advantages.