Apple Patent | Head-Mounted Display With Low Light Operation

Patent: Head-Mounted Display With Low Light Operation

Publication Number: 20200341563

Publication Date: 20201029

Applicants: Apple

Abstract

A display system includes a controller and a head-mounted display. The head-mounted display includes a display, a head support coupled to the display for supporting the display on a head of a user to be viewed by the user, and sensors coupled to the head support for sensing an environment from the head-mounted display unit in low light. The sensors include one or more of an infrared sensor for sensing the environment with infrared electromagnetic radiation, or a depth sensor for sensing distances to objects of the environment, and also include an ultrasonic sensor for sensing the environment with ultrasonic sound waves. The controller determines graphical content according to the sensing of the environment with the one or more of the infrared sensor or the depth sensor and with the ultrasonic sensor, and operates the display to provide the graphical content concurrent with the sensing of the environment.

CROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 62/838,995, filed Apr. 26, 2019, the entire disclosure of which is incorporated by reference herein.

TECHNICAL FIELD

[0002] This disclosure relates to display systems and, in particular, head-mounted display units and operation thereof.

BACKGROUND

[0003] Human eyes have different sensitivity in different lighting conditions. Photopic vision is human vision with high levels of ambient light (e.g., luminance of approximately 10 to 10{circumflex over ( )}8 cd/m{circumflex over ( )}2), such as daylight conditions. Photopic vision is provided by cone cells of the eye that provide sensitivity to different colors (i.e., wavelengths) of light. Scotopic vision is human vision with low levels of ambient light (e.g., luminance of approximately 10{circumflex over ( )}-6 to 10{circumflex over ( )}-3.5 cd/m{circumflex over ( )}2), such as at night with overcast skies (e.g., with no moonlight). Scotopic vision is provided by rod cells of the eye. Mesopic vision is human vision with levels of ambient light between those for photopic vision and scotopic vision (e.g., luminance of approximately 10{circumflex over ( )}-3 to 10{circumflex over ( )}0.5 cd/m{circumflex over ( )}2), such as at night without overcast skies (e.g., with moonlight) to early twilight times. Mesopic vision is provided by both the cone cells and the rod cells. As compared to photopic vision, scotopic vision or even mesopic vision may result in a loss of color vision, changing sensitivity to different wavelengths of light, reduced acuity, and more motion blur. Thus, in poorly lit conditions, such as when relying on scotopic vision, a person is less able to view the environment than in well lit conditions.

SUMMARY

[0004] Disclosed herein are implementations of display systems, including head-mounted display units and methods of providing content. In an implementation, a display system includes a controller and a head-mounted display unit. The head-mounted display unit includes a display, a head support coupled to the display for supporting the display on a head of a user to be viewed by the user, and sensors coupled to the head support for sensing an environment from the head-mounted display unit in low light. The sensors include one or more of an infrared sensor for sensing the environment with infrared electromagnetic radiation, or a depth sensor for detecting distances to objects of the environment, and also include an ultrasonic sensor for sensing the environment with ultrasonic sound waves. The controller determines graphical content according to the sensing of the environment with the one or more of the infrared sensor or the depth sensor and with the ultrasonic sensor, and operates the display to provide the graphical content concurrent with the sensing of the environment.

[0005] In an implementation, a display system includes a controller, and a head-mounted display unit. The head-mounted display unit includes a display for displaying graphical content to a user wearing the head-mounted display unit and sensors for sensing an environment from the head-mounted display unit. The sensors include an infrared sensor, a depth sensor, an ultrasonic sensor, and a visible light camera. In high light conditions, the sensors sense the environment to obtain first sensor data that is stored in a storage. The first sensor data includes first visible light sensor data obtained with the visible light camera and first non-visible light sensor data obtained from one or more of the infrared sensor, the depth sensor, or the ultrasonic sensor. In low light conditions after the first sensor data is stored, the sensors sense the environment to obtain current sensor data, and the controller determines the graphical content according to the current sensor data and first visible light sensor data.

[0006] In an implementation, a method of providing graphical content with a display system includes sensing an environment, processing sensor data, determining graphical content, and outputting the graphical content. The sensing includes sensing with sensors an environment to obtain sensor data in low light. The sensors are coupled to a head-mounted display unit of the display system and include an infrared sensor, a depth sensor, and an ultrasonic sensor. The processing includes processing the sensor data with a controller. The determining of the graphical content is performed with the controller according to the processing. The graphical content includes an ultrasonic graphical component and one or more of an infrared graphical component based on the sensor data obtained with the infrared sensor, a depth graphical component based on the sensor data obtained with the depth sensor, or a combined graphical component based on the sensor data obtained with both the infrared sensor and the depth sensor. The outputting of the graphical content is performed with a display of the head-mounted display unit.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 is a top view of a head-mounted display unit on a head H of a user.

[0008] FIG. 2 is a schematic view of the head-mounted display unit.

[0009] FIG. 3 is a schematic view of an example hardware configuration of a controller of the head-mounted display unit.

[0010] FIG. 4 is a schematic view of sensors of the head-mounted display unit.

[0011] FIG. 5 is a block diagram of a display system.

[0012] FIG. 6 is a flowchart of a process for providing graphical content with the display system of FIG. 5.

[0013] FIG. 7 is a block diagram of a display system.

[0014] FIG. 8 is a flowchart of a process for determining graphical content with the display system of FIG. 7.

[0015] FIG. 9 is a block diagram of a display system.

[0016] FIG. 10 is a flowchart of another process for determining graphical content with the display system of FIG. 9.

[0017] FIG. 11 is a block diagram of a display system.

[0018] FIG. 12 is a flowchart of another process for providing graphical content with the display system of FIG. 11.

DETAILED DESCRIPTION

[0019] Disclosed herein are embodiments of head-mounted display units that are configured for operating in low-light conditions, such as at night and/or when a user might otherwise use scotopic or mesopic vision to view the environment. More particularly, the head-mounted display units include one or more sensors configured to observe the environment and/or to detect objects in low-light conditions, which may include one or more of an infrared sensor, a depth sensor, and/or an ultrasound sensor. The head-mounted display unit provides content according to the sensors, which may include providing graphical content (e.g., displaying one or more of stored images, renderings of objects, graphical indicators).

[0020] Referring to FIGS. 1 and 2, a display system 100 includes a head-mounted display unit 102. The display system 100 may be configured to provide a computer generated reality, as discussed below. The display system 100 generally includes a head support 110, one or more displays 120, and one or more sensors 130 coupled the head support 110, which cooperatively form the head-mounted display unit 102. The head support 110 may, for example, include a chassis 112 and a head-engagement mechanism 114 coupled to the chassis 112. The one or more displays 120 and the one or more sensors 130 are coupled to the chassis 112, while the head-engagement mechanism 114 engages the head H of the user for supporting the displays 120 for displaying graphical content to eyes of the user. The one or more displays 120 may each be configured as a display panel (e.g., a liquid crystal display panel (LCD), light-emitting diode display panel (LED), organic light-emitting diode display panel (e.g., OLED)), or as a projector (e.g., that projects light onto a reflector back to the eyes of the user), and may further be considered to include any associated optical components (e.g., lenses or reflectors). The sensors 130 are configured to sense the environment and are discussed below with reference to FIG. 4.

[0021] The display system 100 further includes a controller 140 and other electronics 150. The controller 140 and the other electronics 150 may be coupled to the head-mounted display unit 102 (e.g., to the chassis), be provided separate from and operatively connectable to the head-mounted display unit 102 (e.g., wired or wirelessly to transfer signals and/or power therebetween), or be partially coupled to the head-mounted display unit 102 (e.g., with various components of the controller 140 and/or the electronics 150 being coupled to the head-mounted display unit 102 and other components thereof being operatively connectable thereto). The controller 140 controls various operations of the display system 100, for example, sensing various conditions with the sensors 130 and providing content with the display 120 according thereto. An example hardware configuration for the controller 140 is discussed below with reference to FIG. 3. The other electronics 150 may include, for example, power electronics (e.g., a battery), communications devices (e.g., modems and/or radios for communicating wirelessly with other devices), and/or other output devices (e.g., speakers for aural output, haptic devices for tactile output).

[0022] Referring to FIG. 3, the controller 140 is a computing device configured to implement the systems and methods described herein. The controller 140 generally includes a processor 342, a memory 344, a storage 346, a communications interface 348, and a bus 349 allowing communication therebetween. The processor 342 may be any suitable processor, such as a central processing unit (CPU) configured to execute instructions. The memory 344 is a short-term volatile memory (e.g., random access memory module). The storage 346 is a long-term, non-volatile memory that stores software programming containing the instructions executed by the processor 342 (e.g., a hard disk or solid-state drive). The communications interface 348 sends and receives signals from and to the controller 140, such as from and to other electronic components of the display system 100 (e.g., the displays 120, the sensors 130, and/or the other electronics 150). While the controller 140 is illustrated as a singular device, various of the components thereof may be provided in any suitable manner, and the controller 140 may include fewer and/or more components. For example, the controller 140 may be considered to include processing devices or other electronic hardware particularly associated with each of the sensors.

[0023] Referring to FIG. 4, the sensors 130 include one or more infrared sensors 432, one or more depth camera sensors 434, and/or one or more ultrasonic sensors 436, which face outward from the head-mounted display unit 102 to observe the environment E. The sensors 130 also include one or more visible light cameras 438 that face outward from the head-mounted display unit 102 to observe the environment, as well as one or more movement sensors 440 that detects the position, orientation, and/or movement of the head-mounted display unit 102.

[0024] The infrared sensor 432 detects infrared light in the environment. The infrared sensor 432 may be any suitable type of infrared sensor for detecting electromagnetic radiation in the infrared frequency ranges. In one example, the infrared sensor 432 is an infrared camera, which captures images (e.g., video) in the infrared frequency ranges using an image sensor, such as a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) sensor, and other suitable hardware components (e.g., an image processor). Images captured by the infrared sensor 432 may be referred to as IR images. The infrared sensor 432 may be a passive sensor that observes infrared radiation emitted or reflected by objects in the environment. Alternatively, the infrared sensor 432 may include an infrared illuminator, which emits electromagnetic radiation in the infrared frequency range to thereby illuminate the environment and objects therein.

[0025] Information about the environment obtained and/or derived from (e.g., after processing) the infrared sensor 432 may be referred to as infrared sensor data 552. As discussed in further detail below, the infrared sensor data 552 may be processed, stored, and/or used to determine graphical content in different manners.

[0026] The depth sensor 434 detects the environment and, in particular, detects the depth (e.g., distance) therefrom to objects of the environment. The depth sensor 434 generally includes an illuminator 434a and a detector 434b. The illuminator 434a emits electromagnetic radiation (e.g., infrared light) from the head-mounted display unit 102 into the environment. The detector 434b observes the electromagnetic radiation reflected off objects in the environment. In two specific examples, the depth sensor 434 is a depth camera that uses structured light or time of flight. In the case of the depth sensor 434 being a structured light sensor, the illuminator 434a projects a pattern of electromagnetic radiation in the infrared frequency ranges (e.g., a grid or array of infrared dots, such as tens of thousands of dots), while the detector 434b is a camera that captures images of the pattern of the infrared light as reflected by objects in the environment, which may be referred to as structured light images. The structured light images are then analyzed (e.g., by the controller 140) to determine the depth (e.g., distances) from the depth sensor 434 to the objects of the environment (or points thereon), for example, by analyzing deformation of the light pattern as reflected off the objects. The detector 434b may be a camera that includes an image sensor, such as a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) sensor, and other suitable hardware components (e.g., an image processor). In the case of the depth sensor 434 being a time-of-flight camera, the illuminator 434a projects electromagnetic radiation, while the detector 434b is a suitable camera according to which the time of flight is measured of the electromagnetic radiation emitted from the illuminator 434a as reflected off the object to determine distances thereto. The depth sensor 434 may operate in different frequency ranges of the electromagnetic radiation spectrum than the infrared sensor 432, so as to not detect or otherwise be sensitive to electromagnetic radiation of the other (e.g., using appropriate filters, camera image sensors, and/or illuminators and the projector 434a in suitable frequency ranges). In other examples, the depth sensor 434 may be a radar detection and ranging sensor (RADAR) or a light detection and ranging sensor (LIDAR). It should be noted that one or multiple types of depth sensors 434 may be utilized, for example, incorporating one or more of a structured light sensor, a time-of-flight camera, a RADAR sensor, and/or a LIDAR sensor. In one preferred embodiment, the depth sensors 434 include only the structured light sensor.

[0027] Information about the environment obtained and/or derived from (e.g., after processing) the depth sensor 434 may be referred to as depth sensor data 554. As discussed in further detail below, the depth sensor data 554 may be processed, stored, and/or used to determine graphical content in different manners.

[0028] The ultrasonic sensor 436 detects the environment using ultrasonic sound waves. The ultrasonic sensor 436 may, for example, include an ultrasonic transmitter that transmits ultrasonic sound waves and an ultrasonic receiver that detects those ultrasonic sound waves reflected by physical objects in the environment, or may alternatively use an include an ultrasonic transceiver that performs the function of both the ultrasonic transmitter and the ultrasonic receiver. The ultrasonic sound waves are then processed, such as by the controller 140 or another suitable processor, to identify and/or locate features of an environment. Advantageously, the ultrasonic sensor 436 may detect objects that are otherwise not observable with the infrared sensors 432 and/or the depth sensor 434, such as a sheet of glass (e.g., of a window or door).

[0029] Information about the environment obtained and/or derived from (e.g., after processing) the ultrasonic sensor 436 may be referred to as ultrasonic sensor data 556. As discussed in further detail below, the ultrasonic sensor data 556 may be processed, stored, and/or used to determine graphical content in different manners.

[0030] The one or more visible light cameras 438 detects visible light in the environment. The visible light cameras 438 includes an image sensor, such as a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) sensor, and other suitable hardware components (e.g., an image processor) and optical components (e.g., lenses).

[0031] Information about the environment obtained and/or derived from (e.g., after processing) the visible light camera 438 may be referred to as visible light sensor data 558. The visible light sensor data 558 may include images (e.g., video), which may be referred as visible light images. As discussed in further detail below, the visible light sensor data 558 may be processed, stored, and/or used to determine graphical content in different manners.

[0032] One or multiple (e.g., two) of each of the infrared sensors 432, the depth sensors 434, the ultrasonic sensor 436, and/or the visible light camera 438 may be coupled to the head-mounted display unit 102, for example, to sense the environment E stereoscopically from the head-mounted display unit 102. The one or more infrared sensors 432, the depth sensors 434, the ultrasonic sensor 436, and/or the visible light camera 438 may provide the same or different fields of view as each other, which are generally represented in FIG. 1 by the dashed arrows emanating from the sensors 130. In the systems and processes discussed below, the various types of sensors are referred to singularly, though it should be understood that the systems and processes may be applied to multiple such sensors (e.g., to determine and provide graphical content stereoscopically).

[0033] The one or more movement sensors 440 detects the position, orientation, and/or movement of the head-mounted display unit 102. The one or more movement sensors 440 may, for example, include a global positioning system sensor (e.g., GPS), one or more accelerometers, one or more gyroscopes, and/or an inertial measurement unit (IMU), which function to determine the position, orientation, and/or movement of the head-mounted display unit 102.

[0034] Information obtained and/or derived from (e.g., after processing) the movement sensor 440 may be referred to as movement sensor data 560. As discussed in further detail below, the movement sensor data 560 may be processed, stored, and/or used to determine graphical content in different manners.

[0035] Referring to FIG. 5, the display system 100 outputs graphical content to the user, which enhances the user’s ability to see in low light conditions. Low light conditions may be defined, for example, according to ambient light conditions (e.g., luminance less than 10{circumflex over ( )}-3.5 cd/m{circumflex over ( )}2) and/or the type of vision that the human would otherwise use to directly observe the environment (e.g., when scotopic vision is predominantly or substantially only used by the human). The graphical content may, for example, include images that enhance contrast between objects of the environment, renderings of detected objects or other indicators, and/or captured images. Renderings are computer-generated graphical reproductions of the detected objects, which are generated according to sensor information. Renderings may be accurate reproductions that accurately depicted the detected object (e.g., with corresponding textures, colors, sizes, etc.), may be characterized reproductions that may alter various features of the detected object (e.g., changing to a “cartoon” form with uniform and/or different colors, different textures, contrasting outlines), or may be a highlighting rendering (e.g., overlaying an object). As an illustrative and non-limiting example of a characterized reproduction, a wood table may have an accurate reproduction that depicts the varied color and graining of the wood and shading to emphasize different surfaces, or a characterized reproduction that depicts a uniform color and black outlines between surfaces. A highlighting rendering, for example, might include highlighting the table in green by providing a colored translucent outline over other graphical content of the table. Renderings may be of the environment (e.g., objects or structures that define an environment, such as walls of a room) or objects within the environment.

[0036] The infrared sensor 432, the depth sensor 434, the ultrasonic sensor 436, and the visible light camera 438 sense the environment E from the head-mounted display unit 102, while the movement sensor 440 senses the position, orientation, and/or movement of the head-mounted display unit 102 on the head of the user. The sensor data, including the infrared sensor data 552, the depth sensor data 554, the ultrasonic sensor data 556, the visible light sensor data 558, and/or the movement sensor data 560, is sent from the respective sensors to the controller 140 with one or more sensor signals 550.

[0037] The controller 140 receives the sensor signals 550, processes the sensor data received thereby, determines the graphical content according to the sensor data, and in turn sends one or more signals to the display 120, which may be referred to as a display signal 570. The display 120 receives the display signal 570 and outputs the graphical content. As referenced above, the controller 140 may include one or more local or distributed processing devices, which process the sensor data and/or determine the graphical content, such as processors that may be associated with the different sensors 130. The processing and/or determining may, however, be performed by different processors and/or different controllers.

[0038] Referring to FIG. 6, a process 600 provides graphical content with a display system that includes a head-mounted display unit, such as the display system 100 and the head-mounted display unit 102. The process 600 generally includes sensing 610 the environment in low light with sensors to generate sensor data, processing 620 the sensor data, determining 630 graphical content, and outputting 640 the graphical content. The process 600 may also include another operation 615 of sensing movement of the head-mounted display unit.

[0039] The sensing 610 of the environment to generate the sensor data is performed with sensors, such as the sensors 130, which face outward from the head-mounted display unit, such as the head-mounted display unit 102. The sensors include, for example, one or more of the infrared sensor 432, the depth sensor 434, the ultrasonic sensor 436, and/or the visible light camera 438. The sensor data, such as the infrared sensor data 552, the depth sensor data 554, the ultrasonic sensor data 556, the visible light sensor data 558, and/or the movement sensor data 560, is sent to a processor, such as the processor 342 of the controller 140, via one or more signals, such as the sensor signal 550. Different combinations of sensor data may be obtained and/or sent. For example, the visible light camera 448 and/or the movement sensors 450 may omitted and/or not operated.

[0040] The processing 620 of the sensor data is performed with a processor, for example, with the processor 342 of the controller 140 and/or other processing devices particularly associated with one or more of the sensors (e.g., an image processor associated with a camera). Processing the sensor data may be performed in various manners as discussed in further detail below with respect to the systems and processes in FIGS. 7-12.

[0041] The determining 630 of the graphical content is performed, for example, with a processor, such as the processor 342 of the controller 140 and/or other processing devices particularly associated with the display 120. Determining the graphical content may be performed in various manners as discussed in further detail below with respect to the systems and processes in FIGS. 7-12. The processing 620 and the determining 630 may be performed as a singular operation, for example, when simply converting infrared light into visible light. The graphical content is sent, for example, via the display signal 570.

[0042] The outputting 640 (e.g., display) of the graphical content is performed by one or more displays, such as the display 120, according to the display signal 570 received from the controller 140. The outputting 640 is performed substantially concurrent with the sensing 610, such that the graphical content is displayed concurrent (e.g., in real-time) with the environment sensed by the head-mounted display. Concurrent or substantially concurrent should be understood to account for latency of the display system 100 associated with sensing, processing, determining, and/or transmitting operations.

[0043] The sensing 610, the processing 620, the determining 630, and the outputting 640 are repeated, so as to provide the user with the graphical content as a stream of images (e.g., a video stream).

[0044] Referring to FIGS. 7 and 8, the display system 100 processes the sensor data from each of the sensors independent of each other and may further determine graphical content components independent of each other. For example, the display system 100 may display graphical content that includes an infrared graphical component 772 based on the infrared sensor data 552 from the infrared sensor 432, a depth graphical component 774 based on the depth sensor data 554 from the depth sensor 434, an ultrasonic graphical component 776 based on the ultrasonic sensor data 556 from the ultrasonic sensor 436, and/or a visible light graphical component 778 based on the visible light sensor data 558 from the visible light camera 438. The infrared graphical component 772, the depth graphical component 774, the ultrasonic graphical component 776, and/or the visible light graphical component 778 may be displayed simultaneously and concurrent with detection of the associated information, for example, being overlaid each other with suitable transparency for viewing of each of the components. The graphical content is displayed substantially concurrent with the sensing thereof, such that the user may observe the environment in real time via the graphical content. Aural content may also be provided (e.g., output by speakers of the other electronics 350), for example, based on the ultrasonic graphical component 776 based on the ultrasonic sensor data 556 from the ultrasonic sensor 436 to indicate distance to an object.

[0045] The infrared sensor data 552, such as the IR images captured with the infrared sensor 432, may be processed in various manners for determining the graphical content. In one example, the infrared sensor data 552 is processed to convert the infrared images from infrared light to visible light that forms the infrared graphical component 772. Instead or additionally, the infrared sensor data 552 is processed to enhance contrast of the infrared images. Instead or additionally, the infrared sensor data 552 is processed to detect objects of the environment using, for example, suitable computer vision or other object detection programming or algorithms. Detecting may include locating, characterizing, and/or identifying objects, or other object recognition functions related to the environment. Locating generally refers to determining a position of objects or features thereof in a real coordinate system and/or relative to the head-mounted display unit 102. Characterizing generally refers to determining characteristics of the physical object, such as size, shape, and/or color. Identifying generally refers to identifying a type of object (e.g., a door, wall, chair) or uniquely identifying a particular object (e.g., a door in a certain room of a certain house).

[0046] The infrared graphical component 772 of the graphical content may, for example, include the IR image converted to visible light and/or with enhanced. Instead or additionally, the infrared graphical component 772 may include other graphics, such as renderings generated according to the infrared sensor data 552.

[0047] The depth sensor data 554, such as the structured light images captured with the depth sensor 434, may be processed in various manners for determining the graphical content and, in particular, to determine distances from the depth sensor 434 on the head-mounted display unit 102 to locations (e.g., points) on objects of the environment. Such distances may be represented by a depth map, which is a mathematical and/or visual representation of such distances. The depth map, or other information derived from the depth sensor 434, may be further processed (e.g., analyzed) to detect (e.g., locate, characterize, and/or identify) objects in the environment. The depth map, or other depth information, may also be processed to determine the relative position, orientation, and/or movement of the head-mounted display unit 102 relative to the environment and/or the relative position, orientation, and/or movement of objects of the environment.

[0048] The depth graphical component 774 of the graphical content may, for example, include the structured light images. Instead or additionally, the depth graphical component 774 includes renderings determined (e.g., generated) according to the depth sensor data 554, such as a rendering of the depth map itself, renderings of the environment, renderings of objects therein, and/or renderings to highlight detected objects.

[0049] The ultrasonic sensor data 556 detected with the ultrasonic sensor 436 is processed to determine the graphical or other content output by the head-mounted display unit 102. In particular, the ultrasonic sensor data 556 is processed to detect (e.g., locate, characterize, and/or identify) objects of the environment, such as by determining a distance from the head-mounted display unit 102 (e.g., the ultrasonic sensor 436 thereof) from the objects of the environment and/or relative movement therebetween.

[0050] The ultrasonic graphical component 776 of the graphical content may, for example, include a graphical representation of a distance to the detected object or a graphical representation of the detected object (e.g., a glass panel or wall). The ultrasonic graphical component 776 of the graphical content may instead include another graphical indicator, such as numerical or color indicator that indicates distance to the object. Instead or additionally, an aural ultrasonic indicator may be provided, such as a verbal indicator or sound indicator for indicating type and/or distance to the object.

[0051] The visible light sensor data 558 (e.g., the visible light images) detected with the visible light camera 438 may be processed in various manners for determining the graphical content. In one example, the visible images may be processed in a suitable manner for display to the user. Instead or additionally, the visible images may be processed to enhance contrast. Instead or additionally, the visible light sensor data is processed to detect (e.g., locate, characterize, and/or identify) physical objects of the environment, which may include physical features of an environment and/or physical objects of the environment, such as with object-recognition programming (e.g., computer vision software).

……
……
……