Sony Patent | Information processing apparatus, information processing method, and program

Patent: Information processing apparatus, information processing method, and program

Drawings: Click to check drawins

Publication Number: 20220004250

Publication Date: 20220106

Applicant: Sony

Assignee: Sony Group Corporation

Abstract

An information processing apparatus according to an embodiment of the present technology includes a display control section. The display control section performs, when a display size of a plurality of virtual objects is changed, a notification regarding switching of a change mode of the display size of a specific virtual object among the plurality of virtual objects on the basis of display size change information set for the specific virtual object. This makes it possible to exhibit a high degree of usability to display the virtual objects.

Claims

1. An information processing apparatus, comprising a display control section that performs, when a display size of a plurality of virtual objects is changed, a notification regarding switching of a change mode of the display size of a specific virtual object among the plurality of virtual objects on a basis of display size change information set for the specific virtual object.

2. The information processing apparatus according to claim 1, wherein the display control section performs the notification regarding the switching of the change mode at a timing corresponding to a timing at which the change mode of the display size of the specific virtual object is switched.

3. The information processing apparatus according to claim 2, wherein the timing at which the notification regarding the switching of the change mode is performed is the timing at which the change mode of the display size is switched or a predetermined timing before the change mode of the display size is switched.

4. The information processing apparatus according to claim 1, wherein the change of the display size includes at least one of a change of a display scale of the virtual object, a change of a length of the virtual object, or a change of an area of the virtual object.

5. The information processing apparatus according to claim 1, wherein the notification regarding the change mode includes at least one of highlighting of the specific virtual object or output of related information regarding the switching of the change mode.

6. The information processing apparatus according to claim 1, wherein the display control section stops the change of the display size at a timing at which the change mode of the display size is switched.

7. The information processing apparatus according to claim 1, wherein the display size change information includes a threshold value regarding the display size, the threshold value being a reference for switching the change mode of the display size.

8. The information processing apparatus according to claim 1, wherein the display size change information includes at least one of a maximum display size or a minimum display size.

9. The information processing apparatus according to claim 1, wherein each of the plurality of virtual objects is a three-dimensional model simultaneously displayed in a virtual space at a predetermined size ratio.

10. The information processing apparatus according to claim 9, wherein the display size change information includes information regarding switching between a first change mode in which the display size is changed at a first change rate and a second change mode in which the display size is changed at the second change rate different from the first change rate.

11. The information processing apparatus according to claim 10, wherein in the first change mode, the display size of the plurality of virtual objects is simultaneously changed with the predetermined size ratio being maintained, and in the second change mode, the change of the display size of the specific virtual object is restricted to a change at a size ratio different from the predetermined size ratio.

12. The information processing apparatus according to claim 1, wherein the display size change information includes restriction information regarding restriction on the change of the display size of another virtual object among the plurality of virtual objects.

13. The information processing apparatus according to claim 12, wherein the restriction information includes information as to whether the change of the display size of the other virtual object is restricted or not in response to the switching of the change mode of the display size.

14. The information processing apparatus according to claim 12, wherein the restriction information includes information as to whether the change of the display size of the other virtual object is restricted or not when the display size reaches the threshold value.

15. The information processing apparatus according to claim 1, further comprising an input section to which a user instruction is input, wherein the display control section changes the display size of the plurality of virtual objects on a basis of an instruction to change the display size that is input to the input section.

16. The information processing apparatus according to claim 1, wherein the display control section stops the change of the display size of the plurality of virtual objects at a timing at which the change mode of the display size of the specific virtual object is switched, and subsequently performs the change of the display size of the plurality of virtual objects again when an instruction to change the display size is input to the input section again.

17. The information processing apparatus according to claim 1, further comprising a generation section that generates the display size change information set for the specific virtual object, wherein the generation section generates the display size change information on a basis of at least one of user information regarding the user, environmental information, or apparatus information regarding a display apparatus that displays the specific virtual object.

18. The information processing apparatus according to claim 1, further comprising a sound control section that controls output of sound regarding each of the plurality of virtual objects, wherein the sound control section controls output of sound regarding the specific virtual object on a basis of the display size change information of the specific virtual object.

19. An information processing method that is performed by a computer system, the method comprising performing, when a display size of a plurality of virtual objects is changed, a notification regarding switching of a change mode of the display size of a specific virtual object among the plurality of virtual objects on a basis of display size change information of the specific virtual object.

20. A program that causes a computer system to perform the step of performing, when a display size of a plurality of virtual objects is changed, a notification regarding switching of a change mode of the display size of a specific virtual object among the plurality of virtual objects on a basis of display size change information of the specific virtual object.

Description

TECHNICAL FIELD

[0001] The present technology relates to an information processing apparatus, an information processing method, and a program that are applicable to display of virtual content such as virtual reality (VR) or augmented reality (AR).

BACKGROUND ART

[0002] In the display control device disclosed in Patent Literature 1, the position and pose of an imaging target are recognized from a captured image. A virtual object associated with the recognized imaging target is controlled for display in accordance with the imaging magnification of the imaging target. The ratio of the movement amount of the virtual object is controlled on the basis of the movement amount of the imaging target according to the imaging magnification. Thus, in the case of displaying the virtual object corresponding to the imaging position, usability for a user is improved (paragraphs [0047] and [0050] in the specification, FIGS. 5 and 6, etc. of Patent Literature 1).

CITATION LIST

Patent Literature

[0003] Patent Literature 1: Japanese Patent No. 6304238

DISCLOSURE OF INVENTION

Technical Problem

[0004] For the display of virtual objects in VR, AR, and the like, the technology capable of exhibiting a high degree of usability is expected.

[0005] In view of the circumstances described above, it is an object of the present technology to provide an information processing apparatus, an information processing method, and a program that are capable of exhibiting a high degree of usability to display virtual objects.

Solution to Problem

[0006] In order to achieve the object described above, an information processing apparatus according to an embodiment of the present technology includes a display control section.

[0007] The display control section performs, when a display size of a plurality of virtual objects is changed, a notification regarding switching of a change mode of the display size of a specific virtual object among the plurality of virtual objects on the basis of display size change information set for the specific virtual object.

[0008] In this information processing apparatus, when the display size is changed for a specific virtual object among a plurality of virtual objects, a notification regarding switching of a change mode of the display size of the specific virtual object is performed. This makes it possible to exhibit a high degree of usability to display the virtual objects.

[0009] The display control section may perform the notification regarding the switching of the change mode at a timing corresponding to a timing at which the change mode of the display size of the specific virtual object is switched.

[0010] The timing at which the notification regarding the switching of the change mode is performed may be the timing at which the change mode of the display size is switched or a predetermined timing before the change mode of the display size is switched.

[0011] The change of the display size may include at least one of a change of a display scale of the virtual object, a change of a length of the virtual object, or a change of an area of the virtual object.

[0012] The notification regarding the change mode may include at least one of highlighting of the specific virtual object or output of related information regarding the switching of the change mode.

[0013] The display control section may stop the change of the display size at a timing at which the change mode of the display size is switched.

[0014] The display size change information may include a threshold value regarding the display size, the threshold value being a reference for switching the change mode of the display size.

[0015] The display size change information may include at least one of a maximum display size or a minimum display size.

[0016] Each of the plurality of virtual objects may be a three-dimensional model simultaneously displayed in a virtual space at a predetermined size ratio.

[0017] The display size change information may include information regarding switching between a first change mode in which the display size is changed at a first change rate and a second change mode in which the display size is changed at the second change rate different from the first change rate.

[0018] In the first change mode, the display size of the plurality of virtual objects may be simultaneously changed with the predetermined size ratio being maintained. In this case, in the second change mode, the change of the display size of the specific virtual object may be restricted to a change at a size ratio different from the predetermined size ratio.

[0019] The display size change information may include restriction information regarding restriction on the change of the display size of another virtual object among the plurality of virtual objects.

[0020] The restriction information may include information as to whether the change of the display size of the other virtual object is restricted or not in response to the switching of the change mode of the display size.

[0021] The restriction information may include information as to whether the change of the display size of the other virtual object is restricted or not when the display size reaches the threshold value.

[0022] The information processing apparatus may further include an input section to which a user instruction is input. In this case, the display control section may change the display size of the plurality of virtual objects on the basis of an instruction to change the display size that is input to the input section.

[0023] The display control section may stop the change of the display size of the plurality of virtual objects at a timing at which the change mode of the display size of the specific virtual object is switched, and subsequently perform the change of the display size of the plurality of virtual objects again when an instruction to change the display size is input to the input section again.

[0024] The information processing apparatus may further include a generation section that generates the display size change information set for the specific virtual object. In this case, the generation section may generate the display size change information on the basis of at least one of user information regarding the user, environmental information, or apparatus information regarding a display apparatus that displays the specific virtual object.

[0025] The information processing apparatus may further include a sound control section that controls output of sound regarding each of the plurality of virtual objects. In this case, the sound control section may control output of sound regarding the specific virtual object on the basis of the display size change information of the specific virtual object.

[0026] An information processing method according to an embodiment of the present technology is an information processing method that is performed by a computer system, the method including performing, when a display size of a plurality of virtual objects is changed, a notification regarding switching of a change mode of the display size of a specific virtual object among the plurality of virtual objects on the basis of display size change information of the specific virtual object.

[0027] A program according to an embodiment of the present technology causes a computer system to perform the step of performing, when a display size of a plurality of virtual objects is changed, a notification regarding switching of a change mode of the display size of a specific virtual object among the plurality of virtual objects on the basis of display size change information of the specific virtual object.

Advantageous Effects of Invention

[0028] As described above, according to the present technology, it is an object of the present technology to exhibit a high degree of usability to display virtual objects. Note that the effect described here is not necessarily limitative, and any of the effects described in the present disclosure may be provided.

BRIEF DESCRIPTION OF DRAWINGS

[0029] FIG. 1 is a diagram showing an example of a configuration of an HMD according to an embodiment.

[0030] FIG. 2 is a block diagram showing an example of a functional configuration of the HMD.

[0031] FIG. 3 is a block diagram showing an example of a functional configuration of a controller.

[0032] FIG. 4 shows graphs of specific examples when the display scale of a specific virtual object is reduced along the time axis.

[0033] FIG. 5 shows graphs of specific examples when the display scale of a specific virtual object is reduced along the time axis.

[0034] FIG. 6 shows graphs of specific examples when the display scale of a specific virtual object is increased along the time axis.

[0035] FIG. 7 shows graphs of specific examples when the display scale of a specific virtual object is increased along the time axis.

[0036] FIG. 8 is a flowchart showing an example of changing the display scale for a plurality of virtual objects.

[0037] FIG. 9 is a schematic view showing an example of a plurality of virtual objects and a notification displayed on a display.

[0038] FIG. 10 is a table showing an example of scale change information of a specific virtual object and the display scale (display magnification) in FIG. 7.

[0039] FIG. 11 is a schematic view showing an example of a plurality of virtual objects and a notification displayed on the display according to a second embodiment.

[0040] FIG. 12 is a perspective view showing an exemplary appearance of an HMD according to another embodiment.

MODE(S)* FOR CARRYING OUT THE INVENTION*

[0041] Embodiments according to the present technology will now be described below with reference to the drawings.

First Embodiment

[0042] [Head-Mounted Display (HMD)] FIG. 1 is a diagram showing an example of a configuration of an HMD 10 according to a first embodiment of the present technology. A of FIG. 1 is a schematic perspective view of an appearance of the HMD 10, and B of FIG. 1 is a schematic exploded perspective view of the HMD 10.

[0043] The HMD 10 includes a base 11, an attachment band 12, a headphone 13, a display unit 14, an inward-oriented camera 15 (15a, 15b), an outward-oriented camera 16, and a cover 17.

[0044] The base 11 is a member arranged in front of the right and left eyes of a user, and the base 11 is provided with a front-of-head support 18 that is brought into contact with the front of the head of the user.

[0045] The attachment band 12 is attached to the head of the user. As shown in FIG. 1, the attachment band 12 includes a side-of-head band 19 and a top-of-head band 20. The side-of-head band 19 is connected to the base 11, and is attached to surround the head of the user from the side to the back of the head. The top-of-head band 20 is connected to the side-of-head band 19, and is attached to surround the head of the user from the side to the top of the head.

[0046] The headphone 13 is connected to the base 11 and arranged to cover the right and left ears of the user. The headphone 13 includes right and left speakers. The position of the headphone 13 is manually or automatically controllable. The configuration for that is not limited, and any configuration may be adopted.

[0047] The display unit 14 is inserted into the base 11 and arranged in front of the eyes of the user. A display 22 is arranged within the display unit 14. Any display device using, for example, a liquid crystal or an electroluminescence (EL) may be used as the display 22. Further, a lens system (whose illustration is omitted) that guides an image displayed using the display 22 to the right and left eyes of the user is arranged in the display unit 14.

[0048] The inward-oriented camera 15 includes a left-eye camera 15a and a right-eye camera 15b that are respectively capable of capturing images of the left eye and the right eye of the user. The left-eye camera 15a and the right-eye camera 15b are respectively arranged in predetermined positions in the HMD 10, specifically, in predetermined positions in the base 11. For example, it is possible to detect, for example, line-of-sight information regarding a line of sight of the user on the basis of the images of the left eye and the right eye that are respectively captured by the left-eye camera 15a and the right-eye camera 15b.

[0049] A digital camera that includes, for example, an image sensor such as a complementary metal-oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor is used as the left-eye camera 15a and the right-eye camera 15b. Further, for example, an infrared camera that includes an infrared illumination such as an infrared LED may be used.

[0050] The outward-oriented camera 16 is disposed in a center portion of the cover 17 to be oriented outward (toward the side opposite to the user). The outward-oriented camera 16 is capable of capturing an image of a real space on a front side of the user. A digital camera that includes, for example, an image sensor such as a CMOS sensor or a CCD sensor is used as the outward-oriented camera 16.

[0051] The cover 17 is mounted on the base 11, and is configured to cover the display unit 14. The HMD 10 having such a configuration serves as an immersive head-mounted display configured to cover the field of view of the user. For example, a three-dimensional virtual space is displayed by the HMD 10. When the user wears the HMD 10, this results in providing, for example, a virtual reality (VR) experience to the user.

[0052] FIG. 2 is a block diagram showing an example of a functional configuration of the HMD 10. The HMD 10 further includes a connector 23, an operation button 24, a communication section 25, a sensor section 26, a storage 27, and a controller 40.

[0053] The connector 23 is a terminal used to establish a connection with another device. For example, a terminal such as a universal serial bus (USB) and a high-definition multimedia interface (HDMI) (registered trademark) is provided. Further, upon charging, a charging terminal of a charging dock (cradle) and the connector 23 are connected to perform charging.

[0054] The operation button 24 is provided at, for example, a predetermined position in the base 11. The operation button 24 makes it possible to perform an ON/OFF operation of a power supply, and an operation related to various functions of the HMD 10, such as a function related to display of an image and output of sound, and a function of a network communication. When the operation button 24 is operated by the user, an operation signal corresponding to the operation is generated and input to the controller 40. Thus, the user’s instruction is input to the controller 40.

[0055] The communication section 25 is a module used to perform network communication, near-field communication, or the like with another device. For example, a wireless LAN module such as Wi-Fi, or a communication module such as Bluetooth (registered trademark) is provided. When the communication section 25 is operated, this makes it possible to perform wireless communication with an external apparatus or the like.

[0056] The sensor section 26 includes a nine-axis sensor 29, a GPS 30, a biological sensor 31, and a microphone 32.

[0057] The nine-axis sensor 29 includes a three-axis acceleration sensor, a three-axis gyroscope, and a three-axis compass sensor. The nine-axis sensor 29 makes it possible to detect acceleration, angular velocity, and azimuth of the HMD 10 in three axes. The GPS 30 acquires information regarding the current position of the HMD 10. Results of detection performed by the nine-axis sensor 29 and the GPS 30 are used to detect, for example, the pose and the position of the user (the HMD 10), and the movement (motion) of the user. These sensors are provided at, for example, predetermined positions in the base 11.

[0058] The biological sensor 31 is capable of detecting biological information regarding the user. For example, a brain wave sensor, a myoelectric sensor, a pulse sensor, a perspiration sensor, a temperature sensor, a blood flow sensor, a body motion sensor, and the like are provided as the biological sensor 31.

[0059] The microphone 32 detects information regarding sound around the user. For example, a voice from speech of the user is detected as appropriate. This enables the user to, for example, enjoy VR experience while making a voice call and perform input of an operation of the HMD 10 using voice input.

[0060] The type of sensor provided as the sensor section 26 is not limited, and any sensor may be provided. For example, a temperature sensor, a humidity sensor, or the like that is capable of measuring a temperature, humidity, or the like of the environment in which the HMD 10 is used may be provided. The inward-oriented camera 15 and the outward-oriented camera 16 can also be considered a portion of the sensor section 26.

[0061] The storage 27 is a nonvolatile storage device, and, for example, a hard disk drive (HDD), a solid state drive (SSD), or the like is used. Moreover, any non-transitory computer readable storage medium may be used.

[0062] The storage 27 stores therein a control program 28, model information 29, and scale change information 30 used to control an operation of the overall HMD 10. The method for installing the control program 28, the model information 29, and the scale change information 30 on the HMD 10 is not limited.

[0063] The model information 29 is information for rendering a virtual object and includes the shape, color, reference size, and the like of the virtual object. The reference size is a size as a reference at which the virtual object is displayed on the display 22. In this embodiment, the virtual object is displayed in a display size corresponding to the display scale (display magnification) with reference to the reference size.

[0064] For example, when the display scale is a factor of 1, the virtual object is displayed such that the reference size and the display size are equal to each other. When the display scale is a factor of 2, the virtual object is displayed such that the display size is twice the reference size. When the display scale is a factor of 0.5, the virtual object is displayed such that the display size is 0.5 times the reference size. The reference size can also be the display size when the display scale is a factor of 1.

[0065] The user can change the display scale to change the display size of the virtual object displayed on the display 22. For example, an instruction to change the display scale can be input by the operation of the operation button 22, voice input, gesture input, input based on the movement of the line of sight, or the like. Note that the gesture input can be performed on the basis of an image of a hand or the like of the user, which is captured by the outward-oriented camera 16. The input based on the movement of the line of sight can be performed on the basis of the line-of-sight information.

[0066] Of course, even when the user does not input an instruction to change the display size, the display scale may be changed. For example, when the distance between the HMD 10 and the virtual object changes, or when a predetermined visual effect is generated in video content, the display scale may be changed separately from the user’s instruction. In any case, the present technology is applicable to change the display size of the virtual object.

[0067] In this embodiment, three-dimensional models, which are simultaneously displayed at a predetermined size ratio in a virtual space, are displayed as a plurality of virtual objects. For example, in a state where the virtual objects are displayed in the respective reference sizes, the virtual objects are simultaneously displayed at a predetermined size ratio. That is, the ratio of the reference sizes corresponds to a predetermined size.

[0068] Note that the method of changing the display size according to the change of the display scale is not limited to the method using the reference size, and any algorithm may be adopted. Even in this case, for example, the size ratio of the plurality of virtual objects in a predetermined display state can be defined as a predetermined size ratio.

[0069] The scale change information 30 is information set for a specific virtual object among a plurality of virtual objects displayed on the display 22. Conversely, among a plurality of virtual objects displayed on the display 22, a virtual object for which the scale change information 30 is set is a specific virtual object.

[0070] All the virtual objects included in the video content may be specific virtual objects, or only one virtual object may be a specific virtual object.

[0071] The scale change information 30 is information regarding the change mode of the display size of a specific virtual object. In this embodiment, information regarding the change mode of the display scale is included. Specifically, information regarding switching of the change mode of the display scale (the change mode of the display size) is included.

[0072] For example, as the scale change information 30, information regarding two change modes different from each other and information regarding switching between two change modes different from each other are stored.

[0073] Examples of the two change modes different from each other include a first change mode in which the display scale is changed at a first change rate, and a second change mode in which the display scale is changed at a second change rate different from that of the first change mode. That is, two change modes in which the change rates are different from each other are included.

[0074] Further, the two change modes different from each other include a mode in which the display scale is changed and a change mode in which the change of the display scale is restricted. That is, in the present disclosure, the change mode includes a mode in which the display scale is not changed. Note that the change mode in which the change of the display scale is restricted can also be a change mode in which the change rate is zero.

[0075] Examples of the information regarding the switching of the two change modes include a timing at which the change mode is switched and a threshold value related to a display size serving as a reference for the switching of the change mode. In this embodiment, a threshold value related to the display scale is stored as the threshold value related to the display size. For example, when the display scale of a specific virtual object reaches a predetermined threshold value, the change mode of the display scale is switched.

[0076] Further, the maximum display scale in which the display scale is not further increased or the minimum display scale in which the display scale is not further decreased may be set as a threshold value related to the display scale. The maximum display scale and the minimum display scale correspond to threshold values for switching between the mode in which the display scale is changed and the change mode in which the change of the display scale is restricted. In this embodiment, the maximum display scale and the minimum display scale correspond to the maximum display size and the minimum display size.

[0077] Of course, if the maximum display scale and the minimum display scale are set, the display size of the virtual object is changed within the range of the maximum display scale and the minimum display scale. Of course, only one of the maximum display scale and the minimum display scale may be set.

[0078] Further, the scale change information 30 includes restriction information regarding the restriction of change in the display size (display scale) of other virtual objects other than the specific virtual object for which the scale change information 30 is set. For example, information on whether to restrict the change of the display size (display scale) of other virtual objects in response to the switching of the change mode of the display size (display scale) is stored as the restriction information. Alternatively, when the display size (display scale) reaches the threshold value, information on whether to restrict the change of the display size (display scale) of other virtual objects is stored.

[0079] The display of a plurality of virtual objects using the scale change information 30 will be described later in detail. In this embodiment, the scale change information 30 corresponds to display size change information.

[0080] The controller 40 controls operations of the respective blocks of the HMD 10. The controller 40 has a hardware configuration necessary for a computer, such as a CPU, a GPU, and a memory (a RAM and a ROM). Various processes are performed by the CPU or the like loading, into the RAM, the control program 28 stored in the storage 27 and performing the control program 28.

[0081] For example, a programmable logic device (PLD) such as a field programmable gate array (FPGA), or other devices such as an application specific integrated circuit (ASIC) may be used as the controller 40.

[0082] FIG. 3 is a block diagram showing an example of a functional configuration of the controller 40. In this embodiment, a scale change processor 41, a notification control section 42, a display generation section 43, a scale change information generation section 44, and a sound control section 45 are implemented as functional blocks by the CPU or the like of the controller 40 performing a program (such as an application program) according to this embodiment.

[0083] The information processing method according to this embodiment is performed by these functional blocks. Note that, in order to implement each functional block, dedicated hardware such as an integrated circuit (IC) may be used as appropriate. In this embodiment, the information processing apparatus according to the present technology is achieved by the HMD 10 including the controller 40. Further, the HMD 10 also functions as a display apparatus according to the present technology.

[0084] The scale change processor 41 reads model information of a plurality of virtual objects displayed on the display 22 from the storage 27. The scale change processor 41 then changes the display scale (display size) of the plurality of virtual objects. At this time, for a specific virtual object, the scale change information set for the specific virtual object is read from the storage 27, and the display scale is changed on the basis of the read scale change information.

[0085] Here, the case where the scale change processing is performed on the basis of a user’s change instruction, which is input through the operation of the operation button 24, will be described as an example. Of course, a change instruction may be input by voice input or gesture input. Also, the display scale may be changed regardless of the user’s change instruction. In any case, the scale change processor 41 performs the scale change processing on the plurality of virtual objects.

[0086] In this embodiment, the display scale is changed in different change modes for a specific virtual object. Further, the change of the display scale for the plurality of virtual objects is stopped at the timing at which the change mode of the display scale of any specific virtual object is switched. Subsequently, when the user’s instruction to change the display scale is input again, the change of the display size for the plurality of virtual objects is performed again. Of course, the present technology is not limited to such a method of changing the display size.

[0087] The notification control section 42 performs a notification regarding the switching of the change mode of the display scale of the specific virtual object when the display scale of the plurality of virtual objects is changed. As the notification regarding the switching of the change mode, any notification by which the user can recognize that the change mode of the specific virtual object has been switched may be performed.

[0088] For example, highlighting of the specific virtual object, output of related information regarding the switching of the change mode, and the like are performed. As the highlighting of the specific virtual object, any display for highlighting the specific virtual object may be performed, such as blinking, vibrating, enlarging or/and reducing the specific virtual object in size, changing the color, changing animations or the like, attaching a specific marker, etc.

[0089] As the output of the related information regarding the switching of the change mode, a text indicating that the change mode has been switched (the change mode is to be switched soon), a text indicating the current change mode, and the like are displayed. In addition, display of the current display scale (display magnification) and display of an indicator or the like indicating a threshold value of the display scale of the specific virtual object are performed. In addition, any related information regarding the switching of the change mode may be output.

[0090] In this embodiment, at least one of the highlighting of the specific virtual object or the output of the related information regarding the switching of the change mode is performed, but other notification methods may be performed.

[0091] The notification regarding the switching of the change mode (hereinafter, referred to as change mode switching notification) is performed at the timing at which the change mode of the display scale of the specific virtual object is switched. Therefore, in this embodiment, the switching notification is performed at the timing at which the change mode is switched, and the change of the display scale is stopped. This makes it possible for the user to sufficiently grasp that the change mode of the display scale of the specific virtual object is to be switched.

[0092] It is needless to say that the timing of the switching notification is not limited, and the timing corresponding to the timing at which the change mode of the display scale of the specific virtual object is switched may be appropriately set. For example, the change mode switching notification may be performed at a predetermined timing before the change mode of the display scale is switched. In addition, any timing at which the user can grasp the switching of the change mode may be adopted.

[0093] In the example shown in FIG. 3, the notification control section 42 generates an image for highlighting a specific object, an image of related information regarding the switching of the change mode, and the like, and outputs them to the display generation section 43. The notification control section 42 may be referred to as a notification expression generation unit. Note that the output of the related information regarding the switching of the change mode or the like may be performed by the output of sound through the headphone 13.

[0094] The notification by the notification control section 42 is performed on the basis of the scale change information 30 set for the specific virtual object. In the example shown in FIG. 3, the scale change processor 41 reads the scale change information 30 from the storage 27 and outputs it to the notification control section 42. Of course, the notification control section 42 may read the scale change information 30 from the storage 27.

[0095] The display generation section 43 generates an image (rendering image) for displaying on the display 22 a plurality of virtual objects whose display scale has been changed by the scale change processor 41. Further, the scale change processor 41 generates an image (rendering image) for displaying on the display 22 a notification display (highlight display or display of related information) output by the notification control section 42. By the generated rendering images, the plurality of virtual objects and the change mode switching notification are displayed on the display 22.

[0096] The scale change information generation section 44 generates the scale change information 30 set for the specific virtual object. For example, the scale change information 30 can be dynamically generated on the basis of user information regarding a user, surrounding environmental information, apparatus information of the HMD 10 for displaying the specific virtual object, and the like.

[0097] The user information includes any information such as the standing position of the user, the visual acuity of the user, the physical condition of the user, and the like. The environmental information includes any information such as ambient brightness and temperature. The apparatus information includes any information such as the resolution of the HMD 10 and information of a lens. The user information, the environmental information, and the apparatus information can be obtained on the basis of, for example, a detection result by the sensor section 26 of the HMD 10. Of course, the user information, the environmental information, and the apparatus information may be input in advance and stored in the storage 27.

[0098] In this embodiment, the scale change information generation section 44 generates display scale change information on the basis of at least one of the user information, the environmental information, or the apparatus information, but the scale change information may be generated on the basis of other information.

[0099] Further, it may be possible that the scale change information generation section 44 is not provided and the scale change information stored in advance in the storage 27 is used. That is, even when the scale change information is not dynamically generated, the present technology is applicable.

[0100] The sound control section 45 controls output of sound for each of the plurality of virtual objects. In this embodiment, the sound control section 45 controls output of sound related to a specific virtual object on the basis of the scale change information 30 of the specific virtual object. The user can hear the sound related to the specific virtual object output through the headphone 13.

[0101] For example, it is assumed that when the user is watching a tennis match in VR by using the HMD 10, the user performs an operation of changing the display scale of the tennis ball that is a specific virtual object. At this time, the volume of the sound generated when the ball is hit with the racquet of a player or the sound of the ball bouncing on the court is controlled according to the display scale of the ball, which is changed on the basis of the scale change information 30.

[0102] In this embodiment, when the scale change processor 41, the notification control section 42, and the display generation section 43 change the display size of the plurality of virtual objects, a display control section is implemented, which performs a notification regarding the switching of the change mode of the display size of a specific virtual object among the plurality of virtual objects on the basis of the display size change information of the specific virtual object.

[0103] Further, the scale change processor 41 implements an input section to which a user instruction is input. Further, the scale change information generation section 44 implements a generation section that generates display size change information set for a specific virtual object.

[Change of Display Scale of Specific Virtual Object]

[0104] FIGS. 4 to 7 are graphs for describing examples of changing the display scale of a specific virtual object. FIGS. 4 and 5 are graphs showing specific examples when the display scale of a specific virtual object is reduced along the time axis. FIGS. 6 and 7 are graphs showing specific examples when the display scale of a specific virtual object is increased along the time axis.

[0105] FIGS. 4 to 7 show two different examples of a change mode M1 and a change mode M2 for the change of the display scale of a specific virtual object. The change mode M1 and the change mode M2 correspond to one embodiment of a “first change mode in which the display scale is changed at a first change rate” and a “second change mode in which the display scale is changed at a second change rate different from the first change rate”.

[0106] Further, FIGS. 4 to 7 each show a switching scale at a factor of X, which is a threshold value related to the display scale as a reference for switching of the change mode. A timing T at which the display scale of a specific virtual object reaches the switching scale at a factor of X is a switching timing at which the change mode M1 is switched to the change mode M2. At this switching timing, the switching notification is performed.

……
……
……