Sony Patent | Live image display support apparatus, game system, and live image display support method

For example, a score assignment rule that gives higher priority order to stronger characters, larger clusters, and scenes with a higher level of importance is set up in advance and stored in the control information generation section 60. It is noted that the control information generation section 60 may combine a plurality of above-described perspectives and rank them as the display target. For example, if there are a plurality of locations where clusters of the same scale are formed, higher priority order is given to a cluster having a character with a higher score. If there are a plurality of characters with the same score, higher priority order is given to a character in battle. By evaluating the importance in display from a plurality of perspectives in this way, it is possible to easily display a suitable scene with high accuracy.

The control information generation section 60 may also generate information regarding a suitable position and pose of the virtual camera as the control information. For example, in a case where a location in which a cluster is formed is a display target, the control information generation section 60 may acquire the position and pose of the virtual camera such that the entire cluster fits within the field of view. This makes it easier for spectators to grasp the overall picture of the cluster. However, in this case, if the range of the cluster is too wide, an image of each character may possibly become small, making it difficult to view the movement or making the live image less powerful.

Therefore, the control information generation section 60 may limit the field of view according to a predetermined rule. In this case as well, the control information generation section 60 may select targets to be included in the field of view, by prioritizing regions within the cluster from the perspectives as described above. Further, the control information generation section 60 may generate control information by using information other than game parameters. For example, the control information generation section 60 may use the three-dimensional structure of the virtual world to prioritize display targets and determine the position and pose of the virtual camera.

Here, the three-dimensional structure of the virtual world includes the inclination angle and height of the ground, the arrangement and height of buildings, and the like. For example, in a case where characters forming a cluster are distributed on a slope or a cliff of a mountain, the pose of the virtual camera is derived such that a screen faces the slope or the cliff. This makes it possible to grasp, at a glance, a top-bottom relation of the positions where the characters are present. Further, a region that is difficult to view due to the relation between the inclination of the ground and the pose of the virtual camera is excluded from the field of view even if the region is within the range of the cluster, so that the above-described limitation of the field of view can appropriately be realized.

The control information generation section 60 may perform either one of determination or prioritization of an optimum display target and derivation of a suitable position and pose of the virtual camera or may perform both of them. For example, even in a case where the display target is fixed due to the nature of the game, it is possible to represent the live image at a suitable angle with the function of the control information generation section 60. Alternatively, even in a case where a player image is used as the live image, it is possible to easily select an image including an optimum display target. Needless to say, the control information generation section 60 may determine an optimum display target and then determine a suitable position and pose of the virtual camera with respect to the optimum display target.

The live image acquisition section 62 acquires the live image on the basis of the control information. For example, the live image acquisition section 62 sets the position and pose of the virtual camera according to the control information and then draws the virtual world of the game to generate the live image. Alternatively, the live image acquisition section 62 selects a player image to be used as the live image, on the basis of a suitable display target and the priority order which are indicated by the control information. In this case, the live image acquisition section 62 requests a player image including the determined display target from the data acquisition section 58 and acquires the player image transmitted from the corresponding player device 13.

The live image acquisition section 62 may continue to generate the live image by itself or may continue to acquire the selected player image. In the latter case, the player image to be acquired may appropriately be switched on the basis of the control information. Alternatively, the live image acquisition section 62 may switch between an image generated by itself and a player image as the live image. It is noted that, as described above, the live image acquisition section 62 may accept, via the input apparatus 18, virtual camera control or a screen switching operation performed by the live image administrator and generate the live image or acquire the player image accordingly.

In any mode, the live image acquisition section 62 may superimpose and display, on the live image, various pieces of information that are not displayed on the player displays 16. For example, the live image acquisition section 62 may represent which player each character in the live image corresponds to by letters or graphics and indicate a score, a hit point, a list of possessed weapons and the like, provisional ranking, and the like of each character. This makes it easier for spectators to understand the scene and the situation of the game represented by the live image.

The data output section 64 sequentially outputs the frame data of the live image acquired by the live image acquisition section 62, to cause the spectator display 8, the terminals 24, and the administrator display 20 to display the frame data. In a mode in which the live image administrator performs a field-of-view control or switching operation of the live image, the data output section 64 further causes the administrator display 20 to display the control information. For example, the data output section 64 represents information such as the priority order of a display target and a suitable position and pose of the virtual camera by using letters or graphics. Alternatively, the data output section 64 may process the live image being displayed, to highlight a character to be placed in the center next.

Next, the operation of the game system that can be implemented by the above configuration is described. FIG. 5 illustrates a processing procedure for controlling the live image and the transition of data in the present embodiment. Here, it is assumed that the player devices 13 and the game server 12 cooperate to continue the game processing corresponding to the operations performed by the players. In this process, the game data storage section 54 of the game server 12 continues to store various kinds of game data including game parameters according to the present embodiment (S10).

The parameter transmission section 56 of the game server 12 extracts predetermined game parameters from the game data storage section 54, for example, by using an API provided by game software (S12). In the illustrated example, the score and position of each character (player) are extracted as the game parameters. In this example, moreover, the API also provides data representing the three-dimensional structure of the virtual world. These pieces of data are transmitted from the parameter transmission section 56 to the live image display support apparatus 10. It is noted that the live image display support apparatus 10 may acquire the data representing the three-dimensional structure of the virtual world in advance.

The control information generation section 60 of the live image display support apparatus 10 generates control information by using the game parameters and data of the three-dimensional structure transmitted. In this example, first, the control information generation section 60 generates intermediate information directly acquired from those pieces of data (S14) and then derives the position and pose of the virtual camera (S16). Specifically, the control information generation section 60 simply sorts the scores to prioritize characters to be displayed (S14a). Further, the control information generation section 60 performs clustering on the basis of pieces of position information of the characters to extract regions of display target candidates (S14b).

By comprehensively evaluating these pieces of information, it is possible to derive an optimal display target, such as, for example, a location where a cluster to which the strongest character belongs is formed. Once the approximate positions of such a display target and, further, the virtual camera are determined, the control information generation section 60 further calculates the normal of the terrain or the like by using the data of the three-dimensional structure of the location, to derive a suitable pose of the virtual camera (S14c). At this time, the control information generation section 60 may adjust the position of the virtual camera to obtain a suitable field of view on the basis of the three-dimensional structure.

According to the position and pose of the virtual camera derived by the processing above, the live image acquisition section 62 acquires the live image by, for example, drawing the game world in the corresponding field of view and outputs the live image to the spectator display 8 and the like (S18). By repeating the illustrated processing at a predetermined frequency or as necessary, it is possible to keep displaying a suitable live image in such a manner as to correspond to changes in the game situation. However, the illustrated procedure and used data are examples only and do not limit the present embodiment.

FIG. 6 is a view for describing an example of determining a suitable position of the virtual camera on the basis of clustering. (a) illustrates the distribution of characters in the virtual world. The control information generation section 60 performs clustering by using a general algorithm such as a k-means method, on the basis of the position coordinates of each character indicated by a rectangle in the figure. In the illustrated example, three clusters 70a, 70b, and 70c are detected. In a case where a plurality of clusters are formed in this way, the control information generation section 60 selects one of the clusters as a display target according to a predetermined rule.

For example, the control information generation section 60 selects a cluster to which a character with the highest score or number of kills belongs or a cluster with the highest total or average score or number of kills by characters belonging to the cluster. There are no particular limitations on the game parameters used for cluster selection, such as the scale of movement and the type of action, in addition to the scores and the number of kills. For example, clusters may be scored from a plurality of perspectives and a cluster with the highest score may be selected, as described above. At this time, various parameters that are not displayed on the player displays 16 (not known to the players) may be added.

For example, the priority order of display corresponding to the attributes, contracts, and the like of players may be set to characters in advance, and the priority order may be reflected in the score of each cluster. Alternatively, a cluster satisfying a predetermined condition may immediately be selected without comparison of scores. For example, a cluster to which a character who is predetermined to continue to be displayed belongs or to which a character holding a predetermined important object in the game belongs may be selected without comparison with other clusters.

Further, upper and lower limits may be set for the area of a cluster or the number of characters belonging to the cluster, and any cluster that deviates from these limits may be excluded from options or its priority order may be lowered. Accordingly, for example, it is possible to avoid, as much as possible, displaying a cluster in which individual characters are difficult to view due to an excessively large area or a cluster in which the number of characters is small and the scene is likely to lack excitement. After selecting one cluster 70b through the selection process as described above, the control information generation section 60 derives a suitable position of the virtual camera according to the position and area of the cluster 70b.

For example, the control information generation section 60 performs alignment such that an optical axis of the virtual camera matches the center of gravity of the cluster 70b. Further, the control information generation section 60 determines the height of the virtual camera relative to the ground such that the diameter of the cluster 70b occupies a predetermined proportion such as 90% of the size of a screen in a short direction. (b) of the figure schematically illustrates the live image acquired by the live image acquisition section 62 by setting the virtual camera in this way. This example illustrates how characters are dispersed in an outdoor parking lot or the like. Here, the pose of the virtual camera is such that an imaging plane (view screen) faces the ground, which is a horizontal plane.

It is noted that the position and pose of the virtual camera are not limited to being fixed as they are, but may be caused to change over time within a predetermined range centering on that state to give dynamism to a video. The operation may be automatically performed by the live image acquisition section 62 according to a preset rule or may be manually performed by the live image administrator. Further, as described above, the live image acquisition section 62 may superimpose and display additional information, such as the names of the players corresponding to the characters, the identification of the team, and a list of scores, on the live image. The live image illustrated in the figure allows spectators to look over the overall appearance of the characters gathering and fighting at an easy-to-view magnification.

FIG. 7 is a view for describing an example of determining the pose of the virtual camera in consideration of the three-dimensional structure of the virtual world. Upper parts of (a) and (b) represent the height of the ground in the virtual world in a longitudinal direction of the figure. This example illustrates characters (e.g., characters 82) indicated by rectangles forming a cluster on a slope of a mountain 80 in the virtual world. When the cluster is detected as described with reference to FIG. 6 and a virtual camera 84a is set vertically downward as illustrated in the upper part of (a), a live image 86a illustrated in a lower part thereof is generated.

In this case, the distance between characters decreases according to the inclination, making it difficult to grasp the actual positional relation. As the inclination of the mountain 80 becomes steeper and closer to a cliff, the characters in the live image overlap with each other, becoming more difficult to view. Therefore, the control information generation section 60 adjusts the pose of the virtual camera on the basis of the three-dimensional structure of the virtual world as a display target. Specifically, as illustrated in the upper part of (b), the control information generation section 60 acquires a normal vector n of the ground as the display target and derives the pose of a virtual camera 84b such that an optical axis o matches the normal vector n.

Here, the normal vector n only needs to be obtained for a point represented in the center of the live image, for example. In a case where a cluster is a display target as in FIG. 6, the center of gravity of the cluster corresponds to this. In this case, as in FIG. 6, the height of the virtual camera 84b is adjusted such that the entire cluster fits within the angle of view. With this configuration, as illustrated in the lower part of (b), it is possible to display a live image 86b, which represents the actual distance between the characters. In this case as well, the position and pose of the virtual camera may be caused to change over time within a predetermined range to make the relation between the characters and the slope easier to understand.

FIG. 8 is a view for describing an example of determining the position and pose of the virtual camera in consideration of the three-dimensional structure of the virtual world. An upper part represents the height of the ground in the virtual world in a longitudinal direction of the figure. This example also illustrates how characters indicated by rectangles form a cluster on slopes of a mountain 90 in the virtual world. However, in this case, the characters (e.g., characters 92a and 92b) are distributed not only on a slope on one side of the mountain 90 but also on a slope on the other side beyond a summit A. When the cluster is detected as described with reference to FIG. 6 and a virtual camera 94a is set vertically downward, a live image is generated as illustrated in (a) of a lower part.

As described with reference to FIG. 7, deriving the pose of the virtual camera based on the normal vector n at the center of gravity of a cluster also yields approximately the same result. As a result, as with the case of FIG. 7, the distance between the characters reduces, making it difficult to grasp the actual positional relation. Further, in this case, if the position of the summit A is unknown, it is difficult to grasp the vertical positional relation of the characters. Therefore, as indicated by arrows in the figure, the control information generation section 60 acquires normal vectors of the ground at predetermined intervals in a display range within or including the cluster, for example.

Then, on the basis of the relation between these angles, the control information generation section 60 limits the range of the display target in the cluster. For example, the control information generation section 60 divides the cluster into regions according to the ranges of angles of the normal vectors. Then, the control information generation section 60 excludes, from the display target, any region having a normal vector forming an angle equal to or greater than a predetermined angle, such as 90°, with respect to a normal vector (e.g., a normal vector n′) at the center of gravity of the largest region among the regions. The angle between normal vectors can be calculated by an inner product or the like. In the illustrated example, the region of the slope on the opposite side of the summit A is excluded on the basis of a normal vector n″.

Then, the position and pose of a virtual camera 94b are derived as described with reference to FIG. 7, for a new cluster formed by the remaining characters (e.g., the characters 92a). That is, an optical axis o of the virtual camera 94b is caused to match a normal vector (e.g., a normal vector n′) at the center of gravity of the new cluster, and the height of the virtual camera 94b is adjusted such that the entire cluster is included in the angle of view. In this way, as illustrated in (b), it is possible to display a live image representing the actual distance and top-bottom relation between the characters.

Further, since the regions that cannot be viewed from the virtual camera 94b are excluded from the cluster serving as the display target, only the remaining characters can be displayed at a high magnification. It is noted that, although the mountain in the virtual world is exemplified in the figure, a suitable position and pose of the virtual camera can similarly be derived for buildings, the bottom of the sea, and the like. Further, in this description, for the location where a cluster is detected, the control information generation section 60 obtains a normal vector on the spot and generates the control information in consideration of the inclination. By contrast, the control information generation section 60 may perform region division in advance within a range of the inclination angle of the ground by, for example, acquiring the distribution of normal vectors for all the regions of the virtual world.

For example, the control information generation section 60 may prepare a terrain map in which regions are tagged according to the type of three-dimensional structure such as a plain, a mountain, a valley, or a building, by using a three-dimensional model of the virtual world. In this case, like the mountain 90 illustrated in the figure, in the case of a terrain with adjacent slopes having an angle that would not be captured in a case where the virtual camera is set to face one slope, clustering may be performed under a condition that the boundary between the slopes is not straddled in the first place.

FIG. 9 is a diagram for describing a method for generating a terrain map by the control information generation section 60. First, the control information generation section 60 uses the distribution of normal vectors acquired at predetermined intervals, to divide regions of the virtual world on the basis of the angular range thereof. For example, a region in which the inner product of normal vectors at the predetermined intervals continues to be a positive predetermined value or more is determined as a plain or a gentle slope. Since the other regions are mountains or valleys, the control information generation section 60 determines which one they are as illustrated in (a) of the figure.

That is, focusing on two surfaces 100 and 102 whose inner product is equal to or less than the predetermined value and thus having a sharp angle change, the control information generation section 60 sets vectors h and h′ directed from a midpoint 104 of the center of gravity of these surfaces toward the center of gravity of each of the surfaces 100 and 102. Then, the control information generation section 60 respectively calculates the inner products of the vectors h and h′ and normal vectors N and N′ of the surfaces 100 and 102 at points at which the respective vectors h and h′ reach. In a case where the inner product is positive, the control information generation section 60 determines that the surfaces 100 and 102 form a mountain as illustrated on the left side of (a). In a case where the inner product is negative, the control information generation section 60 determines that the surfaces 100 and 102 form a valley as illustrated on the right side of (a).

By changing the orientations of the vectors h and h′, it is possible to acquire a change in the mountain and valley on the basis of the orientations. By performing such calculation, the control information generation section 60 can add tags such as a “plain,” a “mountain,” and a “valley” to locations in the virtual world, like the terrain map illustrated in (b) of the figure. However, the above-described calculation method is an example only, and it is to be understood by those skilled in the art that there are various possible methods for identifying the type of three-dimensional structure with use of the three-dimensional model of the virtual world.

In any case, the control information generation section 60 can efficiently generate the control information by acquiring the terrain map in advance. For example, as described above, the control information generation section 60 can perform clustering in such a manner as not to straddle the summit or ridge of a mountain. Further, it is also possible to switch a policy for determining the position and pose of the virtual camera, depending on the type of terrain. For example, as illustrated in FIGS. 7 and 8, in the case of a cluster formed on a mountain, the pose of the virtual camera may be such that the virtual camera faces a slope on one side while, in the case of a cluster formed on a valley, the pose of the virtual camera may be such that the virtual camera faces a horizontal direction in such a manner as to capture both slopes.

FIG. 10 exemplifies screens for an administrator displayed on the administrator display 20 by the live image display support apparatus 10 in a mode in which the live image administrator controls the live image. In this example, it is assumed that the display target is set for each character. In this case, it is conceivable to use, as the live image, a player image corresponding to a character as the display target. However, there is no intention to limit the present invention thereto.

The examples illustrated in (a) and (b) of the figure each illustrate an image for the administrator based on the live image being displayed. In other words, the display target at this time is a character 110, a back view of the character 110 is placed in the vicinity of the bottom of the center, and its surrounding virtual world is represented at a predetermined angle of view in the live image. Here, for example, in a case where the HP of the character 110 falls below a predetermined value, it is conceivable to change the display target to another character. The control information generation section 60 may continue to update the priority order of display given to the characters on the basis of the above-described game parameters, without limiting to the HP, and recommend changing the display target, when the highest-ranking character is replaced.

It is noted that the control information generation section 60 may set a lower limit on the time interval for changing the display target, such that the display target is not changed too frequently. When a condition for changing the display target to a character 112 is satisfied, the control information generation section 60 highlights the character 112 as illustrated in (a), to recommend the live image administrator in this regard. In this example, an arrow 114, which points to the character 112, is superimposed and displayed.

The live image administrator who has recognized by the arrow 114 that it is desirable to change the display target to the character 112 performs input to confirm the change of the display target via the input apparatus 18, for example. In response, the live image acquisition section 62 starts acquiring a live image in which the character 112 is placed in the vicinity of the bottom of the center. This image may be a player image of a player operating the character 112 or may be an image separately generated by the live image acquisition section 62 with the virtual camera brought closer to the character 112.

Through these processes, the character mainly displayed in the live image is switched from the character 110 to the character 112. It is noted that means for indicating the next display target candidate on the screen for the administrator is not limited to the arrow 114. For example, the outline of the character may be represented in a different color, or the entire silhouette may be masked with a predetermined color.

By contrast, the control information generation section 60 may display information that serves as a reference allowing the live image administrator to make a final determination, without specifying the next display target. For example, as illustrated in (b), among game parameters of candidate characters 112 and 116, game parameters that can be a basis to become the display target are displayed. In this example, gauges 118a and 118b, which represent the HPs, and icons (e.g., icons 120a and 120b), which represent possessed weapons, are represented in the vicinity of the character 112 and 116, respectively.

Among these, a basis for selecting the character 112 is the HP close to 100%. Therefore, the gauge 118a is highlighted with a bold line around the gauge 118a. Meanwhile, a basis for selecting the character 116 is the possessed weapon. Therefore, the icon 120b is highlighted with a bold line around the icon 120b. The live image administrator determines by himself/herself which basis is effective and selects one of the characters with an unillustrated cursor or the like to confirm and input the next display target. Subsequent processing by the live image acquisition section 62 is similar to that in the case of (a).

It is noted that, in this case as well, there is no particular limitation on how game parameters such as gauges and weapons are highlighted, as long as the administrator can easily recognize them. Further, in the case of using a quantitatively comparable game parameter such as the degree of rarity of a weapon, an HP, or a past battle record of a corresponding player, a list of results sorted by category can be displayed or the ranking may be displayed in the vicinity of each character, such that the administrator can recognize the priority order. Further, in a case where a character or a cluster that is not included in the live image and is present in another location is a candidate for the next display target, an image or map with a bird's eye view of the virtual world may separately be displayed to allow selection thereof.

It is noted that the information to be presented to the live image administrator is not limited to the one illustrated in the figure and may be any control information. For example, the control information generation section 60 may display information regarding suitable positions and poses of virtual cameras and the priority order thereof and allow the administrator to select one of them. At this time, the control information generation section 60 may further accept a fine correction of the position and pose of a virtual camera from the live image administrator. Alternatively, the control information generation section 60 may give the live image administrator a notification that a battle has started in a location that is not being displayed, and accept switching of the display target. After that, the control information generation section 60 may further accept, from the live image administrator, detailed specifications such as the state of the virtual camera in this location and the selection of a character to be mainly displayed.

According to the present embodiment described above, in an electronic game such as e-sports, predetermined game parameters are extracted from data acquired in the course of game processing and are used to generate control information relating to a suitable field of view of a live image. This facilitates the work of generating a live image or selecting one from player images according to the progress of the game. As a result, a suitable live image can be displayed regardless of the skill levels of staff or the number of staff, and an exciting event can be realized at a low cost.

Not only individual characters but also clusters of characters are also display target candidates of the live image. This makes it possible to convey an entire large-scale scene such as a team battle in an easy-to-understand manner. Meanwhile, it is possible to efficiently represent an important part of the game in an easy-to-view manner by narrowing down the display target according to a predetermined rule or by adjusting the position and pose of the virtual camera in consideration of the three-dimensional structure of the virtual world. By deriving a suitable position and pose of the virtual camera as the control information for the live image, the live image can be controlled not only manually but also completely automatically. The embodiment can flexibly be set depending on the scale of the event, funds, the contents of the game, the processing power of apparatuses, or the like.

The present invention has been described above on the basis of the embodiment. The above-described embodiment is an example only, and it is to be understood by those skilled in the art that various modifications can be made to combinations of the individual constituent components and individual processes of the embodiment and that such modifications also fall within the scope of the present invention.

INDUSTRIAL APPLICABILITY

As described above, the present invention is applicable to various information processing apparatuses, such as a live image display apparatus, a game server, and a personal computer, a game system including them, and the like.

REFERENCE SIGNS LIST

10: Live image display support apparatus

12: Game server

13: Player device

14: Input apparatus

16: Player display

18: Input apparatus

20: Administrator display

22: Network

24: Terminal

30: CPU

32: GPU

34: Main memory

40: Communication section

42: Storage section

44: Output section

46: Input section

48: Recording medium drive section

50: Game data transmission/reception section

52: Game processing section

54: Game data storage section

56: Parameter transmission section

58: Data acquisition section

60: Control information generation section

62: Live image acquisition section

64: Data output section