Samsung Patent | Display Apparatus And Server, And Control Methods Thereof
Patent: Display Apparatus And Server, And Control Methods Thereof
Publication Number: 20190058862
Publication Date: 20190221
Applicants: Samsung
Abstract
A display apparatus is provided. The display apparatus includes: an image processor configured to process an image signal; a display; and a processor configured to control the image processor to produce, from a 360-degree image, a plurality of split images to be arranged along a circumference within a predetermined region that includes a great circle of a sphere shape being mapped, based on a viewpoint corresponding to a user input, the plurality of split images corresponding to a plurality of screens, and to produce a projection image to be displayed on the display based on the viewpoint from the split image of the screen that corresponds to the display apparatus from among the plurality of split images.
CROSS-REFERENCE TO RELATED APPLICATION
[0001] This application is based on and claims priority under 35 U.S.C. .sctn. 119 to Korean Patent Application No. 10-2017-0103678, filed on Aug. 16, 2017, in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety.
BACKGROUND
1.* Field*
[0002] The disclosure relates to a display apparatus and a server, and control methods thereof, and more particularly to a display apparatus and a server, which control a 360-degree image on multi-screens, and control methods thereof.
2.* Description of the Related Art*
[0003] With recent increasing interest in virtual reality (VR) and release of various personal photographing devices, production of 360-degree content for broadcast, movies or the like has been rapidly on the rise. Such 360-degree content is displayable through not only a head mounted display (HMD) but also various devices such as a personal computer (PC), a television (TV), a smart phone, etc.
[0004] The 360-degree content enables a viewer to experience omnidirectional views. In particular, when a viewer watches the 360-degree content through multi-screens, the viewer can be immersed in a watching experience through a wide view angle.
[0005] In the related art, to display a panoramic image on the multi-screens, a corresponding image is received and displayed in accordance with information of a display from a source device. However, a method of controlling the panoramic image in response to viewpoint movement and zoom instruction has not been proposed.
[0006] Further, when the 360-degree image is reproduced through the single screen of the existing HMD, TV, smartphone, PC, etc., a projection image is displayed in accordance with longitude and latitude of a viewpoint as the viewpoint moves. However, when this method is used in moving the viewpoint on the multi-screens, there is an overlapping region of an image displayed on each individual screen.
SUMMARY
[0007] Provided are a display apparatus and a server, which control a 360-degree image on multi-screens, and control methods thereof.
[0008] Also provided are a display apparatus and a server, which maintain seamless connection between regions displayed on screens even though a viewpoint and a view angle are changed based on user control in case of reproducing a 360-degree image on multi-screens, and control methods thereof.
[0009] In accordance with an aspect of the disclosure, there is provided a display apparatus comprising: an image processor configured to process an image signal; a display; and a processor configured to control the image processor to produce, from a 360-degree image, a plurality of split images to be arranged along a circumference within a predetermined region that includes a great circle of a sphere shape being mapped and corresponding to a plurality of screens, based on a viewpoint corresponding to a user input, and to produce a projection image to be displayed on the display based on the viewpoint from the split image of the screen corresponding to the display apparatus from among the plurality of split images.
[0010] Thus, according to an embodiment, it is possible to control the 360-degree image on the multi-screens. Further, when the 360-degree image is reproduced on the multi-screens, it is possible to seamlessly connect the regions displayed on the screens even though the viewpoint and the view angle are changed by a user’s control.
[0011] The processor may obtain a view angle of the split image based on a number of a plurality of screens and a position at which the screen of the display apparatus is arranged. Thus, it is possible to determine a position to which the split image provided to each screen is mapped along the great circle of the sphere.
[0012] The display apparatus further comprises an input receiver configured to receive a user input, wherein: the processor may transmit control information based on the received user input to each of the plurality of display apparatuses corresponding to the plurality of screens, and the control information may comprise at least one of information about a change in the view angle, and information about a pitching rotation angle or a yawing rotation angle. Thus, when a user controls the 360-degree image through the remote controller or the like, the control information of the remote controller received in the main screen can be transmitted to another screen, and thus control operations are organically implemented in the screens.
[0013] The processor may set a control reference point with regard to the screen of the display apparatus based on a user input, and may obtain a center point of each respective projection image displayed on each corresponding screen based on a set position of the control reference point. Thus, a user can set a reference position with regard to the main screen in order to control the 360-degree image on the multi-screens.
[0014] The processor may change the position at which the respective projection image is displayed based on the information about a change of the viewpoint of the control reference point and the angular distance of the center point of each screen from the control reference point when the respective projection image is subjected to pitching or yawing rotation based on a user input. Thus, the rotation angle and rotation direction of the great circle are taken into account at pitching or yawing rotation control, and therefore images displayed on the screens are seamlessly connected without an overlapping or lost region.
[0015] The processor may adjust a view angle of the respective projection image displayed on each screen so that the plurality of screens can have a constant total range of the view angle, when the respective projection image is subjected to zoom control based on a user input. Thus, the total range of the view angle is maintained at zoom-in or zoom-out control, and therefore an image can be displayed without losing a major object in the screen.
[0016] In accordance with an aspect of the disclosure, there is provided a display apparatus comprising: an image processor configured to process an image signal; a display; and a processor configured to control the image processor to select a plurality of display apparatuses to be connected on a network based on a user input, to produce, from a 360-degree image, a plurality of split images corresponding to viewpoints of screens of the plurality of selected display apparatuses, and to map each of the produced split images to a sphere shape.
[0017] Thus, a user can directly select the multi-screens for reproducing the 360-degree image from among peripheral apparatuses.
[0018] The processor may be configured to display a user interface (UI) for setting a respective position of the sphere shape to which each respective split image corresponding to each screen of the plurality of selected display apparatuses is mapped. Thus, when a user directly selects the multi-screens for reproducing the 360-degree image, it is also possible to designate an arranged position of each screen.
[0019] The processor may be further configured to display a UI for showing images of apparatuses arranged in a surrounding space of a user, and to select the plurality of display apparatuses based on a user input to the UI. Thus, a user can more easily select apparatuses through the UI displayed on the screen to select the multi-screens for reproducing the 360-degree image.
[0020] In accordance with an aspect of the disclosure, there is provided a computer program product comprising: a memory configured to store an instruction; and a processor, wherein, when executed by the processor, the instruction causes the computer to implement a method that includes: producing, from a 360-degree image, a plurality of split images to be arranged along a circumference within a predetermined region that includes a great circle of a sphere shape being mapped and corresponding to a plurality of screens, based on a viewpoint corresponding to a user input, and producing a projection image to be displayed based on the viewpoint from the split image of the screen corresponding to the display apparatus from among the plurality of split images. The computer program product may be implemented as a non-transitory computer readable storage medium storing a computer program that includes the instruction.
[0021] In accordance with an aspect of the disclosure, there is provided a server comprising: a communication interface configured to communicate with a plurality of display apparatuses; and a processor configured to control the communication interface to produce, from a 360-degree image, a plurality of split images to be arranged along a circumference within a predetermined region comprising a great circle of a sphere shape being mapped and corresponding to respective screens of a plurality of display apparatuses, and to transmit the plurality of produced split images to the corresponding display apparatuses.
[0022] Thus, according to an embodiment, it is possible to control the 360-degree image on the multi-screens. Further, when the 360-degree image is reproduced on the multi-screens, it is possible to seamlessly connect the regions displayed on the screens even though the viewpoint and the view angle are changed by a user’s control.
[0023] The processor may be further configured to receive information about a number of screens of the plurality of display apparatuses and a position at which each of the plurality of screens is arranged, and to produce the plurality of split images based on the received information. Thus, it is possible to determine a position to which the split image provided to each screen is mapped along the great circle of the sphere.
[0024] In accordance with an aspect of the disclosure, there is provided a computer program product comprising: a memory configured to store an instruction; and a processor, wherein, when executed by the processor, the instruction causes the computer to implement a method that includes: producing, from a 360-degree image, a plurality of split images to be arranged along a circumference within a predetermined region comprising a great circle of a sphere shape being mapped and corresponding to a screens of the plurality of display apparatuses, and transmitting each of the plurality of produced split images to the corresponding display apparatuses. The computer program product may be implemented as a non-transitory computer readable storage medium storing a computer program that includes the instruction.
[0025] According to an exemplary embodiment, there is provided a method of controlling a display apparatus, comprising: producing, from a 360-degree image, a plurality of split images to be arranged along a circumference within a predetermined region comprising a great circle of a sphere shape being mapped and corresponding to a plurality of screens, based on a viewpoint corresponding to a user input; and producing a projection image to be displayed on the display based on the viewpoint from the split image of the screen corresponding to the display apparatus from among the plurality of split images.
[0026] Thus, according to an embodiment, it is possible to control the 360-degree image on the multi-screens. Further, when the 360-degree image is reproduced on the multi-screens, it is possible to seamlessly connect the regions displayed on the screens even though the viewpoint and the view angle are changed by a user’s control.
[0027] The method further comprises obtaining a view angle of the split image based on a number of the plurality of screens and a position at which the screen of the display apparatus among the plurality of screens is arranged. Thus, it is possible to determine a position to which the split image provided to each screen is mapped along the great circle of the sphere.
[0028] In accordance with an aspect of the disclosure, there is provided a method of controlling a display apparatus, comprising: selecting a plurality of display apparatuses to be connected on a network based on a user input; producing, from a 30-degree image, a plurality of split images corresponding to viewpoints of screens of the plurality of selected display apparatuses; and mapping each of the produced split images to a sphere shape.
[0029] Thus, a user can directly select the multi-screens for reproducing the 360-degree image from among peripheral apparatuses.
[0030] The method further comprises displaying a user interface (UI) for setting a position of the sphere shape to which the split image corresponding to each screen of the plurality of selected display apparatuses is mapped, based on a user input. Thus, when a user directly selects the multi-screens for reproducing the 360-degree image, it is also possible to designate an arranged position of each screen.
[0031] The method may further comprises: displaying a UI for showing images of apparatuses arranged in a surrounding space of a user; and selecting the plurality of display apparatuses based on a user input to the UI. Thus, a user can more easily select apparatuses through the UI displayed on the screen in order to select the multi-screens for reproducing the 360-degree image.
[0032] In accordance with an aspect of the disclosure, there is provided a method of controlling a server, comprising: producing, from a 360-degree image, a plurality of split images to be arranged along a circumference within a predetermined region comprising a great circle of a sphere shape being mapped and corresponding to a screens of the plurality of display apparatuses; and transmitting each of the plurality of produced split images to the corresponding display apparatuses.
[0033] Thus, according to an embodiment, it is possible to control the 360-degree image on the multi-screens. Further, when the 360-degree image is reproduced on the multi-screens, it is possible to seamlessly connect the regions displayed on the screens even though the viewpoint and the view angle are changed by a user’s control.
[0034] The method may further comprises: receiving information about a number of screens of the plurality of display apparatuses and a position at which each of the plurality of screens is arranged; and producing the plurality of split images corresponding to the plurality of screens based on the received information. Thus, it is possible to determine a position to which the split image provided to each the screen is mapped along the great circle of the sphere.
BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:
[0036] FIG. 1 illustrates a configuration between a server and a plurality of display apparatuses, according to an embodiment;
[0037] FIG. 2 illustrates an example of mapping split images generated from a 360-degree image along a great circle of a sphere, according to an embodiment;
[0038] FIG. 3 is a block diagram of a display apparatus, according to an embodiment;
[0039] FIG. 4 is a block diagram of a server, according to an embodiment;
[0040] FIG. 5 is a flowchart of operations between a server and a display apparatus, according to an embodiment;
[0041] FIG. 6 illustrates a configuration between a server and a plurality of display apparatuses, according to an embodiment;
[0042] FIG. 7 is a block diagram of a display apparatus, according to an embodiment;
[0043] FIG. 8 illustrates an example of displaying a projection image in response to user control;
[0044] FIG. 9 illustrates an example of arranging projection images for screens on a great circle of a sphere, according to an embodiment;
[0045] FIG. 10 illustrates a formula to calculate coordinates of a projection image for each screen arranged on a great circle of a sphere at rotation control, according to an embodiment;
[0046] FIG. 11 illustrates an example of a shape of a great circle displayed on a sphere and a 360-degree image at rotation control, according to an embodiment;
[0047] FIG. 12 illustrates an example of a projection region for each individual screen on a 360-degree image, according to an embodiment;
[0048] FIG. 13 illustrates an example of a projection region for each individual screen on a 360-degree image, according to an embodiment;
[0049] FIG. 14 illustrates an example of a projection region for each individual screen on a 360-degree image, according to an embodiment;
[0050] FIG. 15 illustrates an example of a projection image displayed on each individual screen at rotation control, according to an embodiment;
[0051] FIG. 16 illustrates an example of change in a projection image displayed on each individual screen at rotation control, according to an embodiment;
[0052] FIG. 17 is a graph of showing view-angle setting values in accordance with viewpoints of each individual screen at zoom control, according to an embodiment;
[0053] FIG. 18 is a graph of a function of view-angle setting values, according to an embodiment;
[0054] FIG. 19 illustrates an example of change in a view angle with regard to each screen at zoom control, according to an embodiment;
[0055] FIG. 20 illustrates an example of setting a control reference point with regard to multi-screens, according to an embodiment;
[0056] FIG. 21 illustrates an example of setting a control reference point at a center point of a main screen, according to an embodiment;
[0057] FIG. 22 illustrates an example of setting a control reference point in an outside region of a main screen, according to an embodiment;
[0058] FIG. 23 is a flowchart of showing operations of a display apparatus at rotation control, according to an embodiment;
[0059] FIG. 24 is a flowchart of showing operations of a display apparatus at zoom control, according to an embodiment;
[0060] FIG. 25 illustrates an example of selecting multi-screens within a space, according to an embodiment;
[0061] FIG. 26 illustrates an example of a user interface (UI) for selecting multi-screens to display a 360-degree image, according to an embodiment;
[0062] FIG. 27 illustrates an example of a UI for mapping selected multi-screens to a sphere, according to an embodiment;
[0063] FIG. 28 illustrates an example of configuring a virtual sphere by connecting center points of screens, according to an embodiment;* and*
[0064] FIG. 29 illustrates an example of determining a projection center of screens by configuring a virtual sphere, according to an embodiment.
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS
[0065] Hereinafter, exemplary embodiments will be described in detail with reference to accompanying drawings so as to be easily realized by a person having ordinary skill in the art. The present disclosure may be materialized in various different forms and not limited to the embodiments set forth herein.
[0066] Herein, expressions such as “at least one of,” when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list. For example, the expression, “at least one of a, b, and c,” should be understood as including only a, only b, only c, both a and b, both a and c, both b and c, or all of a, b, and c.
[0067] FIG. 1 illustrates a configuration between a server and a plurality of display apparatuses, according to an embodiment. As shown in FIG. 1, a server 10 is connected to a splitter 37, and a plurality of display apparatuses 20, 30, 31, 32, 33 and 34 are connected in parallel to the splitter 37. The server 10 communicates with the splitter 37 via a local area network (LAN) or another wired communication method, or via a wireless communication method, and content and information of a 360-degree image (refer to item 950 in FIG. 2) are transmitted and received between the server 10 and the splitter 37. In this aspect, the 360-degree image 950 is produced by stitching two or more images respectively photographed by two or more lenses, and is thus viewable in all directions.
[0068] The splitter 37 communicably connects with each of the plurality of display apparatuses 20, 30, 31, 32, 33 and 34 via a high-definition multimedia interface (HDMI) or a digital visual interface (DVI).
[0069] The server 10 produces a plurality of split images (for example, split images 91, 92, 93, … in FIG. 2) corresponding to viewpoints of the plurality of display apparatuses 20, 30, 31, 32, 33 and 34 from each frame of the 360-degree image 950 received from an external source or stored in a memory.
[0070] The splitter 37 receives the produced split images 91, 92, 93, … of the 360-degree image 950 from the server 10, and distributively transmits each of the plurality of received split images 91, 92, 93, … to the corresponding display apparatuses 20, 30, 31, 32, 33 and 34.
[0071] Alternatively, the splitter 37 may communicably connect with a universal serial bus (USB) device 381 in which the 360-degree image 950 is stored. In this case, the splitter 37 receives the 360-degree image 950 stored in the USB device 381, performs a process of producing the plurality of split images 91, 92, 93, … which respectively correspond to the plurality of display apparatuses 20, 30, 31, 32, 33 and 34 from each frame of the 360-degree image 950, and distributively transmits each of the produced split images 91, 92, 93, … to the corresponding display apparatuses 20, 30, 31, 32, 33 and 34.
[0072] Further, the splitter 37 may communicably connect with any one or more of a Blu-ray disc player 382, a game playing device 383, a laptop computer 384, etc., and receive and distributively transmit the 360-degree image 950 stored therein or a plurality of previously processed split images 91, 92, 93, from them to the display apparatuses 20, 30, 31, 32, 33 and 34.
[0073] FIG. 2 illustrates an example of mapping split images generated from a 360-degree image along a great circle of a sphere, according to an embodiment. As shown in FIG. 2, the server 10 produces the split images 91, 92, 93, … corresponding to the screens of the plurality of display apparatuses 20, 30, 31, … from the 360-degree image 950.
[0074] The server 10 causes the split images 91, 92, 93 … to be arranged along a great circle of a sphere 120 mapping to the plurality of display apparatuses 20, 30, 31, … when producing the split images 91, 92, 93, … . In particular, the great circle 100 refers to a circumference of a circle created when the sphere 120 is cut passing through the center point thereof.
[0075] According to an embodiment, the server 10 may cause the split images 91, 92, 93, … to be arranged along a circumference within a predetermined region that includes the great circle 100 of the sphere 120 mapping to the plurality of display apparatuses 20, 30, 31, … when producing the split images 91, 92, 93, … . In particular, when the split images 91, 92, 93, … are mapped to the sphere 120, one way of securing connectivity of images displayed on the screens is to arrange the images along the great circle 100. However, another way may be to arrange the images along a circumference positioned at a relatively close distance from the great circle 100.
[0076] The display apparatuses 20, 30, 31, … use a central processing unit (CPU) or a graphic processing unit (GPU) to cause the split images 91, 92, 93, … received from the server 10 to be texture-mapped along the great circle of the sphere 120. In this case, the display apparatuses 20, 30, 31, … determine a respective view angle of each of the split images 91, 92, 93, … based on a total number of screens and a position at which the screen of the corresponding display apparatus is arranged within an entire screen.
[0077] For example, when a total number of screens is six (6) and a certain screen within the entire screen is positioned at the third from the left, the split image corresponding to the certain screen may have a view angle of 60 degrees and the view angle may range from 120 degrees to 180 degrees within the whole range of 360 degrees on the great circle of the sphere.
[0078] As described above, an operation of determining the respective view angle of each split image to be mapped along the great circle of the sphere may be performed in the server 10 in accordance with connection structures of the plurality of display apparatuses 20, 30, 31, … or may be performed in the corresponding display apparatus.
[0079] In this aspect, when the plurality of display apparatuses 20, 30, 31, … are connected in parallel to the server 10, the server 10 may determine the respective view angle of the split image corresponding to each screen and provide information about the determined view angle to the corresponding display apparatus.
[0080] Alternatively, when the plurality of display apparatuses 20, 30, 31, … are connected in series to the server 10, each of the display apparatuses 20, 30, 31, … may determine the respective view angle of the split image corresponding to its own screen.
[0081] According to an embodiment, the display apparatuses 20, 30, 31, … may directly set the view angles of the split images corresponding to their own screens in response to a user input. In this case, unlike the case of setting the view angle of the split image uniformly by taking the total number of screens into account, a user may set a certain screen to have a larger or smaller view angle of a split image than those of the other screens.
[0082] FIG. 3 is a block diagram of a display apparatus, according to an embodiment. The display apparatus 20 includes a communication interface 21, a processor 22, an image processor 23, a display 24, a storage 25 and an input receiver 26, and communicates with a server 10 and a plurality of display apparatuses 30, 31, … via the communication interface 21. In this case, the display apparatus 20 and the plurality of display apparatuses 30, 31 … are connected in parallel to the server 10. The display apparatus 20 may for example include a television (TV), a smartphone, a projector, a head-mounted display (HMD), etc. There are no limits to the elements of the display apparatus 20, and the display apparatus 20 may include another element.
[0083] The display apparatus 20 is connected to and synchronized with the plurality of display apparatuses 30, 31, … via a network, and may operate as a main apparatus to be controlled by a user to reproduce and control the 360-degree image 950.
[0084] The communication interface 21 may be a transceiver the communicates with the server 10 using a wireless communication method or a wired communication method. The communication interface 21 may communicate with the server 10 by a wired communication method such as Ethernet, etc., or communicate with the server 10 via a wireless router by a wireless communication method such as wireless fidelity (Wi-Fi), Bluetooth, etc. There are no limits to the communication method of the communication interface 21, and the communication interface 21 may use another communication method.
[0085] According to an embodiment, the communication interface 21 may be not directly connected to the server 10, but may instead be connected to the splitter 37 as shown in FIG. 1. In this case, the communication interface 21 may receive data, which is routed by the splitter 37, from the server 10 via the splitter 37.
[0086] The input receiver 26 is configured to receive a user’s input for controlling at least one function of the display apparatus 20. For example, the input receiver 26 may receive a user’s input for selecting a part of a user interface (UI) displayed on the display 24. The input receiver 26 may be provided as an input panel provided outside the display apparatus 20 or as a remote controller capable of performing infrared communication with the display apparatus 20. Further, the input receiver 26 may be implemented as a keyboard, a mouse, etc. connected to the display apparatus 20, and may be implemented as a touch screen provided on the display apparatus 20.
[0087] The storage 25 is configured to store a split image 91 received from the server 10. The split image 91 is obtained by partitioning a region that corresponds to a view angle of a screen 81 from the whole region of the 360-degree image 950. The storage 25 causes the data of the stored split image 91 to undergo reading, writing, edition, deleting, update, etc. The storage 25 may be implemented as a nonvolatile memory such as a flash memory, a hard disk drive, etc. in order to retain data regardless of whether the display apparatus 20 is powered on or off.
[0088] The image processor 23 performs imaging processing functions with regard to the split image 91 stored in the storage 25. Further, the image processor 23 performs imaging processing functions with regard to a projection image (see item 812 in FIG. 21) produced in correspondence with the viewpoint of the screen 81 from the split image 91 mapped onto the great circle 100 of the sphere 120. As examples of the imaging processing functions performed in the image processor 23, these functions may include any one or more of demultiplexing, decoding, de-interlacing, scaling, noise reduction, detail enhancement, etc. without limitations. The image processor 23 may be implemented as a system on chip (SOC) where many functions are integrated, or as individual elements for independently performing each function.
[0089] The display 24 includes a screen 81, which displays an image based on an image signal processed by the image processor 23. There are no limits to the types of the display 24, and the display 24 may be implemented as any of various types such as a plasma display panel (PDP), a liquid crystal display (LCD), an organic light emitting diode (OLED), a flexible display, etc.
[0090] The processor 22 receives the split image 91 produced from each frame of the 360-degree image including the plurality of frames based on a viewpoint in response to a user input received from the input receiver 26.
[0091] In particular, the processor 22 receives the split image 91, which is produced to be arranged along the great circle 100 of the sphere 120 mapping to the display apparatus 20, from the server 10. In this case, the view angle of the split image 91 may be determined by considering a total number of screens and a relative position where the screen 81 is positioned within the entire screen. The processor 22 may include at least one central processing unit (CPU) and/or at least one microprocessor.
[0092] The processor 22 controls the image processor 23 to map the split image 91 received from the server 10 onto the great circle 100 of the sphere 120, and causes a projection image 812 to be produced based on a viewpoint from the mapped split image 91 and displayed on the display 24.
[0093] According to an embodiment, the foregoing operations of the processor 22 may be implemented by a computer program stored in a computer program product provided separately from the display apparatus 20.
[0094] In this case, the computer program product includes a memory in which an instruction corresponding to the computer program is stored, and a processor. When executed by the processor, the instruction causes a computer to receive the split image 91 of the screen 81 corresponding to the display apparatus 20 among the plurality of split images 91, 92, 93, … which is produced, from the 360-degree image 950, to be arranged along the great circle 100 of the sphere 120 being mapped and corresponding to the plurality of screens, based on a viewpoint according to a user input, and to produce and display the projection image 812 based on the viewpoint from the received split image 91.
[0095] Thus, the display apparatus 20 may download and execute a computer program stored in a separate computer program product, and perform the operations of the processor 22.
[0096] FIG. 4 is a block diagram of a server, according to an embodiment. As shown in FIG. 4, the server 10 includes a communication interface 11, a processor 12, and a storage 15, and communicates with the plurality of display apparatuses 20, 30, 31, … via the communication interface 11. In this case, the plurality of display apparatuses 20, 30, 31, … are connected in parallel to the server 10. However, there are no limits to the configuration of the server 10. For example, when a display apparatus is provided as the server 10, the server 10 may further include any of an image receiver, an image processor, and/or a display.
[0097] The communication interface 11 communicates with the plurality of display apparatuses 20, 30, 31, … by a wireless or wired communication method. The communication interface 11 may communicate with the plurality of display apparatuses 20, 30, 31, … by the wired communication method such as Ethernet, etc., or communicate with the plurality of display apparatuses 20, 30, 31, … via a wireless router by the wireless communication method such as Wi-Fi, Bluetooth, etc. There are no limits to the communication method of the communication interface 21, and the communication interface 21 may use another communication method.
[0098] According to an embodiment, the communication interface 11 may not directly communicate with the plurality of display apparatuses 20, 30, 31, … , but may instead be connected to the splitter 37 as shown in FIG. 1. In this case, the communication interface 11 may transmit data to the splitter 37, so that the splitter 37 can split the data and transmit the split data to the plurality of display apparatuses 20, 30, 31, … .
[0099] The storage 15 is configured to store the 360-degree image 950 received from an external device. The storage 15 causes each piece of the data of the stored 360-degree image 950 to undergo any one or more of reading, writing, editing, deleting, update, etc. The storage 15 may be implemented as a nonvolatile memory such as a flash memory, a hard disk drive, etc. in order to retain data regardless of whether the display apparatus 20 is powered on or off.
[0100] The processor 12 produces a plurality of split images 91, 92, 93, … corresponding to each of a plurality of screens (see screens 81, 82, 83, … of FIG. 21) to be arranged along the mapping great circle 100 of the sphere 120 in the plurality of display apparatuses 20, 30, 31, from each frame of the 360-degree image 950 that includes the plurality of frames.
[0101] In this case, the 360-degree image 950 may be stored in the storage 15, or may, for example, be received from the Blu-ray disc 382, the game playing device 383, the laptop computer 384, and the like connected to the server 10.
[0102] According to an embodiment, the processor 12 may receive information about the number of screens corresponding to the plurality of display apparatuses 20, 30, 31, … and respective positions where the plurality of screens are respectively arranged from each of the display apparatuses 20, 30, 31, … or from the display apparatus 20 that includes the main screen which is being controlled by a user. Alternatively, the processor 12 may receive information about a total number of screens and an arranged position of each screen via a user input to the remote controller or the like.
[0103] The processor 12 transmits each respective one of the plurality of split images 91, 92, 93, … generated according to frames to the corresponding display apparatuses 20, 30, 31, … .
[0104] According to an embodiment, the foregoing operations of the processor 12 may be implemented by a computer program stored in a computer program product provided separately from the server 10.
[0105] In this case, the computer program product includes a memory in which an instruction corresponding to the computer program is stored, and a processor. When executed by the processor, the instruction causes the server 10 to produce the plurality of split images 91, 92, 93, … corresponding to the screens of the plurality of display apparatuses 20, 30, 31, … to be arranged along the mapping great circle 100 of the sphere 120, from the 360-degree image 950, and transmit the plurality of produced split images 91, 92, 93, … to the display apparatuses 20, 30, 31, … .
[0106] Thus, the server 10 may download and execute a computer program stored in a separate computer program product, and perform the operations of the processor 12.
[0107] FIG. 5 is a flowchart of operations between a server and a display apparatus, according to an embodiment. As shown in FIG. 5, at operation S702, the server 10 produces the split images 91, 92, 93, … corresponding to a plurality of screens 81, 82, 83, … to be arranged along the mapping great circle 100 of the sphere 120, from the 360-degree image 950.
[0108] In this aspect, the operation S702 may include an operation of receiving information about a total number of a plurality of screens 81, 82, 83, … and respective positions at which the plurality of screens 81, 82, 83, … are arranged, and an operation of producing the plurality of split images 91, 92, 93, … by determining respective view angles corresponding to the screens 81, 82, 83, … based on the received information.
[0109] Next, at operation S703, the display apparatus 20 receives the split image 91 that corresponds to the screen 81 from the server 10.
[0110] Last, at operation S704, the display apparatus 20 produces and displays the projection image 812 (referring also to FIG. 9) based on the viewpoint from the received split image 91.
[0111] The operation S704 further includes operations of mapping the split image 91 to a corresponding region on the great circle 100 of the sphere 120, and producing a projection image that corresponds to the viewpoint from the mapped split image 91.
[0112] FIG. 6 illustrates a configuration between a server and a plurality of display apparatuses, according to an embodiment. As shown in FIG. 6, the server 10 connects with one display apparatus 20 from among the plurality of display apparatuses 20, 30, 31, 32, 33 and 34. In this case, the plurality of display apparatuses 20, 30, 31, 32, 33 and 34 are connected in series. For example, the display apparatus 20 connected to the server 10 in a daisy chain scheme bypasses a signal to other display apparatuses 30, 31, 32, 33 and 34.
[0113] The server 10 communicates with the display apparatus 20 via a LAN or a wireless communication method, and content and information of the 360-degree image 950 are transmitted and received between them.
[0114] According to an embodiment, the server 10 may transmit data about a unit frame of the 360-degree image 950 having a resolution of 8K to the display apparatus 20. In this case, the server 10 may transmit data that corresponds to the entire region or a partial region of the 360-degree image 950 in accordance with network conditions.
[0115] The display apparatus 20 may determine the view angle of the screen 81 based on a total number of connected screens 81, 82, 83, … and a position at which the screen 81 from among the screens 81, 82, 83, … is arranged. Thus, the split image 91 may be produced by cropping an region that corresponds to the determined view angle of the screen 81 from the frame of the received 360-degree image 950.
[0116] Next, the display apparatus 20 transmits data about the frame of the 360-degree image 950 received from the server 10 to a connected first display apparatus 30.
[0117] The first display apparatus 30 may produce a split image 92, which is based on the total number of screens 81, 82, 83, … and the position at which the screen 82 among all the screens 81, 82, 83, … is arranged, from the data about the frame of the 360-degree image 950 received from the display apparatus 20.
[0118] Similarly, the first display apparatus 30 may transmit the data about the frame of the 360-degree image 950 to the second display apparatus 31, and cause the second display apparatus 32 to produce a split image 93 based on an arranged position of a screen 83 from among all the screens 81, 82, 83, … .
[0119] As described above, when the plurality of display apparatuses 20, 30, 31, 32, 33 and 34 are connected in series to the server 10, each of the display apparatuses 20, 30, 31, 32, 33 and 34 is configured to produce the split image in such a manner that one display apparatus 20 bypasses the data about the frame of the 360-degree image 950 received from the server 10 to another first display apparatus 30.
[0120] FIG. 7 is a block diagram of a display apparatus, according to an embodiment of the present disclosure. As shown in FIG. 7, the display apparatus 20 includes the communication interface 21, the processor 22, the image processor 23, the display 24, the storage 25 and the input receiver 26, and communicates with the server 10 and the first display apparatus 30 via the communication interface 21. The display apparatus 20 may, for example, include any of a TV, a smartphone, a projector, an HMD, etc. There are no limits to the elements of the display apparatus 20, and the display apparatus 20 may further include another element. In this aspect, the elements of the display apparatus 20 are the same as those of FIG. 3, and thus only different features will be described in order to avoid repetitive descriptions.
[0121] In the illustrated configuration, the display apparatus 20 and the plurality of display apparatuses 30, 31 … are connected in series to the server 10, and may, for example, be connected in the daisy chain scheme.
[0122] In particular, the display apparatus 20 transmits a signal and data from the server 10 to the first display apparatus 30, and similarly the first display apparatus 30 transmits the signal and data to the second display apparatus 31.
[0123] According to an embodiment, the processor 22 receives data about the entire 360-image 950 or a partial region of the 360-degree image 950 from the server 10, and produces the split image 91 that corresponds to the screen 81 from the received data of the 360-degree image 950.
[0124] In this case, the processor 22 may determine the view angle of the split image 91 based on the number of screens corresponding to the plurality of display apparatuses 20, 30, 31, … and the arranged position of the screen 81 within the entire plurality of screens.
[0125] Further, the processor 22 transmits data of the 360-degree image 950 received from the server 10 as described above to the connected first display apparatus 30. Thus, the first display apparatus 30 is configured to produce the split image 92 based on the total number of screens and the arranged position of the screen 81 from the data of the 360-degree image 950 received from the display apparatus 20.
[0126] According to an embodiment, the processor 22 may transmit any of information about a user input received via the input receiver 26, information about a change in the view angle, and/or information about rotation control and zoom control, etc. to the first display apparatus 30. For example, the processor 22 transmits remote-controller control information received in the main screen 81 that is controlled by a user to the connected first display apparatus 30, so that the operations of the remote-controller control information can be mutually performed with respect to another screen 82.
[0127] Further, the processor 22 transmits any of information about a change in the view angle caused by a user input, information about rotation control in a pitching or yawing direction, and/or information about zoom-in or zoom-out control to the first display apparatus 30, so that an operation corresponding to a user input can be organically connected to another screen 82.
[0128] FIG. 8 illustrates an example of displaying a projection image in response to user control in the related art. As shown in FIG. 8, when projection images 812, 822 and 832 respectively corresponding to multi-screens 81, 82 and 83 are displayed around an equator of the sphere 120 (as illustrated in item 851), the projection images 812, 822 and 832 may be moved in upward and downward directions or zoom-controlled in response to a user’s input.
[0129] In the illustrated example, when the projection images 812, 822 and 832 are moved up and/or down in response to a user’s input (as illustrated in item 852), the viewpoint moves in accordance with longitude and latitude, thereby causing an overlapping region between the projection images 812, 822 and 832 displayed on the screens.
[0130] Further, when the projection images 812, 822 and 832 are zoom-controlled in response to a user’s input (as illustrated in item 853), the view angles are uniformly adjusted with regard to all the screens 81, 82 and 83 and it is therefore impossible to maintain the same range of the view angle, since total view angles are decreased or increased. Thus, there may be an overlapping region or a discontinuous gap region between the projection images 812, 822 and 832 at the zoom control.
[0131] FIG. 9 illustrates an example of arranging projection images for screens on a great circle of a sphere, according to an embodiment. As shown in FIG. 9, the display apparatus 20 produces projection images 812, 822, 832, … for the screens 81, 82, 83, … by mapping the split images 91, 92, 93, … of the 360-degree image 950 corresponding to the screens 81, 82, 83, … received from the server 10 onto the great circle 100 of the sphere 120.
[0132] In the illustrated example, when each of the multi-screens 81, 82, 83, … is a flat screen (as illustrated in item 131), the surfaces of the great circle 100 onto which the projection images 812, 822, 832, … are projected are flat.
[0133] Further, when each of the multi-screens 81, 82, 83, … is a curved screen (as illustrated in item 132), the surfaces of the great circle 100 onto which the projection images 812, 822, 832, … are projected are curved.
[0134] In this case, a projection method used for each screen may include any of rectilinear projection, stereographic projection, fisheye projection, etc. and is properly determined by taking any of various parameters such as an aspect ratio, curvature, etc. of each screen into account.
[0135] FIG. 10 illustrates a formula to calculate coordinates of a projection image for each screen arranged on a great circle of a sphere at rotation control, according to an embodiment. As shown in FIG. 10, as a result of performing pitching or yawing rotation control for the 360-degree image 950 reproduced on the multi-screens 81, 82, 83, … , the coordinates 140 of the projection images 812, 822, 832, … on the respective screens 81, 82, 83, … are calculated by multiplying the coordinates of the sphere 120 by a projection matrix (P), a rotation matrix (R’) 141 and an angular matrix (M) 142.
[0136] In this case, the projection matrix (P) is defined by a transformation function that corresponds to general projection, and the rotation matrix (R’) 141 is defined by a function about change in the viewpoint of a control reference point 850.
[0137] Further, the angular matrix (M) 142 is defined by using angular distances (.DELTA..theta., .DELTA..PHI.) of respective center points 811, 821, 831, … of the screens 81, 82, 83, … with respect to the control reference point 850.
[0138] FIG. 11 illustrates an example of a shape of a great circle displayed on a sphere and a 360-degree image at rotation control, according to an embodiment. As shown in FIG. 11, as a result of performing rotation control for the 360-degree image 950 reproduced on the multi-screens 81, 82, 83, … , the shapes of the great circles 100, 101, 102 displayed on the sphere 120 and the 360-degree image 950 are varied based on a pitching rotation angle (.theta.) and a yawing rotation angle (.PHI.).
[0139] In the illustrated example, at pitching rotation control, a projection image 8122 is displayed on a great circle 102 pitching as indicated by a rotation angle (.theta.). Further, at yawing rotation control, a projection image 8123 is displayed on a great circle 103 yawing as indicated by a rotation angle (.PHI.).
[0140] FIG. 12 illustrates an example of a projection region for each individual screen on a 360-degree image, according to an embodiment. As shown in FIG. 12, when the great circle 100 is aligned with the equator, a projection region 80 where the projection images 812, 822, 832, … of the multi-screens 81, 82, 83, … are displayed is displayed in the form of a straight line that is parallel to a equator line of the 360-degree image 950
[0141] FIG. 13 illustrates an example of a projection region for each individual screen on a 360-degree image, according to an embodiment. As shown in FIG. 13, when the pitching rotation control is performed, the projection region 80 is displayed such that a middle portion thereof is moved upward based on the rotation angle (.theta.) from the equator line of the 360-degree image 950.
[0142] In this case, the shape of the projection region 80 is varied based on the sizes of the rotation angle (.theta.) (e.g. 30 degrees, 60 degrees and 90 degrees).
[0143] Thus, when the pitching rotation operation is performed, the rotation angle of the great circle 100 is taken into account to seamlessly display images on the screens without an overlapping region or a lost region.
[0144] FIG. 14 illustrates an example of a projection region for each individual screen on a 360-degree image, according to an embodiment. As shown in FIG. 14, when a yawing rotation control is performed, the projection region 80 is displayed as having moved rightward based on the rotation angle (.PHI.) from the center of the 360-degree image 950.
[0145] In this case, the shape of the projection region 80 is varied based on the sizes of the rotation angle (.PHI.) (e.g. 0 degrees, 30 degrees).
[0146] Thus, when the yawing rotation operation is performed, the rotation angle and direction of the great circle 100 are taken into account to seamlessly display images on the screens without an overlapping region or a lost region.
[0147] FIG. 15 illustrates an example of a projection image displayed on each individual screen at rotation control, according to an embodiment. As shown in FIG. 15, as a result of the pitching rotation control, projection images 812, 822, 832, 842, 852 and 862 are displayed corresponding to the screens 81, 82, 83, 84, 85 and 86 on the great circle 100 of the 360-degree image 950.
[0148] By the foregoing methods, the projection images 812, 822, 832, 842, 852 and 862 respectively displayed on the screens 81, 82, 83, 84, 85 and 86 are seamlessly displayed without an overlapped or lost region between them.
[0149] FIG. 16 illustrates an example of change in a projection image displayed on each individual screen at rotation control, according to an embodiment. As shown in FIG. 16, when the 360-degree image 950 is subjected to the pitching rotation control, projection images 822 and 832 displayed on left and right end screens 82 and 83 are rotated in a rolling direction with respect to the main screen 81 which includes the control reference point 850.
[0150] In the illustrated example, in the state 200 that a user watches the 360-degree image 950 while looking at the main screen 81 from the front, when pitching rotation control is performed as much as a rotation angle of 30 degrees (as illustrated in item 201), the projection images 822 and 832 are displayed as rotated in a rolling direction as much as the angle of about 30 degrees.
[0151] In this case, when a user further performs the pitching rotation control as much as an angle of about 45 degrees (as illustrated in item 202), the projection images 822 and 832 are displayed as rotated in a rolling direction as much as the angle of about 45 degrees.
[0152] As described above, according to an embodiment, the projection images respectively displayed on the screens are seamlessly displayed without an overlapping or lost region even when the 360-degree image 950 is subjected to the rotation control.
[0153] FIG. 17 is a graph of showing view-angle setting values in accordance with viewpoints of each individual screen at zoom control, according to an embodiment. As shown in FIG. 17, when the 360-degree image 950 is controlled to be zoomed in or zoomed out, setting values for the view angle may be derived in accordance with the angular distances (see .DELTA..theta. and .DELTA..PHI. of FIG. 29) from the control reference point 850.
[0154] The setting values for the view angle are required to have successive values according to the pitching and yawing rotation angles (see .theta. and .PHI. of FIG. 11). Thus, the projection images displayed on the screens are seamlessly connected.
[0155] Further, values obtained by integrating the setting values for the view angle are required to be within a total range of view angle. In particular, when the view angle of one screen is decreased by zoom-in control, the view angles of the other screens are increased within the total range of the view angle.
[0156] In the illustrated example, when the horizontal view angles for the multi-screens have a total range of 160 degrees and the control reference point 850 has an angle of 100 degrees, the graph of the setting values for the view angle is varied based on zoom-in control 211 and zoom-out control 212.
[0157] First, in a case 210 of the setting values for the view angle before the zoom control, each screen has a uniform view angle throughout the total range of the view angles. In a case 211 of the setting values for the view angle resulting from the zoom-in control, the view angle of the screen corresponding to the angle of 100 degrees at the control reference point 850 is decreased, but the view angles of the other screens are increased within the total range of the view angle.
[0158] Further, in a case 212 of the setting values for the view angle resulting from the zoom-out control, the view angle of the screen corresponding to the angle of 100 degrees at the control reference point 850 is increased, but the view angles of the other screens are decreased within the total range of the view angle.
[0159] According to an embodiment, the total range of the view angle displayed on the multi-screens may be determined by using at least one from among the following methods.
[0160] First, when the multi-screens are arranged in a circular form, the total range of the view angle is maintained to a maximum of 360 degrees.
[0161] Second, the total range of the view angle may range from a to b degrees.
[0162] Third, the view angle may be greater than c degrees so that a major object can be always displayed on the screen within the 360-degree image 950. In this case, the major object may be selected in response to a user input or determined based on an analysis of the 360-degree image 950.
[0163] According to an embodiment, the control reference point 850 may be changed for the zoom control. For example, a reference point may be generated at a position that is different from the control reference point 850 without returning to an original view angle after zooming in or out from the control reference point 850.
[0164] Alternatively, operations may be made with respect to another reference point at a different position from the control reference point 850 after returning to the original view angle from the zoom-in or zoom-out operation.
[0165] Alternatively, operations may be made with respect to a new reference point while maintaining the view angle changed by the zoom-in or zoom-out operation.
[0166] FIG. 18 is a graph of a function of view-angle setting values, according to an embodiment. As shown in FIG. 18, a function of the setting values for the view angle with respect to the control reference point 850 may be variously represented in any of a Gaussian graph 220, a pyramid graph 221, etc.
[0167] For example, in a case of the Gaussian graph 220, the area of the region where the setting values for the view angle are continuously changed with respect to the control reference point 850 in accordance with variance, the difference in the setting value for the view angle between the viewpoint of the control reference point 850 and the farthest viewpoint, etc. may be determined.
[0168] In this case, the function of the setting values for the view angle may be predetermined, or may be optimized in accordance with a usage log of a user.
[0169] FIG. 19 illustrates an example of change in a view angle with regard to each screen at zoom control, according to an embodiment. As shown in FIG. 19, in a state that the projection images 812, 822 and 832 having a uniform view angle are displayed on the screens 81, 82 and 83 while reproducing the 360-degree image 950, the middle screen 81 may include the control reference point 850 and be subjected to the zoom-in control.
[0170] In the illustrated example, a projection image 8125 having a narrower view angle than that before the zoom-in control is displayed on the middle screen 81 in response to the zoom-in control. In this case, the other screens 82 and 83 respectively display projection images 8225 and 8325, which have a wider view angle than those before the zoom-in control, so as to maintain the total range of the view angle of all the screens 81, 82 and 88.
[0171] Further, by the zoom-in control, an object displayed on the middle screen 81 is enlarged, and objects displayed on the other screens 82 and 83 are reduced.
[0172] As described above, the total range of the view angle for all the screens 81, 82 and 83 are maintained during the zoom control, and therefore the 360-degree image 950 is displayed without losing a major object on each screen.
[0173] FIG. 23 is a flowchart showing operations of a display apparatus at rotation control, according to an embodiment. As shown in FIG. 23, a user’s input, a sensor’s sensing, a user recognition, etc. are performed at operation S2711, and the control reference point 850 for controlling the 360-degree image 950 is set at operation S2712. Further, a user’s input, metadata referring, connection auto-sensing, etc. are performed at operation S2721, and an arranged state of screens is set at operation S2722.
[0174] Thus, when the control reference point and the arranged state of the screen are completely set, the view angle of each screen and the initial value of the viewpoint are determined at operation S273, and the respective projection center corresponding to the viewpoint on each screen is set at operation S274.
[0175] Next, at operation S275, a rotation matrix (R’) 141 and an angular matrix (M) 142 are derived for calculating the coordinates of the respective projection image displayed on each screen. In this aspect, the rotation matrix (R’) 141 is defined by a function about the change of the viewpoint at the control reference point 850. Further, the angular matrix (M) 142 is defined by using the angular distances (.DELTA..theta., .DELTA..PHI.) at the center point of each screen from the control reference point 850.
[0176] At operation S276, the rotation matrix (R’) 141 and the angular matrix (M) 142 are used to calculate the coordinates of the respective projection image projected onto each corresponding screen, thereby displaying the projection image on each screen.
[0177] Next, when the pitching or yawing rotation control is performed with regard to the projection image by an input device such as a remote controller, a touch pad, a motion sensor, etc. at operation S277, the viewpoint of the control reference point 850 is changed at operation S278.
[0178] Then, the operation S275 and the operation S276 are performed again to display the projection image changed by the rotation control, and the following operations are repeated.
[0179] According to the foregoing embodiment, the rotation angle and direction of the great circle are taken into account at the pitching rotation control and/or the yawing rotation control, so that the images displayed on the screens are seamlessly displayed without an overlapping or lost region.
[0180] FIG. 24 is a flowchart showing operations of a display apparatus at zoom control, according to an embodiment. As shown in FIG. 24, a user’s input, a sensor’s sensing, a user recognition, etc. are performed at operation S2811, and the control reference point 850 for controlling the 360-degree image 950 is set at operation S2812. Further, a user’s input, metadata referring, connection auto-sensing, etc. are performed at operation S2821, and an arranged state of screens is set at operation S2822.
[0181] Thus, when the control reference point and the arranged state of the screen are completely set, the view angle of each screen and the initial value of the viewpoint are determined at operation S283, and the setting values for the view angles according to the screens are calculated at operation S284. In this case, the setting values for the view angles are derived based on the angular distances (.DELTA..theta., .DELTA..PHI.) from the control reference point 850, and have continuous values in accordance with the pitching and yawing rotation angles (.theta., .PHI.).
[0182] In this case, the respective setting values for the view angles according to the screens are set to maintain the total range of the view angle, and therefore a major object in the projection image displayed on each screen is not lost.
[0183] Next, the respective projection center corresponding to the viewpoint of each screen is set at operation S285, and the respective projection image is displayed on each corresponding screen at operation S286.
[0184] Next, at operation S287, when zoom control for the projection image is performed via any of a remote controller, a touch pad, a motion sensor and the like input device, the setting values for the view angles according to the screens are calculated again at operation S284.
[0185] Then, the operation S285 and the operation S286 are performed again to display the projection image changed by the zoom control, and the following operations are repeated.
[0186] According to the foregoing embodiment, the total range of the view angle is maintained during the zoom-in or zoom-out control, so that an image in the screen can be displayed without losing a major object.
[0187] FIG. 25 illustrates an example of selecting multi-screens within a space, according to an embodiment. As shown in FIG. 25, according to an embodiment, the 360-degree image may be reproduced and controlled based on the multi-screens selected by a user via a UI that includes images corresponding to apparatuses arranged in a surrounding space.
[0188] According to an embodiment, a user may select a plurality of screens, placed at various positions in a home, via the UI, thereby configuring the multi-screens 81, 82 and 83. In this case, each of the multi-screens 81, 82 and 83 may include a physical and independent screen, and a virtual screen created by an HMD or the like. Further, when a projector projects an image to an object, the object may be regarded as one of the multi-screens. In addition, flat and curved screens are all possible for the multi-screens 81, 82 and 83.
[0189] The multi-screens 81, 82 and 83 may be connected in up, down, left and right directions. Alternatively, the screens may be arranged so as to leave a space therebetween. Further, there are no limits to the arranged shape of the multi-screens 81, 82 and 83. For example, the multi-screens may be arranged having a cylindrical or dome shape so as to surround a user.
[0190] According to the foregoing embodiment, a user may directly select the multi-screens for reproducing the 360-degree image from among peripheral apparatuses, and watch the 360-degree image through a desired apparatus.
[0191] FIG. 26 illustrates an example of a user interface (UI) for selecting multi-screens to display a 360-degree image, according to an embodiment. As shown in FIG. 26, the display apparatus 20 according to the embodiment may display a screen setting UI 105 for selecting a plurality of screens from among the screens placed within a space to reproduce the 360-degree image 950.
[0192] According to an embodiment, a user may select a menu for setting a plurality of screens from among a plurality of menus provided by the display apparatus 20. In this case, the screen setting UI 105 is displayed on a screen in response to a user’s menu selection, and the screen setting UI 105 shows a 360-degree mode setting menu 106 and an apparatus selecting menu 107.
[0193] A user may select the 360-degree mode setting menu 106, and select a plurality of apparatuses connected via a network on the apparatus selecting menu 107.
[0194] In the illustrated example, a user selects a living room TV, a main room TV, a main-room beam-projector, and a room #1 PC from among the connectable apparatuses via the apparatus selecting menu 107.
[0195] Thus, a user can more easily select the multi-screens for reproducing the 360-degree image via the UI displayed on the screen.
[0196] FIG. 27 illustrates an example of a UI for mapping selected multi-screens to a sphere, according to an embodiment. As shown in FIG. 27, the display apparatus 20 may display a sphere mapping UI 109 for setting a position of a sphere 121, to which a respective split image corresponding to each screen is mapped, with regard to the multi-screens selected via the screen setting UI (see item 105 of FIG. 26). In this case, a user can easily set the position of the sphere 121, to which the split image corresponding to each screen is mapped, via the UI by a visual method.
[0197] In the illustrated example, the sphere mapping UI 109 shows a position selecting menu 108 for sphere mapping, so that the living room TV, the main room TV, the main-room beam-projector and the room #1 PC previously selected via the screen setting UI 105 can be mapped to the desired regions of the sphere 121. For example, the living room TV, the main room TV, the main-room beam-projector and the room #1 PC may be respectively mapped to a region #2, a region #3, a region #1 and a region #4 as illustrated.
[0198] In this case, a user may directly input a desired mapping-region number of the sphere 121, and may drag and map an corresponding apparatus icon to a desired position on the sphere 120.
[0199] Alternatively, the display apparatus 20 may automatically recognize the positions of the multi-screens selected on the screen setting UI 105 by an image sensor or the like of a camera, and map each screen to the region of the sphere 121 in accordance with the recognized positions.
[0200] FIG. 28 illustrates an example of configuring a virtual sphere by connecting center points of screens, according to an embodiment. As shown in FIG. 28, the display apparatus 20 according to the present embodiment may configure a virtual sphere 121 through center points 811, 821, 831 and 841 of the respective screens so as to reproduce and control the 360-degree image in the multi-screens 81, 82, 83 and 84.
[0201] In the illustrated example, the virtual sphere 121 may be configured by connecting the center points 811, 821, 831 and 841 of the respective screens. Alternatively, when the center points 811, 821, 831 and 841 of the respective screens are not suitable for configuring one sphere, the sphere 121 may be configured to have the minimum sum of distances between the center of the virtual sphere 121 and the center points.
[0202] FIG. 29 illustrates an example of determining a respective projection center of each of a plurality of screens by configuring a virtual sphere, according to an embodiment. As shown in FIG. 29, the display apparatus 20 according to the present embodiment may calculate the viewpoint of each screen, i.e. the center point of the respective projection image displayed on each of the multi-screens 81, 82, 83 and 84, from the virtual sphere (see 121 in FIG. 28) configured by connecting the center points 811, 821, 831 and 841 of the multi-screens 81, 82, 83 and 84 selected by a user’s input.
[0203] To this end, a method of setting the control reference point 850 of the multi-screens 81, 82, 83 and 84 and determining the projection center will be described below with reference to FIGS. 20, 21, and 22.
[0204] As shown in FIG. 20, the display apparatus 20 may first set the control reference point 850 with regard to the multi-screens 81, 82, 83, … . The control reference point 850 refers to a reference position for controlling the 360-degree image 950 with regard to the multi-screens 81, 82, 83, … , and any of various positions inside or outside the multi-screens 81, 82, 83, … may be set as the control reference point 850.
[0205] For example, the control reference point 850 may be positioned at the center of the multi-screens 81, 82, 83, … , or may be set as the center point 1 of the screen 81 convenient for a user to do watching or controlling.
[0206] Further, the control reference point 850 may be set as a dot 2 positioned at an off-center location within an inside region of one screen 81 from among the multi-screens 81, 82, 83, … .
[0207] Alternatively, the control reference point 850 may be set as a dot 3 positioned in a connection portion, for example, a bezel, between the multi-screens 81, 82, 83, … or may be set as a virtual dot 4 outside the screen when the screens are not in contact with each other.
[0208] Further, the control reference point 850 may be set based on an angle directly input by a user, or may be set based on an angle that corresponds to a position of a recognized user’s action or eye direction. Further, in the state that a sensor is attached to each screen, when a user inputs a signal to a desired screen via a remote controller, the center point of the corresponding screen may be set as the control reference point 850.
[0209] Further, sensors may be attached to the multi-screens 81, 82, 83, … at regular angular intervals, and the control reference point 850 may be set as a desired angle based on a user input. In this case, the control reference point 850 may be set based on an angle between the sensors and further based on the strength of the signals received in the sensors.
[0210] In the example illustrated in FIG. 21, the display apparatus 20 sets the center point 811 of the main screen 81 as the control reference point 850, and determines the respective projection centers 813, 823, 833, … according to the screens 81, 82, 83, … .
[0211] In this case, the projection center 813 of the main screen 81 is aligned with the control reference point 850, and the projection centers 823, 833, … of the other screens 82, 83, … are calculated with respect to the control reference point 850 and then aligned with the center points 821, 831 of the screens 82, 83, … .
[0212] In the example illustrated in FIG. 22, the display apparatus 20 sets the control reference point 850 to be positioned at an off-center location within an inside region or an outside of the screen 81, and determines the projection center 813, 823, 833, … according to the screens 81, 82, 83, … . In this case, the projection centers 813, 823, 833, … of all the screens 81, 82, 83, … are obtained with respect to the control reference point 850, and then aligned with the center point 811, 821 and 831 of the respective screens 81, 82, 83, … .
[0213] In the example illustrated in FIG. 29, the projection centers 813, 823, 833 and 844 respectively displayed on each of the multi-screens 81, 82, 83 and 84 from the virtual sphere 121 may be calculated using the angular distances (.DELTA..theta., .DELTA..PHI.) of the center points 811, 821, 831 and 841 of the respective screens 81, 82, 83 and 84 with respect to the control reference point 850.
[0214] For example, when the viewpoint corresponding to the control reference point 850 is (.theta..sub.0, .PHI..sub.0), the viewpoint corresponding to the center point 821 of the screen 2, i.e., the projection center 823, is calculated by (.theta..sub.0+.DELTA..theta., .PHI..sub.0+.DELTA..PHI.).
[0215] By such a method, the control reference point 850 is set with respect to the multi-screens 81, 82, 83 and 84, and thus the viewpoint of each screen is obtained from the virtual sphere 121 formed by connecting the center points of the screens selected by a user.
[0216] As described above, according to the present disclosure, it is possible to control the 360-degree image on the multi-screens.
[0217] Further, according to the present disclosure, when the 360-degree image is controlled on the multi-screens, regions displayed on the screens can be viewed without an overlapping or lost region even though the viewpoint and the view angle are changed.
[0218] Although a few embodiments have been shown and described, it will be appreciated by persons having ordinary skill in the art that changes may be made in these embodiments without departing from the principles and spirit of the present disclosure, the scope of which is defined in the appended claims and their equivalents.
