Apple Patent | Synchronized, interactive augmented reality displays for multifunction devices
Patent: Synchronized, interactive augmented reality displays for multifunction devices
Publication Number: 10176637
Publication Date: 2019-01-08
Applicants: Apple
Abstract
A device can receive live video of a real-world, physical environment on a touch sensitive surface. One or more objects can be identified in the live video. An information layer can be generated related to the objects. In some implementations, the information layer can include annotations made by a user through the touch sensitive surface. The information layer and live video can be combined in a display of the device. Data can be received from one or more onboard sensors indicating that the device is in motion. The sensor data can be used to synchronize the live video and the information layer as the perspective of video camera view changes due to the motion. The live video and information layer can be shared with other devices over a communication link.
Background
Augmented Reality (AR) technology combines a live view of a real-world, physical environment with computer-generated imagery. Information about the real world environment can be stored and retrieved as an information layer which can be overlaid on the live view and interacted with by a user. Despite strong academic and commercial interest in AR systems, many existing AR systems are complex and expensive making such systems unsuitable for general use by the average consumer.
Summary
A device can receive images and/or live video of a real-world, physical environment on a touch sensitive surface. One or more objects can be identified in the live video. One or more information layers can be generated related to the objects. In some implementations, an information layer can include annotations made by a user through the touch sensitive surface. The information layer and live video can be combined in a display of the device. Data can be received from one or more onboard sensors indicating that the device is in motion. The sensor data can be used to synchronize the live video and the information layer as the perspective of video camera view changes due to the motion. The live video and information layer can be shared with other devices over a communication link.
In one embodiment, a device can provide a split screen display that can include a first display area for displaying the live video combined with the information layer and a second display area for displaying computer-generated imagery representing objects in the live video. The computer-generated imagery can be combined with the information layer in the second display area. A navigation control for allowing the user to navigate the computer-generated imagery can be provided with the split screen display. Alternatively, the user can navigate the computer-generated imagery by physically moving the device.
