Intel Patent | Audio, Video, and Actuation (a/v/a) Synchronization for Mixed Reality

Patent: Audio, Video, and Actuation (a/v/a) Synchronization for Mixed Reality

Publication Number: 20190007726

Publication Date: 2019-01-03

Applicants: Intel

Abstract

Systems, apparatuses and methods for monitoring and adjusting audio/video/actuation (A/V/A) synchronization to ensure immersive mixed reality experiences. The method includes preparing the mixed reality content with timestamp metadata in the content stream to describe modality synchronization requirements. The mixed reality content is presented. Synchronization between components of the mixed reality content is monitored. The system determines whether the components of the mixed reality content are in synchronization. If they are not in synchronization, the system may take action to improve the perception of the A/V/A synchronization.

Background

Humans have a limited tolerance for viewing media in which the audio and video are out of sync. For example, the Advanced Television Systems Committee recommends that audio/video sync be within 15 ms when audio leads video and 45 ms when audio lags video to meet user expectations.

With virtual reality (VR) and augmented reality (AR) added to the mix, additional sync issues may arise. For example, VR and/or AR systems with wearables or objects that supply haptic output not only need the audio and video to be in sync, but the haptic output must be in sync with the video and/or audio output.