Apple Patent | Adaptive resolution and projection format in multi-directional video

Patent: Adaptive resolution and projection format in multi-directional video

Publication Number: 20190004414

Publication Date: 2019-01-03

Applicants: Apple

Abstract

Techniques are described for implementing format configurations for multi-directional video and for switching between them. Source images may be assigned to formats that may change during a coding session. When a change occurs between formats, video coders and decoder may transform decoded reference frames from the first format to the second format. Thereafter, new frames in the second configuration may be coded or decoded predictively using transformed reference frame(s) as source(s) of prediction. In this manner, video coders and decoders may use intra-coding techniques and achieve high efficiency in coding.

Background

The present disclosure relates to coding/decoding systems for multi-directional imaging system and, in particular, to use of coding techniques that originally were developed for flat images, for multi-directional image data.

In multi-directional imaging, a two-dimensional image represents image content taken from multiple fields of view. Omnidirectional imaging is one type of multi-directional imaging where a single image represents content viewable from a single vantage point in all directions -360.degree. horizontally about the vantage point and 360.degree. vertically about the vantage point. Other multi-directional images may capture data in fields of view that are not fully 360.degree..

Modern coding protocols tend to be inefficient when coding multi-directional images. Multi-directional images tend to allocate real estate within the images to the different fields of view essentially in a fixed manner. For example, in many multi-directional imaging formats, different fields of view may be allocated space in the multi-directional image equally. Some other multi-directional imaging formats allocate space unequally but in a fixed manner. And, many applications that consume multi-directional imaging tend to use only a portion of the multi-directional image during rendering, which causes resources spent to code un-used portions of the multi-directional image to be wasted.

Accordingly, the inventors recognized a need to improve coding systems to increase efficiency of multi-directional image data.