Facebook Patent | Systems and Methods for Occluding Images and Videos Subject to Augmented-Reality Effects

Patent: Systems and Methods for Occluding Images and Videos Subject to Augmented-Reality Effects

Publication Number: 20180357480

Publication Date: 2018-12-13

Applicants: Facebook, Inc.

Abstract

 In one embodiment, a method includes a system accessing an image, which may comprise covered and uncovered portions, and an overlay image comprising opaque pixels. The covered portion may be configured to be covered by the opaque pixels of the overlay image. The system may generate a data structure comprising data elements associated with pixels of the image. Each of the data elements associated with a covered pixel in the covered portion of the image may be configured to identify an uncovered pixel in the uncovered portion of the image that is closest to the covered pixel. Each covered pixel in the covered portion of the image may be modified by accessing the data element associated with the covered pixel, determining a distance between the covered pixel and an associated closest uncovered pixel using the accessed data element, and modifying a color of the covered pixel based on the distance.

BACKGROUND 

Augmented reality provides a view of the real or physical world with added computer-generated sensory inputs (e.g., visual, audible). In other words, computer-generated virtual effects augment or supplement the real-world view. For example, a mobile phone with a camera may capture a real-world scene (as an image or video) and display a composite of the captured scene with computer-generated objects. The virtual objects may be, for example, two-dimensional and/or three-dimensional objects, and it may be stationary or animated. The virtual objects may be configured to be conceptually positioned anywhere within the captured scene. For example, from the perspective of a viewer, a virtual object may be positioned behind or in front of an object in the scene (e.g., a tree, a person, etc.). When the virtual object appears in front of the object in the scene, the virtual object in effect covers a portion of that object. Thus, the covered portion may not be visible to the viewer. In certain circumstances, a user who created an image/video with augmented-reality effects may have intended for the covered portion of the underlying image/video to be concealed.

A social-networking system, which may include a social-networking website, may enable its users (such as persons or organizations) to interact with it and with each other through it. The social-networking system may, with input from a user, create and store in the social-networking system a user profile associated with the user. The user profile may include demographic information, communication-channel information, and information on personal interests of the user. The social-networking system may also, with input from a user, create and store a record of relationships of the user with other users of the social-networking system, as well as provide services (e.g., wall posts, photo-sharing, event organization, messaging, games, or advertisements) to facilitate social interaction between or among users.

The social-networking system may send over one or more networks content or messages related to its services to a mobile or other computing device of a user. A user may also install software applications on a mobile or other computing device of the user for accessing a user profile of the user and other data within the social-networking system. The social-networking system may generate a personalized set of content objects to display to a user, such as a newsfeed of aggregated stories of other users connected to the user. One user may also send images and videos to other users of the social-networking system, such as those who are directly or indirectly connected to the user.

A mobile computing device--such as a smartphone, tablet computer, or laptop computer--may include functionality for determining its location, direction, or orientation, such as a GPS receiver, compass, gyroscope, or accelerometer. Such a device may also include functionality for wireless communication, such as BLUETOOTH communication, near-field communication (NFC), or infrared (IR) communication or communication with a wireless local area networks (WLANs) or cellular-telephone network. Such a device may also include one or more cameras, scanners, touch screens, microphones, or speakers. Mobile computing devices may also execute software applications, such as games, web browsers, or social-networking applications. With social-networking applications, users may connect, communicate, and share information with other users in their social networks.

SUMMARY

Particular embodiments of this disclosure relate to systems and methods of occluding portions of an image/video that is intended to be covered by an overlay image (e.g., as part of an augmented-reality effect). In particular embodiments, pixels of the original image/video that is to be covered by an overlay image may be replaced so those pixels could not be uncovered. The covered region of the image/video may comprise border pixels that make up the border of the covered region (but still within the covered region) and interior pixels within the border. In particular embodiments, the interior pixels may be replaced with a masking color, such as black; pixels at the border, on the other hand, may copy color information from adjacent uncovered pixels. By duplicating color information at the border, compression artifacts outside the border may be improved since the compression algorithm would not use the masking cover to determine the color of those uncovered pixels outside and adjacent to the border. Instead, the colors of those uncovered pixels may be determined using their original color, which was copied into the border pixels of the covered region. This particular embodiment does not use or retain any color information within the covered region, which furthers the goal of ensuring that the covered region remains private.

The embodiments disclosed herein are only examples, and the scope of this disclosure is not limited to them. Particular embodiments may include all, some, or none of the components, elements, features, functions, operations, or steps of the embodiments disclosed above. Embodiments according to the invention are in particular disclosed in the attached claims directed to a method, a storage medium, a system and a computer program product, wherein any feature mentioned in one claim category, e.g. method, can be claimed in another claim category, e.g. system, as well. The dependencies or references back in the attached claims are chosen for formal reasons only. However any subject matter resulting from a deliberate reference back to any previous claims (in particular multiple dependencies) can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims. The subject-matter which can be claimed comprises not only the combinations of features as set out in the attached claims but also any other combination of features in the claims, wherein each feature mentioned in the claims can be combined with any other feature or combination of other features in the claims. Furthermore, any of the embodiments and features described or depicted herein can be claimed in a separate claim and/or in any combination with any embodiment or feature described or depicted herein or with any of the features of the attached claims.