LG Patent | Point cloud data transmission device, point cloud data transmission method, point cloud data reception device and point cloud data reception method

The current G-PCC uses, as the qS, the following real numbers: 1, 1.25, 1.5, 1.75, 2, 2.5, 3, 3.5, 4, 5, 6, 7, 8, 10, 12, 14, 16, 20, 24, 28, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 384, 448, 512, 640, 768, 896, 1024, and so on. The larger the qS is, the more quantization is, and thus a high rate of lossy compression is achieved. That is, in this document, the qS refers to the step size used for the following rates: r01 to r06. In the case of the lossy compression, the bitrate may be determined, and the bitrate may be adjusted by the following equation: sampling rate=bitrate/quantization per sample.

According to embodiments, the quantization process is performed immediately after point cloud data is input. When the quantization process is performed, one or more points may have the same quantized position (or position value) based on the quantization step size.

FIG. 15(a) to FIG. 15(d) are diagrams illustrating exemplary quantization methods based on quantization scales.

Specifically, FIG. 15(a) illustrates original (source) points input for quantization. FIG. 15(b) illustrates an exemplary quantization method with a quantization scale of 0.5, FIG. 15(c) illustrates an exemplary quantization method with a quantization scale of 0.25, and FIG. 15(d) illustrates an exemplary quantization method with a quantization scale of 0.125. In this document, the term “quantization scale” refers to a value used for scaling. In one embodiment, specific scaling values are designated for each step.

Assuming that five original points are positioned in a 5×5 space as shown in FIG. 15(a), if the quantization scale is 0.5 as shown in FIG. 15(b), the precision of all points is reduced by half (i.e., ½). For example, in FIG. 15(b), points within a range of (0, 0) to (1, 1) correspond (or map) to a point of (0, 0). If the quantization scale is 0.25 as shown in FIG. 15(c), the precision of all points is reduced by a factor of 0.25 (i.e., ¼). For example, in FIG. 15(c), points within a range of (0, 0) to (2, 2) correspond (or map) to the point of (0, 0). If the quantization scale is 0.125 as shown in FIG. 15(d), the precision of all points is reduced by a factor of 0.125 (i.e., ⅛). For example, in FIG. 15(d), points within a range of (0, 0) to (4, 4) correspond (or map) to the point of (0, 0). In other words, as the quantization scale decreases, the number of lost points increases because more points in space correspond to (or map) to one point.

When quantization is applied as described above, one or more points may have the same quantized position (or position value).

In this case, after the quantization, points may be mapped to a position of (0, 0, 0) 50001 as shown in FIG. 16. Alternatively, the points may be mapped to a position within a space from (0, 0, 1) to (1, 1, 1) or a central position of (0.5, 0.5, 0.5), depending on the precision or characteristics of point cloud.

FIG. 16 is a diagram illustrating exemplary positions of points after quantization according to embodiments.

According to embodiments, an octree structure is generated based on quantized points after the quantization is performed.

According to embodiments, levels are defined in order to transmit an octree with a specific precision level. The space is divided into 8 equal parts up to a depth (L) that represents the geometry precision of the octree maximum level.

To perform lossy compression at the maximum depth of L, the geometry precision is adjusted to level L-M (where M represents the quantization step). The points from level L-M to level L are quantized, and modified leaf nodes are positioned at level L-M. In other words, the points from level L-M to level L, that is, from the lower level to the maximum level, are quantized by dividing level L-M by precision, and the modified leaf nodes are positioned at level L-M. Moreover, the adjusted geometry precision has a geometry value modified to the central value of an octree node or a value defined for the octree node. Then, geometry compression is executed. In this case, octree-based geometry compression is carried out based on lossless compression.

However, as described above, when geometry compression is performed by generating an octree structure based on quantized points, recoloring is essential before performing attribute encoding (also called compression).

According to embodiments, recoloring refers to a process of calculating the average of the attributes of neighboring points within a specific position/radius from the position of a quantized point and setting the average to the attributes of the quantized point. According to embodiments, recoloring is referred to as attribute transform or color readjustment. In addition, recoloring is performed by the attribute transformer 40007 shown in FIG. 4 or the attribute transform processor 12009 shown in FIG. 12. For example, assuming that points at level L are mapped to a specific point at level L-M according to the above-described quantization, since the attributes of the points at level L are unknown due to the quantization, the attributes of the points at level L-M need to be configured by recoloring.

Hereinafter, a recoloring method will be described.

(2) For each point in the reconstructed point cloud, H(i) is defined as a set of original points. In other words, H(i) represents a set of neighboring points.

(3) In this case, points that share X_i are designated as close neighboring points of the reconstructed points, and H(i) may be empty or have one or more values. In this case, if H(i) is empty, it means that there are no points within a specific distance when searching for the close neighboring points.

(4-1) If H(i) is empty, the attribute value a_i* is assigned as the attribute value of the point Xi. That is, one point corresponds to one attribute.

(4-2) If H(i) is not empty, the attribute value a_i is calculated according to Equation 6 below.

$\begin{array}{cc}{\tilde{a}}_i=\frac{1}{H\left(i\right)}{\sum }_{h=1}^{H\left(i\right)}{a}_i^{+}\left(h\right)& \left[\mathrm{Equation}6\right]\end{array}$

Therefore, when the attribute value a_i* of a point is assigned as the attribute values of points that are not related at all by the recoloring process based on Equation 6, the attribute value a_i* of the point is reconstructed into an attribute value different from that of the original point cloud due to the average calculation of attribute values. For example, while the attribute value (i.e., color value) of an original point is red, the attribute value reconstructed according to Equation 6 may be a color other than red. For example, the attribute value (i.e., color value) of the original point might be red, whereas the attribute value reconstructed according to Equation 6 could become a different color other than red.

When the attribute value is reconstructed into a different value by recoloring, performing attribute compression based on the different value may result in a decrease in attribute compression efficiency. In other words, this leads to an issue that the quality of attribute encoding/decoding of point cloud data is degraded.

To address the issue that it is difficult to obtain accurate attribute values, the present disclosure proposes to set the attribute value of a point in reconstructed point cloud data to the attribute value of the original point, thereby improving the quality of attribute encoding/decoding of the point cloud data and enhancing the efficiency of geometry compression.

The present disclosure proposes two methods. One method involves performing quantization of input points for geometry compression, creating an octree structure based on the quantized points, and then performing attribute encoding based on the attribute values of original points. In this document, the method is referred to as a first embodiment. Another method involves performing sampling of input points for geometry compression, creating an octree structure based on the sampled points, and performing attribute encoding based on the attribute values of original points. In this document, the method is referred to as a second embodiment.

In summary, the present disclosure aims to perform geometry compression based on an octree structure and use the attribute values of original points in performing attribute encoding based on the reconstructed geometry without recoloring.

In the present disclosure, the above process is referred to as a single recoloring process. In other words, the single recoloring process means that the attribute values of original points are directly employed in attribute encoding. Therefore, the single recoloring process may be referred to as a ‘recoloring omission process’.

In the first embodiment, the single recoloring process is performed for a representative point in performing steps (1) to (3) among the above-described recoloring steps. In other words, if H(i) is not empty, the attribute value of one point (e.g., second point) among i points is determined to be the attribute value a_i of a reconstructed point according to Equation 7 below.

$\begin{array}{cc}{\tilde{a}}_i=\frac{a_i^{+}}{H\left(i\right)}& \left[\mathrm{Equation}7\right]\end{array}$

In other words, when H(i) is not empty, the average calculation equation for obtaining the attribute average is not used.

According to the first embodiment, the transmitting side operates in the following order: quantization->octree encoding->single recoloring->attribute compression, and the receiving side operates in the following order: octree decoding->attribute decoding.

In the second embodiment, instead of quantization, sampling is performed for the purpose of single recoloring, which means omitting the recoloring process.

According to embodiments, a process for encoding point cloud data according to the first and/or second embodiment may be performed by the point cloud video encoder 10002 of FIG. 1, the encoding step 20001 of FIG. 2, the point cloud video encoder of FIG. 4, the point cloud video encoder of FIG. 12, or a geometry encoder 51003 and an attribute encoder 51004 of FIGS. 19 and 20.

According to embodiments, a process for decoding point cloud data according to the first and/or second embodiment may be performed by the point cloud video decoder 10006 of FIG. 1, the decoding step 20003 of FIG. 2, the point cloud video decoder of FIG. 11, the point cloud video decoder of FIG. 13, or a geometry decoder 61003 and an attribute decoder 61004 of FIGS. 21 and 22. The elements shown in FIGS. 19 to 22 will be described in detail later.

Quantization and sampling have a common point in that the quantization and sampling are to adjust the amount of data (=the number of points) depending on the bitrate in lossy compression of point cloud data. However, in terms of compression efficiency, the quantization and sampling may have up to a 200% difference. The quantization is a process of reconstructing values with a discrete distribution under the assumption that point positions have continuous values, while the sampling is a process of selecting data at regular intervals. Therefore, if the same number of points are output by performing quantization and sampling of a point cloud with the same input values, it is possible to obtain a quantized point cloud and a sampled point cloud having the same number of points but different geometric precision.

Hereinafter, the second embodiment will be described.

The second embodiment uses a sampling method of selecting some of the input points depending on the encoding precision, and the sampled points are used to generate an octree structure. In addition, the attribute values of the sampled points (i.e., original points) are used for attribute compression. In other words, point cloud data is sampled, and attribute encoding/decoding is performed with the geometric precision of the original point cloud data, which is not quantized. The sampled points may be compressed, and differences between the sampled point cloud and the original point cloud may be transmitted as metadata (or referred to as signaling information). In this case, the metadata may be quantized with the sampling rate or quantization per sample value to match the bitrates therebetween. Accordingly, image quality and visual quality may be improved. In the present disclosure, the metadata will be referred to as compression-related information (or information on compression) for convenience of description. Alternatively, the compression-related information may be considered to include the metadata.

For sampling according to the second embodiment, octree levels are divided into 8 parts based on the precision level of the lossy compression. For example, assuming that the precision level (i.e., leaf nodes) used for sampling is N, and the precision level (i.e., encoding precision) used for the lossy compression is M, additional data on levels N-M to N (i.e., depth) may be transmitted as the metadata. Alternatively, the additional data may be included in a geometry bitstream and transmitted after arithmetic coding. According to embodiments, the additional data corresponds to information (e.g., octree_sampling_residual) necessary to perform decoding at levels N-M to N. In other words, since octree compression involves encoding an occupancy pattern, the occupancy of a lower level (i.e., level N) may be transmitted in the form of metadata without being included in the compressed octree.

FIG. 17(a) to FIG. 17(d) are diagrams illustrating exemplary sampling methods according to the second embodiment.

Specifically, FIG. 17(a) shows exemplary original (source) points, FIG. 17(b) shows a sampling method when the sampling scale is 0.5, FIG. 17(c) shows a sampling method when the sampling scale is 0.25, and FIG. 17(d) shows a sampling method when the sampling scale is 0.125.

Assuming that five original points are positioned in a 5×5 space as shown in FIG. 17(a), if the sampling scale is 0.5 as shown in FIG. 17(b), the precision of all points is reduced by half (i.e., ½). For example, in FIG. 17(b), the point of (1, 1) is an original point, and the point of (0, 0) is a sampled point related to the original point. In this case, the sampling arrow displacement is additionally transmitted as metadata. According to embodiments, the sampling arrow displacement may also be included and transmitted in a geometry bitstream. In this document, the sampling arrow displacement may be referred to as sampling displacement. According to embodiments, the sampling arrow displacement represents the difference in position between the original point and the sampled point. In other words, the difference in position between the original point and the sampled point may be transmitted in the form of metadata, or the difference in position may be included and transmitted in a geometry bitstream.

In addition, if the sampling scale is 0.25 as shown in FIG. 17(c), the precision of all points is reduced by a factor of 0.25 (i.e., ¼). For example, in FIG. 17(c), the points of (1, 1) and (2, 1) are original points, and the point of (0, 0) corresponds to a sampled point related to the original points. In this case, the sampling arrow displacement is additionally transmitted as metadata. In other words, the difference in position between each of the original points and the sampled point may be signaled and transmitted in the form of metadata, or the difference in position may also be included and transmitted in a geometry bitstream.

In addition, if the sampling scale is 0.125 as shown in FIG. 17(d), the precision of all points is reduced by a factor of 0.125 (i.e., ⅛). For example, in FIG. 17(d), the points of (1, 1), (2, 1), (1, 3), and (4, 4), i.e., points 50021 to 50024 are original points, and the point of (0, 0) is a sampled point 50010 related to the original points 50021 to 50024. In this case, the sampling arrow displacement is additionally transmitted as metadata. In other words, the difference in position between each of the original points 50021 to 50024 and the sampled point 50010 may be transmitted in the form of metadata, or the difference in position may also be included and transmitted in a geometry bitstream.

FIG. 18 is a diagram illustrating an exemplary octree sampling method and exemplary metadata generation according to embodiments. In FIG. 18, it is assumed that N denotes the precision level for sampling (i.e., leaf nodes) and M denotes the precision level used for lossy compression (i.e., encoding precision). In addition, assuming that the points 50021 to 50024 in FIG. 17(d) correspond to nodes 50021 to 50024 at level N in FIG. 18 and lower nodes of a node 50010 at level M in FIG. 18 are a point group consisting of points 50021 to 50024 at level N, the point 50010 in FIG. 17(d) corresponds to the point 50010 at level M in FIG. 18. In this case, sampling is performed at level N. In other words, sampling is performed on the leaf nodes (i.e., level N), and octree-based occupancy compression extends up to level M. Therefore, according to the present disclosure, the occupancy from level N to level M is transmitted as metadata. In other words, sampling is performed on the leaf nodes (i.e., level N), and octree-based occupancy compression extends up to level M. Information necessary for levels N to M is transmitted as metadata. In this context, the difference in position (referred to as octree_sampling_residual) from each of the points 50021 to 50024 at level N to the point 50010 at level M is signaled as the metadata.

For octree sampling, the same values as those of the original points are used as shown in FIG. 17(a) to FIG. 17(d). Even if the sampling scale changes, points at the same positions are selected, and the number of points decreases. In addition, if encoding is applied such that the sampling shown in FIG. 18 has the encoding precision at octree level M, the displacement of arrows may be transmitted as metadata. In this case, the metadata may be provided in the form of a computed matrix. Moreover, single recoloring may be performed on the octree encoded at level M, and the reconstructed point cloud may be used for attribute compression. According to embodiments, the encoder of the transmitting side and the decoder of the receiving side may calculate the sampling displacement according to Equation 8 below, where a₁, a₂, and a₃ denote transmitted values.

$\begin{array}{cc}\left[\begin{array}{ccc}{r}_{x1}& r_{y1}& r_{z1}\\ r_{x2}& r_{y2}& r_{z2}\\ r_{\mathrm{xN}}& r_{\mathrm{yN}}& r_{\mathrm{zN}}\end{array}\right]\left[\begin{array}{c}{a}_1\\ a_2\\ a_N\end{array}\right]& \left[\mathrm{Equation}8\right]\end{array}$

In Equation 8 above, r denotes a coefficient. When Equation 8 is applied to FIG. 17(d), residuals 5021 to 5010: x5021 to x5010, y5021 to y5010, z5021 to z5010 for one point may be obtained based on a formula for metadata transmission. If the residuals are labeled as ‘residual5021’, the residuals 5021 to 5024 are individually calculated. The coefficient r, which is used to extract the features of the residuals 5021 to 5024, is organized into a matrix and transmitted from the transmitter to the receiver. In this case, if the transmitter transmits the values of a₁, a₂, and a_N (except for feature points), the receiver may reconstruct the residuals 5021 to 5024 by calculating the values of a₁, a₂, and a_N and the matrix of r. In other words, r serves as a coefficient (i.e., feature point) intended to transmit a smaller residual value.

To transmit the difference in position between an original point and a sampled point at the octree level, arithmetic coding may be applied. Specifically, data of 2^N-M×2^N-M×2^N-M (where N and M correspond to levels N and M in FIG. 18) is transmitted as metadata. In addition, additional metadata is required to represent the nodes 50021 to 50024 at level N at the point 50010 at level M in FIG. 18. The metadata has values within a range of 2^N-m×2^N-M×2^N-M. Due to octree occupancy representation, a range of 2^N×2^N×2^N is required to represent the root to level N. However, only levels M to N are required for decoding, and thus the required range is 2^N-M×2^N-M×2^N-M. The size of 2^N-M×2^N-M×2^N-M is the size of metadata before extracting the feature point, while the values of a₁, a₂, and a_N are signaled as the final metadata after the extraction of the feature point r.

Furthermore, if the geometry is reconstructed with sampled points, the attribute value of a single point may be used for attribute compression. That is, the single recoloring may not only improve the accuracy of attribute values but also enhance the efficiency of attribute compression.

According to the second embodiment described above, the transmitting side operates in the following order: sampling->octree encoding->single recoloring->metadata generation->attribute compression, while the receiving side operates in the following order: octree decoding->metadata reconstruction->attribute decoding.

According to embodiments of the present disclosure, it is possible to determine whether to apply quantization or sampling to input point cloud data and signal the determination.

According to embodiments of the present disclosure, it is possible to determine whether to apply recoloring or single recoloring (i.e., skip recoloring) to the reconstructed geometry and signal the determination.

According to embodiments of the present disclosure, it is possible to determine whether to perform attribute encoding based on recolored attribute values or attribute values obtained from single recoloring (i.e., skipping recoloring) and signal the determination.

According to embodiments of the present disclosure, metadata or previously calculated matrices may be signaled to encode accurate points for octree sampling. According to embodiments of the present disclosure, the difference between quantized and sampled values for each octree level is signaled, and a generated bitstream may be entropy encoded. Additionally, the generated bitstream may be combined with the reconstructed geometry to obtain a decoded geometry value, and the decoded geometry value may have attribute values related to the geometry based on the recoloring process. Whether to use the attribute values based on the reconstructed geometry may be determined and signaled depending on the efficiency of attribute compression. The determination may be made after attribute compression based on the distribution of the attribute values.

The compression-related information described above (including metadata) may be included in at least one of an SPS, GPS, APS, TPS, or geometry slice header.

FIG. 19 is a diagram illustrating another exemplary point cloud transmission device according to embodiments. Elements of the point cloud transmission device shown in FIG. 19 may be implemented by hardware, software, processors, and/or any combination thereof.

According to embodiments, a point cloud transmission device may include a data input unit 51001, a signaling processor 51002, a geometry encoder 51003, an attribute encoder 51004, and a transmission processor 51005.

The geometry encoder 51003 and the attribute encoder 51004 may perform some or all of the above-described operations of the point cloud video encoder 10002 of FIG. 1, the encoding step 20001 of FIG. 2, the point cloud video encoder of FIG. 4, and the point cloud video encoder of FIG. 12.

The data input unit 51001 according to embodiments receives or acquires point cloud data. The data input unit 51001 may perform some or all of the operations of the point cloud video acquisition unit 10001 of FIG. 1 or perform some or all of the operations of the data input unit 12000 of FIG. 12.

The data input unit 51001 outputs the positions of points in the point cloud data to the geometry encoder 51003 and outputs the attributes of the points in the point cloud data to the attribute encoder 51004. The data input unit 51001 outputs parameters to the signaling processor 51002. According to embodiments, the parameters may also be provided to the geometry encoder 51003 and the attribute encoder 51004.

The geometry encoder 51003 performs quantization or sampling of the positions of input points, generates an octree structure based on the quantized points or sampled points, and then perform occupancy compression. In other words, geometry information is compressed. The geometry encoder 51003 performs entropy encoding on the compressed geometry information and outputs the entropy encoded geometry information as a geometry bitstream to the transmission processor 51005.

The geometry encoder 51003 reconstructs the geometry information based on positions changed by the compression and outputs the reconstructed (or decoded) geometry information to the attribute encoder 51004.

The attribute encoder 51004 compresses attribute information based on positions to which geometry encoding is not applied and/or the reconstructed geometry information. In one embodiment, the attribute information may be encoded based on at least one of RAHT coding, LOD-based predictive transform coding, or lifting transform coding. Alternatively, the attribute information may be encoded based on any combination thereof. The attribute encoder 51004 performs entropy encoding on the compressed attribute information and outputs the entropy encoded attribute information as an attribute bitstream to the transmission processor 51005.

The signaling processor 51002 may generate and/or process signaling information (e.g., parameters) required for encoding/decoding/rendering of the geometry and attribute information. Then, the signaling processor 5100 may provide the signaling information to the geometry encoder 51003, attribute encoder 51004, and/or transmission processor 51005. Alternatively, the signaling processor 51002 may also receive signaling information generated by the geometry encoder 51003, attribute encoder 51004, and/or transmission processor 51005. The signaling processor 51002 may also receive feedback information from receiving devices (e.g., head orientation information and/or viewport information) and provide the information to the geometry encoder 51003, attribute encoder 51004, and/or transmission processor 51005.

In this specification, signaling information may be signaled and transmitted in units of parameter sets (e.g., SPS, GPS, APS, TPS (or tile inventory), etc.). Alternatively, the signaling information may be transmitted on a coding unit basis such as slices or tiles (or on a compression or prediction unit basis) for each frame.

Methods/devices according to embodiments may signal relevant information to add/perform operations according to embodiments. Signaling information according to embodiments may be used by transmitting and/or receiving devices.

The transmission processor 51005 may perform operations and/or transmission methods similar or equal to the operations and/or transmission methods of the transmission processor 12012 in FIG. 12. Alternatively, the transmission processor 51005 may perform operations and/or transmission methods similar or equal to the operations and/or transmission methods of the transmitter 10003 in FIG. 1. Since the specific details are described above with reference to FIG. 1 and FIG. 12, the details will be omitted in the following.

The transmission processor 51005 multiplexes a geometry bitstream output from the geometry encoder 51003, an attribute bitstream output from the attribute encoder 51004, and a signaling bitstream output from the signaling processor 51002 into a single bitstream. Then, the transmission processor 51005 transmits the single bitstream as it is. Alternatively, the transmission processor 51005 encapsulates the single bitstream as a file, segment, and so on for transmission. According to an embodiment of the present disclosure, the term “file” refers to an ISO base media file format (ISOBMFF) file format.

According to embodiments, files or segments may be transmitted to receiving devices or stored on digital storage media (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). According to embodiments, the transmission processor 51005 may communicate with receiving devices via wired or wireless networks such as 4G, 5G, and 6G. The transmission processor 51005 may perform necessary data processing operations depending on network systems (e.g., communication network systems such as 4G, 5G, and 6G). The transmission processor 51005 may also transmit encapsulated data according to the on-demand approach.

According to embodiments, the compression-related information (or information related to compression) may be included not only in the SPS, GPS, APS, and/or TPS but also in geometry data units (referred to as geometry slice bitstreams). The compression-related information may be transmitted by at least one of the signaling processor 51002, geometry encoder 51003, or transmission processor 51005.

FIG. 20 is a detailed block diagram illustrating the geometry encoder 51003 and attribute encoder 51004 according to embodiments. The elements of the geometry encoder illustrated in FIG. 20 may be implemented by hardware, software, processors, and/or combinations thereof.

According to embodiments, the geometry encoder 51003 may include a coordinate transformer 53001, a quantization/sampling determiner and processor 53002, a metadata generator and quantizer 53003, an octree occupancy code generator 53004, a surface model processor 53005, an arithmetic coder 53006, and a geometry reconstructor 53007.

According to embodiments, the attribute encoder 51004 may include a color transform processor 55001, an attribute transform processor 55002, a prediction/lifting/RAHT transform processor 55003, a coefficient quantization processor 55004, and an arithmetic coder 55005.

The coordinate transformer 53001 may support transformation of the coordinate system of point cloud data, such as changing the xyz axes of input points or converting from xyz Cartesian coordinates to spherical coordinates.

The quantization/sampling determiner and processor 53002 may determine whether to perform quantization or sampling of the positions of input points.

In one embodiment, if the quantization/sampling determiner and processor 53002 selects quantization, the metadata generator and quantizer 53003 adjusts the geometry precision to level N-M to perform lossy compression at the maximum depth level of N. The metadata generator and quantizer 53003 quantizes points from level N-M to level N based on the quantization scale and positions modified leaf nodes at level N-M. In other words, the metadata generator and quantizer 53003 divides level N-M by precision, quantizes all points from level N-M to level N, and positions modified leaf nodes at level N-M.

In another embodiment, if the quantization/sampling determiner and processor 53002 selects sampling, the metadata generator and quantizer 53003 performs sampling at the maximum depth level of N based on the sampling scale. In other words, points at level N are selected based on the sampling scale. Additionally, assuming that the precision level (i.e., encoding precision) used for lossy compression is M, additional data on levels from N-M to N is transmitted as metadata. According to embodiments, the additional data includes information (e.g., octree_sampling_residual) necessary for decoding at levels N-M to N. In other words, the additional data may represent the differences in position between original points and sampled points. That is, sampling is performed at leaf nodes, and octree-based occupancy compression is performed up to level M. The information necessary for levels N to M is included and transmitted as metadata in at least one of an SPS, GPS, TPS, APS, or geometry slice header. Alternatively, the necessary information is included and transmitted in a geometry bitstream.

As described above, the position of each quantized point correspond to a position to which one or more original points are mapped (i.e., a position where there is no point in the original point cloud), while the position of each sampled point is the position of one of the original points. In addition, when quantization is performed, information on a relationship between points at level N and points at level M is not signaled. On the other hand, when sampling is performed, information on a relationship between points at level N and points at level M is signaled in the form of metadata. That is, for octree compression, encoding is performed with an occupancy pattern. Thus, when sampling is performed, the occupancy of a lower level (e.g., level N) is transmitted as metadata, instead of being included in the compressed octree.

When quantization is performed by the metadata generator and quantizer 53003, the octree occupancy code generator 53004 creates an octree structure based on the quantized points. When metadata is generated by the metadata generator and quantizer 53003, the octree occupancy code generator 53004 creates an octree structure based on the sampled points. The leaf nodes of the generated octree structure (e.g., nodes at level M in FIG. 18 are represented by occupancy codes. The octree occupancy code generator 53004 may perform operations and/or methods similar or equal to the operations and/or methods of the point cloud video encoder described in FIG. 4 or FIG. 12 (or perform operations and/or methods similar or equal to the operations and/or methods of the octree analyzer 40002 or octree occupancy code generator 12003).

The surface model processor 53005 may perform trisoup geometry encoding that reconstructs the positions of points within a specific region (or node) based on a surface model using voxels. The surface model processor 53005 may perform operations and/or methods similar or equal to the operations and/or methods of the point cloud video encoder described in FIG. 4 or FIG. 12 (or perform operations and/or methods similar or equal to the operations and/or methods of the surface approximation analyzer 40003 or surface model processor 12004).

The arithmetic coder 53006 performs arithmetic coding (e.g., entropy coding) on the outputs of the octree occupancy code generator 53004 and/or surface model processor 53005 and then outputs the coding results in the form of a geometry bitstream. For example, the arithmetic coder 53006 may perform entropy coding on the occupancy codes outputted by the octree occupancy code generator 53004. Moreover, to enhance compression efficiency, the point cloud video encoder may perform intra/inter coding on the occupancy codes. According to embodiments, the receiving device (or point cloud video decoder) reconstructs the octree based on the occupancy codes.

The geometry reconstructor 53007 reconstructs geometry information based on the occupancy codes from the octree occupancy code generator 53004. For example, if the octree structure is generated based on quantized points, the output of the geometry reconstructor 53007 becomes the positions of the quantized points. If the octree structure is generated based on sampled points, the output of the geometry reconstructor 53007 becomes the positions of the sampled points.

The output of the geometry reconstructor 53007 is provided to the attribute transform processor 55002 and/or prediction/lifting/RAHT transform processor 55003 of the attribute encoder 51004.

According to embodiments, the color transform processor 55001 of the attribute encoder 51004 performs color transform coding on color values included in the attributes of points output from the data input unit 51001. The color transform processor 55001 may perform color transform coding based on the reconstructed geometry. For example, the color transform processor 55001 may convert the format of color information (e.g., from RGB to YCbCr). The color transform processor 55001 may perform operations and/or methods similar or equal to the operations and/or methods of the point cloud video encoder described in FIG. 4 or FIG. 12 (e.g., operations and/or methods similar or equal to the operations and/or methods of the color transformer 40006 or color transform processor 12008).

The attribute transform processor 55002 performs recoloring or single recoloring (i.e., skipping recoloring) based on positions to which geometry encoding is not applied and/or the reconstructed geometry.

In one embodiment, if geometry encoding is performed by the geometry encoder 51003, the attribute transform processor 55002 executes the recoloring or single recoloring. According to embodiments, the recoloring is performed based on Equation 6, and the single recoloring is performed based on Equation 7. In other words, the recoloring involves exploring multiple neighboring points of a point input from the geometry reconstructor 53007 based on K-D tree or Morton code, setting the average of the attribute values of the explored neighboring points to the attributes of the input point, and outputting the corresponding value to the prediction/lifting/RAHT transform processor 55003. The single recoloring involves setting the attributes of a representative point among the explored neighboring points to the attributes of the input point and outputting the corresponding value to the prediction/lifting/RAHT transform processor 55003.

In another embodiment, if sampling is performed by the geometry encoder 51003, the attribute transform processor 55002 performs the single recoloring. In other words, since the points input from the geometry reconstructor 53007 are original points, the attributes of the input points are directly output to the prediction/lifting/RAHT transform processor 55003.

The prediction/lifting/RAHT transform processor 55003 may encode the attributes output from the attribute transform processor 55002 based on at least one of RAHT coding, LOD-based predictive transform coding, lifting transform coding, or any combination thereof. The prediction/lifting/RAHT transform processor 55003 performs at least one of operations similar or equal to those of the RAHT transform transformer 40008, LOD generator 40009, and lifting transformer 40010 described in FIG. 4 or those of the prediction/lifting/RAHT transform processor 12010 described in FIG. 12. Since the details of LOD-based predictive transform coding, lifting transform coding, and RAHT coding are described above with reference to FIGS. 1 to 9, the details will be omitted in the following.

The attributes attribute-coded from the prediction/lifting/RAHT transform processor 55003 may be quantized by the coefficient quantization processor 55004 based on coefficients.

The arithmetic coder 55005 encodes the quantized attributes based on arithmetic coding and outputs the attributes in the form of an attribute bitstream.

The geometry bitstream output from the arithmetic coder 53006 of the geometry encoder 51003 and the attribute bitstream output from the arithmetic coder 55005 of the attribute encoder 51004 are input into the transmission processor 51005.

According to embodiments, the transmission processor 51005 may perform operations and/or transmission methods similar or equal to the operations and/or transmission methods of the transmission processor 12012 in FIG. 12 or perform operations and/or transmission methods similar or equal to the operations and/or transmission methods of the transmitter 10003 in FIG. 1. Since the specific details are described above with reference to FIG. 1 and FIG. 12, the details will be omitted in the following.

According to embodiments, the transmission processor 51005 may transmit the geometry bitstream output from the geometry encoder 51003, the attribute bitstream output from the attribute encoder 51004, and the signaling bitstream output from the signaling processor 51002 separately. Alternatively, the transmission processor 51005 may also multiplex and transmit these streams as a single bitstream.

According to embodiments, the transmission processor 51005 may encapsulate the bitstreams into a file or segment (e.g., streaming segment) and transmit the file or segment through various networks, such as broadcasting networks and/or broadband networks.

According to embodiments, the signaling processor 51002 may generate and/or process signaling information and output the signaling information in the form of a bitstream to the transmission processor 51005. The signaling information generated and/or processed by the signaling processor 51002 may be provided to the geometry encoder 51003, attribute encoder 51004, and/or transmission processor 51005 for the purposes of geometry encoding, attribute encoding, and transmission processing, respectively. Alternatively, the signaling processor 51002 may receive signaling information generated by the geometry encoder 51003, attribute encoder 51004, and/or transmission processor 51005.

In this specification, signaling information may be signaled and transmitted in units of parameter sets (e.g., SPS, GPS, APS, TPS (or tile inventory), etc.). Alternatively, the signaling information may be transmitted on a coding unit basis such as slices or tiles for each frame. In addition, the signaling information may include compression-related information (or information related to compression) which includes metadata. Depending on the application, the signaling information may also be defined at the system level such as file formats, Dynamic Adaptive Streaming over HTTP (DASH), MPEG Media Transport (MMT), etc. or at the hardware (wired) interface level such as (High Definition Multimedia Interface (HDMI), Display Port, Video Electronics Standards Association (VESA), CTA (Consumer Technology Association), etc.

Methods/devices according to embodiments may signal relevant information to add/perform operations according to embodiments. Signaling information according to embodiments may be used by transmitting and/or receiving devices.

FIG. 21 is a diagram showing another exemplary point cloud reception device according to embodiments.

The point cloud reception device according to the embodiments may include a reception processor 61001, a signaling processor 61002, a geometry decoder 61003, an attribute decoder 61004, and a post-processor 61005. According to embodiments, the geometry decoder 61003 and the attribute decoder 61004 may be referred to as a point cloud video decoder. According to embodiments, the point cloud video decoder may be referred to as a PCC decoder, a PCC decoding unit, a point cloud decoder, a point cloud decoding unit, or the like.

The reception processor 61001 according to the embodiments may receive a single bitstream, or may receive a geometry bitstream, an attribute bitstream, and a signaling bitstream, respectively. When a file and/or segment is received, the reception processor 61001 according to the embodiments may decapsulate the received file and/or segment and output the decapsulated file and/or segment as a bitstream.

When the single bitstream is received (or decapsulated), the reception processor 61001 according to the embodiments may demultiplex the geometry bitstream, the attribute bitstream, and/or the signaling bitstream from the single bitstream. The reception processor 61001 may output the demultiplexed signaling bitstream to the signaling processor 61002, the geometry bitstream to the geometry decoder 61003, and the attribute bitstream to the attribute decoder 61004.

When the geometry bitstream, the attribute bitstream, and/or the signaling bitstream are received (or decapsulated), respectively, the reception processor 61001 according to the embodiments may deliver the signaling bitstream to the signaling processor 61002, the geometry bitstream to the geometry decoder 61003, and the attribute bitstream to the attribute decoder 61004.

The signaling processor 61002 may parse signaling information, for example, information contained in the SPS, GPS, APS, TPS, metadata, or the like from the input signaling bitstream, process the parsed information, and provide the processed information to the geometry decoder 61003, the attribute decoder 61004, and the post-processor 61005. In another embodiment, signaling information contained in the geometry slice header and/or the attribute slice header may also be parsed by the signaling processor 61002 before decoding of the corresponding slice data. That is, when the point cloud data is partitioned into tiles and/or slices at the transmitting side, the TPS includes the number of slices included in each tile, and accordingly the point cloud video decoder according to the embodiments may check the number of slices and quickly parse the information for parallel decoding.

Accordingly, the point cloud video decoder according to the present disclosure may quickly parse a bitstream containing point cloud data as it receives an SPS having a reduced amount of data. The reception device may decode tiles upon receiving the tiles, and may decode each slice based on the GPS and APS included in each tile. Thereby, decoding efficiency may be maximized.

That is, the geometry decoder 61003 may reconstruct the geometry by performing the reverse process of the operation of the geometry encoder 51003 of FIG. 19 on the compressed geometry bitstream based on signaling information (e.g., geometry related parameters). The geometry restored (or reconstructed) by the geometry decoder 61003 is provided to the attribute decoder 61004. The attribute decoder 61004 may restore the attribute by performing the reverse process of the operation of the attribute encoder 51004 of FIG. 19 on the compressed attribute bitstream based on signaling information (e.g., attribute related parameters) and the reconstructed geometry. According to embodiments, when the point cloud data is partitioned into tiles and/or slices at the transmitting side, the geometry decoder 61003 and the attribute decoder 61004 perform geometry decoding and attribute decoding on a tile-by-tile basis and/or slice-by-slice basis.

FIG. 22 is a detailed block diagram illustrating the geometry decoder 61003 and the attribute decoder 61004 according to embodiments.

In FIG. 22, the geometry decoder 61003 may include an arithmetic decoder 63001, an occupancy code-based octree reconstruction processor 63002, a surface model processor 63003, a metadata reconstructor 63004, a geometry reconstructor 63005, and a coordinate inverse transformer 63006. The arithmetic decoder 63001, occupancy code-based octree reconstruction processor 63002, surface model processor 63003, geometry reconstructor 63005, and coordinate inverse transformer 63006 in FIG. 22 may perform some or all of the operations of the arithmetic decoder 11000, octree synthesizer 11001, surface approximation synthesizer 11002, geometry reconstructor 11003, and coordinate inverse transformer 11004 in FIG. 11. Alternatively, the arithmetic decoder 63001, occupancy code-based octree reconstruction processor 63002, surface model processor 63003, geometry reconstructor 63005, and coordinate inverse transformer 63006 in FIG. 22 may perform some or all of the operations of the arithmetic decoder 13002, occupancy code-based octree reconstruction processor 13003, surface model processor, and inverse quantization processor 13005 in FIG. 13.

That is, the arithmetic decoder 63001 may perform arithmetic decoding on an input geometry bitstream.

The occupancy code-based octree reconstruction processor 63002 may reconstruct an octree by obtaining occupancy codes from the arithmetic-decoded geometry bitstream (or compression-related information obtained from decoding).

When trisoup geometry encoding is applied, the surface model processor 63003 may perform trisoup geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, up-sampling, and voxelization) based on the surface model method.

When receiving metadata that is generated by the geometry encoder 51003 of the transmitting side through sampling and included in a geometry bitstream after arithmetic coding, the metadata reconstructor 63004 may reconstruct the metadata. Alternatively, the metadata reconstructor 63004 may reconstruct the metadata based on compression-related information included in signaling information.

The geometry reconstructor 63005 may regenerate a geometry based on the processed surface model and/or reconstructed metadata.

The geometry reconstructed by the geometry reconstructor 63005 is output to the coordinate inverse transformer 63006 and a prediction/lifting/RAHT transform processor 65003 of the attribute decoder 61004.

The coordinate inverse transformer 63006 may transform coordinates based on the reconstructed geometry and then obtain the positions (i.e., position values) of points.

The positions obtained from the coordinate inverse transformer 63006 of the geometry decoder 61003 are output to the post-processor 61005.

According to embodiments, if compression-related information (or metadata) is transmitted in at least one of an SPS, GPS, APS, TPS, or geometry slice header, the signaling processor 61002 may obtain the compression-related information and provide the compression-related information to the geometry decoder 61003. Alternatively, the geometry decoder 61003 may directly obtain the compression-related information.

According to embodiments, the attribute decoder 61004 may include an arithmetic decoder 65001, an inverse quantization processor 65002, a prediction/lifting/RAHT transform processor 65003, an attribute reconstructor 65004, and a color inverse transform processor 65005.

According to embodiments, the arithmetic decoder 65001 may perform arithmetic decoding on an input attribute bitstream. The arithmetic decoder 65001 performs operations and/or decoding similar or equal to the operations and/or decoding of the arithmetic decoder 11005 in FIG. 11 or the arithmetic decoder 13007 in FIG. 13.

The inverse quantization processor 65002 inversely quantizes the arithmetic-decoded attribute bitstream and outputs the inverse quantized attributes (or attribute values). Inverse quantization may be optionally applied based on the attribute encoding of the point cloud video encoder.

The prediction/lifting/RAHT transform processor 65003 may decode the attributes output from the inverse quantization processor 65002 using at least one of the RAHT decoding, LOD-based predictive transform decoding, and lifting transform decoding or by combining two or more of the RAHT decoding, LOD-based predictive transform decoding, and lifting transform decoding based on the reconstructed geometry.

The attribute reconstructor 65004 reconstructs the attributes decoded by the prediction/lifting/RAHT transform processor 65003.

The color inverse transform processor 65005 performs inverse transform coding to inversely transform color values (or textures) included in the reconstructed attributes and outputs the results to the post-processor 61005. The color inverse transform processor 65005 performs operations and/or inverse transform coding that are the same as or similar to the operations and/or inverse transform coding of the color inverse transformer 11010 in FIG. 11 or the color inverse transform processor 13010 in FIG. 13.

The post-processor 61005 may reconstruct the point cloud data by matching the positions restored and output from the geometry decoder 61003 with the attributes restored and output from the attribute decoder 61004. The reconstructed point cloud data may then be rendered through a display. In addition, if the reconstructed point cloud data is on a tile and/or slice basis, the post-processor 61005 may perform an inverse process for spatial partitioning of the transmitting side based on signaling information.

FIG. 23 illustrates an exemplary bitstream structure of point cloud data for transmission and reception according to embodiments.

To add or implement the above-described embodiments, relevant information may be signaled. According to embodiments, signaling information may be used by the point cloud video encoder of the transmitting side or the point cloud video decoder of the receiving side.

According to embodiments, the point cloud video encoder may encode geometry information and attribute information as described above to generate a bitstream as shown in FIG. 23. Additionally, signaling information on the point cloud data may be generated and processed by at least one of a geometry encoder, an attribute encoder, and a signaling processor in the point cloud video encoder and then be included in the bitstream.

According to embodiments, the signaling information may be received/acquired by at least one of a geometry decoder, an attribute decoder, and a signaling processor in the point cloud video decoder.

According to embodiments, the bitstream may include a geometry bitstream, an attribute bitstream, and a signaling bitstream, which may be transmitted or received separately. Alternatively, these bitstreams may be combined into a single bitstream and then transmitted/received.

When a geometry bitstream, an attribute bitstream, and a signaling bitstream according to embodiments are configured as one bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to the embodiments may include a sequence parameter set (SPS) for sequence level signaling, a geometry parameter set (GPS) for signaling of geometry information coding, one or more attribute parameter sets (APSs) (APS₀, APS₁) for signaling of attribute information coding, a tile parameter set (TPS) for tile level signaling, and one or more slices (slice 0 to slice n). That is, a bitstream of point cloud data according to embodiments may include one or more tiles, and each of the tiles may be a group of slices including one or more slices (slice 0 to slice n). The TPS according to the embodiments may contain information about each of the one or more tiles (e.g., coordinate value information and height/size information about the bounding box). Each slice may include one geometry bitstream (Geom0) and one or more attribute bitstreams (Attr0 and Attr1). For example, a first slice (slice 0) may include one geometry bitstream (Geom0⁰) and one or more attribute bitstreams (Attr0⁰, Attr1⁰).

A geometry bitstream in each slice (referred to as a geometry slice) may be composed of a geometry slice header (geom_slice_header) and geometry slice data (geom_slice_data). According to embodiments, the geometry bitstream in each slice is referred to as a geometry data unit. The geometry slice header is referred to as a geometry data unit header, and the geometry slice data is referred to as geometry data unit data.

An attribute bitstream in each slice (referred to as an attribute slice) may be composed of an attribute slice header (attr_slice_header) and attribute slice data (attr_slice_data). According to embodiments, the attribute bitstream in each slice is referred to as an attribute data unit. The attribute slice header is referred to as an attribute data unit header, and the attribute slice data is referred to as attribute data unit data.

By transmitting the point cloud data according to the bitstream structure as shown in FIG. 23, the transmission device according to the embodiments may allow the encoding operation to be applied differently according to the importance level, and allow a good-quality encoding method to be used in an important region. In addition, it may support efficient encoding and transmission according to the characteristics of the point cloud data and provide attribute values according to user requirements.

As the reception device according to the embodiments receives the point cloud data according to the bitstream structure as shown in FIG. 23, it may apply different filtering (decoding methods) to the respective regions (divided into tiles or slices) according to the processing capacity of the reception device, rather than using a complex decoding (filtering) method to the entire point cloud data. Thereby, a better image quality may be provided for regions important to the user and appropriate latency may be ensured in the system.

As described above, tiles or slices are provided to divide the point cloud data into the regions for processing. When the point cloud data is divided into the regions, options may be configured to generate different neighboring point sets for each region. Thus, it is allowed to select a method with low complexity but low reliability or a method with high complexity but high reliability.

According to embodiments, at least one of an SPS, GPS, TPS, APS, or geometry slice header may include compression-related information (or information related to compression).

Specifically, signaling (e.g., compression-related information) may have different meanings depending on positions where the signaling is transmitted. If the signaling is defined in the SPS, the signaling may be uniformly applied to the entire sequence. If the signaling is defined in the GPS, it may indicate that the signaling is used for position reconstruction. If the signaling is defined in the APS, it may indicate that the signaling is used for attribute reconstruction. If the signaling is defined in the TPS, it may indicate that the signaling is applied to points within tiles. If the signaling is transmitted at the slice level, it may indicate that the signaling is applied only to slices. In addition, if fields (referred to as syntax elements) defined in the following are capable of being applied not only to a current point cloud data stream but also to multiple point cloud data streams, the signaling may be provided through a higher level concept of parameter set.

According to embodiments, whether recoloring for attribute compression is performed may be signaled in compression-related information, depending on the reconstructed geometry. When no recoloring is performed, that is, when single recoloring is performed, since there is only one attribute value per point, attribute values that exactly match the original point cloud may be compressed. Accordingly, the accuracy of attribute compression, which is affected by geometry reconstruction, may be improved. According to embodiments, whether single recoloring is performed may be signaled in compression-related information, depending on Moltan code generation, distribution of geometry attribute values, similarity of attribute values of neighboring nodes, and distribution of DC coefficients.

The term “field” used for the syntaxes, which will be described in this document, may have the same meaning as the term “parameter” or “element”.

FIG. 24 shows an embodiment of a syntax structure of a sequence parameter set (SPS) (seq_parameter_set_rbsp( )) according to the present disclosure. The SPS may include sequence information about a point cloud data bitstream. In particular, in this example, the SPS includes neighbor point selection related option information.

The SPS according to the embodiments may include a profile_idc field, a profile_compatibility_flags field, a level_idc field, an sps_bounding_box_present_flag field, an sps_source_scale_factor field, an sps_seq_parameter_set_id field, an sps_num_attribute_sets field, and an sps_extension_present_flag field.

The profile_idc field indicates a profile to which the bitstream conforms.

The profile_compatibility_flags field equal to 1 may indicate that the bitstream conforms to the profile indicated by profile_idc.

The level_idc field indicates a level to which the bitstream conforms.

The sps_bounding_box_present_flag field indicates whether source bounding box information is signaled in the SPS. The source bounding box information may include offset and size information about the source bounding box. For example, the sps_bounding_box_present_flag field equal to 1 indicates that the source bounding box information is signaled in the SPS. The sps_bounding_box_present_flag field equal to 0 indicates the source bounding box information is not signaled. The sps_source_scale_factor field indicates the scale factor of the source point cloud.

The sps_seq_parameter_set_id field provides an identifier for the SPS for reference by other syntax elements.

The sps_num_attribute_sets field indicates the number of coded attributes in the bitstream.

The sps_extension_present_flag field specifies whether the sps_extension_data syntax structure is present in the SPS syntax structure. For example, the sps_extension_present_flag field equal to 1 specifies that the sps_extension_data syntax structure is present in the SPS syntax structure. The sps_extension_present_flag field equal to 0 specifies that this syntax structure is not present. When not present, the value of the sps_extension_present_flag field is inferred to be equal to 0.

When the sps_bounding_box_present_flag field is equal to 1, the SPS according to embodiments may further include an sps_bounding_box_offset_x field, an sps_bounding_box_offset_y field, an sps_bounding_box_offset_z field, an sps_bounding_box_scale_factor field, an sps_bounding_box_size_width field, an sps_bounding_box_size_height field, and an sps_bounding_box_size_depth field.

The sps_bounding_box_offset_x field indicates the x offset of the source bounding box in the Cartesian coordinates. When the x offset of the source bounding box is not present, the value of sps_bounding_box_offset_x is 0.

The sps_bounding_box_offset_y field indicates the y offset of the source bounding box in the Cartesian coordinates. When the y offset of the source bounding box is not present, the value of sps_bounding_box_offset_y is 0.

The sps_bounding_box_offset_z field indicates the z offset of the source bounding box in the Cartesian coordinates. When the z offset of the source bounding box is not present, the value of sps_bounding_box_offset_z is 0.

The sps_bounding_box_scale_factor field indicates the scale factor of the source bounding box in the Cartesian coordinates. When the scale factor of the source bounding box is not present, the value of sps_bounding_box_scale_factor may be 1.

The sps_bounding_box_size_width field indicates the width of the source bounding box in the Cartesian coordinates. When the width of the source bounding box is not present, the value of the sps_bounding_box_size_width field may be 1.

The sps_bounding_box_size_height field indicates the height of the source bounding box in the Cartesian coordinates. When the height of the source bounding box is not present, the value of the sps_bounding_box_size_height field may be 1.

The sps_bounding_box_size_depth field indicates the depth of the source bounding box in the Cartesian coordinates. When the depth of the source bounding box is not present, the value of the sps_bounding_box_size_depth field may be 1.

The SPS according to embodiments includes an iteration statement repeated as many times as the value of the sps_num_attribute_sets field. In an embodiment, i is initialized to 0, and is incremented by 1 each time the iteration statement is executed. The iteration statement is repeated until the value of i becomes equal to the value of the sps_num_attribute_sets field. The iteration statement may include an attribute_dimension[i] field, an attribute_instance_id[i] field, an attribute_bitdepth[i] field, an attribute_cicp_colour_primaries[i] field, an attribute_cicp_transfer_characteristics[i] field, an attribute_cicp_matrix_coeffs[i] field, an attribute_cicp_video_full_range_flag[i] field, and a known_attribute_label_flag[i] field.

The attribute_dimension[i] field specifies the number of components of the i-th attribute.

The attribute_instance_id[i] field specifies the instance ID of the i-th attribute.

The attribute_bitdepth[i] field specifies the bitdepth of the i-th attribute signal(s).

The attribute_cicp_colour_primaries[i] field indicates chromaticity coordinates of the color attribute source primaries of the i-th attribute.

The attribute_cicp_transfer_characteristics[i] field either indicates the reference optoelectronic transfer characteristic function of the colour attribute as a function of a source input linear optical intensity with a nominal real-valued range of 0 to 1 or indicates the inverse of the reference electro-optical transfer characteristic function as a function of an output linear optical intensity.

The attribute_cicp_matrix_coeffs[i] field describes the matrix coefficients used in deriving luma and chroma signals from the green, blue, and red, or Y, Z, and X primaries.

The attribute_cicp_video_full_range_flag[i] field indicates the black level and range of the luma and chroma signals as derived from E′Y, E′PB, and E′PR or E′R, E′G, and E′B real-valued component signals.

The known_attribute_label_flag[i] field specifies whether a known attribute label field or an attribute label four bytes field is signaled for the i-th attribute. For example, the value of the known_attribute_label_flag[i] field equal to 0 specifies that the known attribute label field is signaled for the ith attribute. The known_attribute_label_flag[i] field equal to 1 specifies that the attribute label four bytes field is signaled for the ith attribute.

The known_attribute_label[i] field may specify an attribute type. For example, the known_attribute_label[i] field equal to 0 may specify that the i-th attribute is color. The known_attribute_label[i] field equal to 1 specifies that the i-th attribute is reflectance. The known_attribute_label[i] field equal to 2 may specify that the i-th attribute is frame index.

The attribute_label_four_bytes field indicates the known attribute type with a 4-byte code.

In this example, the attribute_label_four_bytes field indicates color when equal to 0 and indicates reflectance when is equal to 1.

According to embodiments, when the sps_extension_present_flag field is equal to 1, the SPS may further include a sps_extension_data_flag field.

The sps_extension_data_flag field may have any value.

FIG. 25 is a diagram illustrating an exemplary syntax structure of a sequence parameter set (SPS) (seq_parameter_set_rbsp( )) including compression-related information according to embodiments.

Referring to 25, the SPS may include a recoloring_skip_flag field, an octree_sampling_location field, a sampling_point_num field, and a reconstructed_geometry_use_flag field.

The recoloring_skip_flag field may indicate whether recoloring is skipped or not. For example, when the value of the recoloring_skip_flag field is true, it may indicate that single recoloring (i.e., skipping recoloring) is performed. When the value of the recoloring_skip_flag field is false, it may indicate that recoloring is performed.

The octree_sampling_location field may indicate the position of points after quantization when quantized values are used for octree coding. Specifically, the octree_sampling_location field may indicate that positions within the cubic region shown in FIG. 16, such as positions of (0, 0, 0) to (1, 1, 1) or a central position of (0.5, 0.5, 0.5), are used for octree quantization. For example, when four bits are allocated to the octree_sampling_location field, if the value of the octree_sampling_location field is 0000, it may indicate that the position of (0, 0, 0) is used for the quantization. If the value of the octree_sampling_location field is 0001, it may indicate that the position of (0, 0, 1) is used for the quantization. If the value of the octree_sampling_location field is 0111, it may indicate that the position of (1, 1, 1) is used for the quantization. If the value of the octree_sampling_location field is 1000, it may indicate that the position of (0.5, 0.5, 0.5) is used for the quantization.

The sampling_point_num field indicates the number of sampled points. For example, referring to FIG. 18, the sampling_point_num field indicates the number of sampled points at level N, which corresponds to the total number of points at level M.

The reconstructed_geometry_use_flag field indicates whether geometry values reconstructed through single recoloring are used for attribute information. For example, if the value of the reconstructed_geometry_use_flag field is true, it may indicate the geometry values reconstructed through single recoloring are used for the attribute information. If the value of the reconstructed_geometry_use_flag field is false, it may indicate that geometry values reconstructed through conventional recoloring are used for the attribute information.

According to embodiments, if the value of the reconstructed_geometry_use_flag field is true, the SPS may include metadata_data_unit( ).

According to embodiments, the SPS may include a loop that iterates as many times as the value of the sampling_point_num field, instead of including metadata_data_unit( ). In one embodiment, the value of i is initialized to 0 and incremented by 1 with each iteration of the loop, and the loop continues until the value of i reaches the value of the sampling_point_num field. The loop may include an octree_sampling_residual [i][3] field and be positioned after the sampling_point_num field.

The octree_sampling_residual [i][3] field represents the difference in position between an i-th sampled point and an original point. The positional difference may be expressed in the form of xyz. For example, the values of x-x′, y-y′, and z-z′ may be represented for each sampled point. Here, xyz represents the position values of the original point, and x′y′z′ represents the position values of the sampled point.

FIG. 26 is a diagram illustrating an exemplary syntax structure of metadata_data_unit( ) according to embodiments.

Referring to FIG. 26, metadata_data_unit( ) may include a loop that iterates as many times as the value of the sampling_point_num field. In one embodiment, the value of i is initialized to 0 and incremented by 1 with each iteration of the loop, and the loop continues until the value of i reaches the value of the sampling_point_num field. The loop may include the octree_sampling_residual [i][3] field.

The octree_sampling_residual [i][3] field represents the difference in position between an i-th sampled point and an original point. The positional difference may be expressed in the form of xyz. For example, the values of x-x′, y-y′, and z-z′ may be represented for each sampled point. Here, xyz represents the position values of the original point, and x′y′z′ represents the position values of the sampled point.

According to embodiments, the sampling_point_num field may be included in metadata_data_unit( ).

According to embodiments, metadata_data_unit( ) may exist in each parameter set or as a standalone data unit. Additionally, the differences in position between a sampled octree and an original point cloud may be transmitted as metadata, or values substituted by calculation according to embodiments may be transmitted in the form of a bitstream using arithmetic coding.

According to embodiments, metadata_data_unit( ) is also referred to as sampling-related information. The sampling-related information may be included and transmitted in at least one of an SPS, GPS, APS, TPS, or geometry slice header. Alternatively, the sampling-related information may be included and transmitted in a geometry bitstream.

According to embodiments, the compression-related information of FIG. 25 may be included at arbitrary positions in the SPS of FIG. 24.

FIG. 27 shows an embodiment of a syntax structure of the geometry parameter set (GPS) (geometry_parameter_set( )) according to the present disclosure.

According to embodiments, the GPS may include a gps_geom_parameter_set_id field, a gps_seq_parameter_set_id field, a gps_box_present_flag field, a unique_geometry_points_flag field, a neighbour_context_restriction_flag field, an inferred_direct_coding_mode_enabled_flag field, a bitwise_occupancy_coding_flag field, an adjacent_child_contextualization_enabled_flag field, a log 2_neighbour_avail_boundary field, a log 2_intra_pred_max_node_size field, a log 2_trisoup_node_size field, and a gps_extension_present_flag field.

The gps_geom_parameter_set_id field provides an identifier for the GPS for reference by other syntax elements.

The gps_seq_parameter_set_id field specifies the value of sps_seq_parameter_set_id for the active SPS.

The gps_box_present_flag field specifies whether additional bounding box information is provided in a geometry slice header that references the current GPS. For example, the gps_box_present_flag field equal to 1 may specify that additional bounding box information is provided in a geometry header that references the current GPS. Accordingly, when the gps_box_present_flag field is equal to 1, the GPS may further include a gps_gsh_box_log 2_scale_present_flag field.

The gps_gsh_box_log 2_scale_present_flag field specifies whether the gps_gsh_box_log 2_scale field is signaled in each geometry slice header that references the current GPS. For example, the gps_gsh_box_log 2_scale_present_flag field equal to 1 may specify that the gps_gsh_box_log 2_scale field is signaled in each geometry slice header that references the current GPS. As another example, the gps_gsh_box_log 2_scale_present_flag field equal to 0 may specify that the gps_gsh_box_log 2_scale field is not signaled in each geometry slice header and a common scale for all slices is signaled in the gps_gsh_box_log 2_scale field of the current GPS.

When the gps_gsh_box_log 2_scale_present_flag field is equal to 0, the GPS may further include a gps_gsh_box_log 2_scale field.

The gps_gsh_box_log 2_scale field indicates the common scale factor of the bounding box origin for all slices that refer to the current GPS.

The unique_geometry_points_flag field indicates whether all output points have unique positions. For example, the unique_geometry_points_flag field equal to 1 indicates that all output points have unique positions. The unique_geometry_points_flag field equal to 0 indicates that in all slices that refer to the current GPS, the two or more of the output points may have the same position.

The neighbor_context_restriction_flag field indicates contexts used for octree occupancy coding. For example, the neighbour_context_restriction_flag field equal to 0 indicates that octree occupancy coding uses contexts determined from six neighboring parent nodes. The neighbour_context_restriction_flag field equal to 1 indicates that octree occupancy coding uses contexts determined from sibling nodes only.

The inferred_direct_coding_mode_enabled_flag field indicates whether the direct_mode_flag field is present in the geometry node syntax. For example, the inferred_direct_coding_mode_enabled_flag field equal to 1 indicates that the direct_mode_flag field may be present in the geometry node syntax. For example, the inferred_direct_coding_mode_enabled_flag field equal to 0 indicates that the direct_mode_flag field is not present in the geometry node syntax.

The bitwise_occupancy_coding_flag field indicates whether geometry node occupancy is encoded using bitwise contextualization of the syntax element occupancy map. For example, the bitwise_occupancy_coding_flag field equal to 1 indicates that geometry node occupancy is encoded using bitwise contextualisation of the syntax element occupancy_map. For example, the bitwise_occupancy_coding_flag field equal to 0 indicates that geometry node occupancy is encoded using the dictionary encoded syntax element occupancy_byte.

The adjacent_child_contextualization_enabled_flag field indicates whether the adjacent children of neighboring octree nodes are used for bitwise occupancy contextualization. For example, the adjacent_child_contextualization_enabled_flag field equal to 1 indicates that the adjacent children of neighboring octree nodes are used for bitwise occupancy contextualization. For example, adjacent_child_contextualization_enabled_flag equal to 0 indicates that the children of neighbouring octree nodes are not used for the occupancy contextualization.

The log 2_neighbour_avail_boundary field specifies the value of the variable NeighbAvailBoundary that is used in the decoding process as follows:

NeighbAvailBoundary=2^{log2_neighbour_avail_boundary}