Nvidia Patent | Watertight ray triangle intersection

小编映维 | 分类：Nvidia | 2021年1月15日

Patent: Watertight ray triangle intersection

Publication Number: 20210012552

Publication Date: 20210114

Applicant: Nvidia

Abstract

A hardware-based traversal coprocessor provides acceleration of tree traversal operations searching for intersections between primitives represented in a tree data structure and a ray. The primitives may include triangles used in generating a virtual scene. The hardware-based traversal coprocessor is configured to properly handle numerically challenging computations at or near edges and/or vertices of primitives and/or ensure that a single intersection is reported when a ray intersects a surface formed by primitives at or near edges and/or vertices of the primitives.

Claims

(canceled)
A system including: memory that stores at least a portion of an acceleration data structure including a plurality of hierarchical nodes, at least one node identifying a primitive range of a virtual scene; and hardware circuitry operatively coupled to the memory and configured to: determine intersections between a ray defined in a ray coordinate space and primitives in the primitive range; and when the ray is determined to intersect an edge or vertex shared by N primitives, where N>1, select less than N primitives to report based on a tie-breaking rule that compares vertex values of vertices shared by the N primitives in the ray coordinate space.
The system of claim 2, wherein the primitives include triangles and determining intersections between the ray and the triangles includes, for each triangle: transforming triangle vertices into the ray coordinate space; determining edge function values of post-transform triangle edges defined by transformed triangle vertices; if all edge function values are nonzero and have a same sign, identifying the triangle as being intersected by the ray; and if one or more of the edge function values is zero, determining whether the triangle is intersected based on whether another triangle shares the triangle vertices and whether one or more edge function values of the other triangle are zero.
The system of claim 2, wherein the hardware circuitry includes a fused floating point operation unit configured to combine multiple floating-point arithmetic operations to determine intersections between the ray and the primitives.
The system of claim 4, wherein the fused floating point operation unit is configured to perform arithmetic operations using single precision floating-point format.
The system of claim 4, wherein the fused floating point operation unit is configured to perform arithmetic operations with expanded 10-bit exponent range.
The system of claim 2, wherein the hardware circuitry is configured to determine primitives in the primitive range received from the memory which are intersected by the ray without using double-precision floating-point arithmetic.
A system including: memory that stores at least a portion of an acceleration data structure including a plurality of hierarchical nodes, at least one node identifying a primitive range of a virtual scene; and hardware circuitry operatively coupled to the memory and comprising: projection circuitry configured to project vertex values of primitives in the primitive range received from the memory into a 2D coordinate space of a ray; and intersection test circuitry coupled to the projection circuitry and configured to determine intersections between the ray and primitives in the primitive range based on the projected vertex values and to report, when the ray is determined to intersect an edge or vertex shared by a plurality of primitives, a single primitive of the plurality of primitives based on a comparison of the projected vertex values of the plurality of primitives.
The system of claim 8, wherein the intersection test circuitry includes a plurality of fused floating point operation units configured to combine multiple floating-point arithmetic operations used to determine primitives in the primitive range intersected by the ray.
The system of claim 9, wherein the fused floating point operation unit is configured to perform arithmetic operations with expanded 10-bit exponent range.
A tree traversal unit comprising: ray-surface testing hardware configured to test whether a ray intersects plural geometric surfaces arranged in 3D space; and tie breaking hardware configured to report a single surface from multiple geometric surfaces when the ray-surface testing hardware determines the ray intersects each of the multiple geometric surfaces at an edge or vertex shared by the multiple geometric surfaces.
The tree traversal unit of claim 11, wherein ray-surface testing hardware is configured to receive, from a processor, a query including ray information and information about a bounding volume hierarchy, traverse the bounding volume hierarchy to determine a bounding volume within said bounding volume hierarchy the ray intersects, and test intersection between the ray and the plural geometric surfaces associated with the bounding volume intersected by the ray.
The tree traversal unit of claim 11, wherein the ray-surface testing hardware includes a spatial transformer that projects the plural geometric surfaces onto a plane for ray intersection testing.
A tree traversal unit comprising hardware circuitry configured to: receive, from a processor, instructions to test intersections between a ray and a plurality of primitives; determine, using an arithmetic unit, whether the ray going thought a surface formed by the plurality of primitives intersects an edge or vertex shared by N primitives, N>1; and when the ray is determined to intersect the edge or vertex shared by N primitives, push intersection information for less than N primitives sharing the edge or vertex onto a stack; and report, to the processor, intersection information in the stack.
The tree traversal unit of claim 14, wherein when the ray is determined to intersect the edge or vertex shared by the N primitives, intersection information for a single primitive of the N primitives is push onto the stack.
The tree traversal unit of claim 14, wherein the hardware circuitry is further configured to, when the ray is determined to intersect the edge or vertex of a primitive not shared by another primitive, push onto the stack intersection information for the intersected primitive.
The tree traversal unit of claim 14, wherein the hardware circuitry is further configured to, when the ray is determined to intersect a primitive not on an edge or vertex, push onto the stack intersection information for the intersected primitive.
The tree traversal unit of claim 14, wherein the intersection information for less than N primitives from the N primitives is determined based on a tie-breaking rule that compares vertex values of vertices shared by the N primitives transformed into ray coordinate space.

Description

CROSS-REFERENCE TO RELATED PATENTS AND APPLICATIONS

[0001] This application is related to the following commonly-assigned US patents and patent applications, the entire contents of each of which are incorporated by reference: U.S. application Ser. No. 14/563,872 titled “Short Stack Traversal of Tree Data Structures” filed Dec. 8, 2014; U.S. Pat. No. 9,582,607 titled “Block-Based Bounding Volume Hierarchy”; U.S. Pat. No. 9,552,664 titled “Relative Encoding For A Block-Based Bounding Volume Hierarchy” as; U.S. Pat. No. 9,569,559 titled “Beam Tracing” filed Mar. 18, 2015; U.S. Pat. No. 10,025,879 titled “Tree Data Structures Based on a Plurality of Local Coordinate Systems”; U.S. application Ser. No. 14/737,343 titled “Block-Based Lossless Compression of Geometric Data” filed Jun. 11, 2015; and the following US applications filed concurrently herewith: [0002] (Atty. Docket: 6610-0032/18-AU-127) titled “Method for Continued Bounding Volume Hierarchy Traversal on Intersection without Shader Intervention”; [0003] (Atty. Docket: 6610-0033/18-AU-0128) titled “Method for Efficient Grouping of Cache Requests for Datapath Scheduling”; [0004] (Atty. Docket: 6610-0034/18-SC-0141) titled “A Robust, Efficient Multiprocessor-Coprocessor Interface”; [0005] (Atty. Docket: 6610-0035/18-SC-0144) titled “Query-Specific Behavioral Modification of Tree Traversal”; [0006] (Atty. Docket 6610-0037/18-SC-0149) titled “Method for Handling Out-of-Order Opaque and Alpha Ray/Primitive Intersections”; and [0007] (Atty. Docket 6610-0039/18-AU-0170) titled “Method for Forward Progress and Programmable Timeouts of Tree Traversal Mechanisms in Hardware”.

FIELD

[0008] The present technology relates to computer graphics, and more particularly to ray tracers. More particularly, the technology relates to hardware acceleration of computer graphics processing including but not limited to ray tracing. Still more particularly, the example non-limiting technology herein relates to a hardware-based traversal coprocessor that efficiently traverses an acceleration data structure e.g., for real time ray tracing. In still more detail, the technology herein provides an improved hardware-based traversal coprocessor for handling intersections at or near edges and/or vertices shared by triangles to provide watertight ray triangle intersections.

BACKGROUND &* SUMMARY*

[0009] If you look around the visual scene before you, you will notice that some of the most interesting visual effects you see are produced by light rays interacting with surfaces. This is because light is the only thing we see. We don’t see objects–we see the light that is reflected or refracted by the objects. Most of the objects we can see reflect light (the color of an object is determined by which parts of light the object reflects and which parts it absorbs). Shiny surfaces such as metallic surfaces, glossy surfaces, ceramics, the surfaces of liquids and a variety of others (even the corneas of the human eyes) act as mirrors that specularly reflect light. For example, a shiny metal surface will reflect light at the same angle as it hit the surface. An object can also cast shadows by preventing light from reaching other surfaces that are behind the object relative to a light source. If you look around, you will notice that the number and kinds of reflections and the number, kinds and lengths of shadows depend on many factors including the number and type of lights in the scene. A single point light such as a single faraway light bulb will produce single reflections and hard shadows. Area light sources such as windows or light panels produce different kinds of reflection highlights and softer shadows. Multiple lights will typically produce multiple reflections and more complex shadows (for example, three separated point light sources will produce three shadows which may overlap depending on the positions of the lights relative to an object).

[0010] If you move your head as you survey the scene, you will notice that the reflections change in position and shape (the shadows do the same). By changing your viewpoint, you are changing the various angles of the light rays your eyes detect. This occurs instantaneously–you move your head and the visual scene changes immediately.

[0011] The simple act of drinking a cup of tea is a complex visual experience. The various shiny surfaces of the glossy ceramic cup on the table before you reflect each light in the room, and the cup casts a shadow for each light. The moving surface of the tea in the cup is itself reflective. You can see small reflected images of the lights on the tea’s surface, and even smaller reflections on the part of the tea’s surface where the liquid curves up to meet the walls of the cup. The cup walls also cast shadows onto the surface of the liquid in the cup. Lifting the cup to your mouth causes these reflections and shadows to shift and shimmer as your viewpoint changes and as the surface of the liquid is agitated by movement.

[0012] We take these complexities of reflections and shadows for granted. Our brains are adept at decoding the positions, sizes and shapes of shadows and reflections and using them as visual cues. This is in part how we discern the position of objects relative to one another, how we distinguish one object from another and how we learn what objects are made of. Different object surfaces reflect differently. Specular (mirror type) reflection of hard metal creates images of reflected objects, while diffuse reflection off of rough surfaces is responsible for color and lights up objects in a softer way. Shadows can be soft and diffuse or hard and distinct depending on the type of lighting, and the lengths and directions of the shadows will depend on the angle of the light rays relative to the object and our eyes.

[0013] Beginning artists typically don’t try to show reflection or shadows. They tend to draw flat scenes that have no shadows and no reflections or highlights. The same was true with computer graphics of the past.

[0014] Real time computer graphics have advanced tremendously over the last 30 years. With the development in the 1980’s of powerful graphics processing units (GPUs) providing 3D hardware graphics pipelines, it became possible to produce 3D graphical displays based on texture-mapped polygon primitives in real time response to user input. Such real time graphics processors were built upon a technology called scan conversion rasterization, which is a means of determining visibility from a single point or perspective. Using this approach, three-dimensional objects are modelled from surfaces constructed of geometric primitives, typically polygons such as triangles. The scan conversion process establishes and projects primitive polygon vertices onto a view plane and fills in the points inside the edges of the primitives. See e.g., Foley, Van Dam, Hughes et al, Computer Graphics: Principles and Practice (2d Ed. Addison-Wesley 1995 & 3d Ed. Addison-Wesley 2014).

[0015] Hardware has long been used to determine how each polygon surface should be shaded and texture-mapped and to rasterize the shaded, texture-mapped polygon surfaces for display. Typical three-dimensional scenes are often constructed from millions of polygons. Fast modern GPU hardware can efficiently process many millions of graphics primitives for each display frame (every 1/30th or 1/60th of a second) in real time response to user input. The resulting graphical displays have been used in a variety of real time graphical user interfaces including but not limited to augmented reality, virtual reality, video games and medical imaging. But traditionally, such interactive graphics hardware has not been able to accurately model and portray reflections and shadows.

[0016] Some have built other technologies onto this basic scan conversion rasterization approach to allow real time graphics systems to accomplish a certain amount of realism in rendering shadows and reflections. For example, texture mapping has sometimes been used to simulate reflections and shadows in a 3D scene. One way this is commonly done is to transform, project and rasterize objects from different perspectives, write the rasterized results into texture maps, and sample the texture maps to provide reflection mapping, environment mapping and shadowing. While these techniques have proven to be useful and moderately successful, they do not work well in all situations. For example, so-called “environment mapping” may often require assuming the environment is infinitely distant from the object. In addition, an environment-mapped object may typically be unable to reflect itself. See e.g., http://developer.download.nvidia.com/CgTitorial/eg_tutorial_chapter07.htm- l. These limitations result because conventional computer graphics hardware–while sufficiently fast for excellent polygon rendering–does not perform the light visualization needed for accurate and realistic reflections and shadows. Some have likened raster/texture approximations of reflections and shadows as the visual equivalent of AM radio.

[0017] There is another graphics technology which does perform physically realistic visibility determinations for reflection and shadowing. It is called “ray tracing”. Ray tracing was developed at the end of the 1960’s and was improved upon in the 1980’s. See e.g., Apple, “Some Techniques for Shading Machine Renderings of Solids” (SJCC 1968) pp. 27-45; Whitted, “An Improved Illumination Model for Shaded Display” Pages 343-349 Communications of the ACM Volume 23 Issue 6 (June 1980); and Kajiya, “The Rendering Equation”, Computer Graphics (SIGGRAPH 1986 Proceedings, Vol. 20, pp. 143-150). Since then, ray tracing has been used in non-real time graphics applications such as design and film making. Anyone who has seen “Finding Dory” (2016) or other Pixar animated films has seen the result of the ray tracing approach to computer graphics–namely realistic shadows and reflections. See e.g., Hery et al, “Towards Bidirectional Path Tracing at Pixar” (2016).

[0018] Ray tracing is a primitive used in a variety of rendering algorithms including for example path tracing and Metropolis light transport. In an example algorithm, ray tracing simulates the physics of light by modeling light transport through the scene to compute all global effects (including for example reflections from shiny surfaces) using ray optics. In such uses of ray tracing, an attempt may be made to trace each of many hundreds or thousands of light rays as they travel through the three-dimensional scene from potentially multiple light sources to the viewpoint. Often, such rays are traced relative to the eye through the scene and tested against a database of all geometry in the scene. The rays can be traced forward from lights to the eye, or backwards from the eye to the lights, or they can be traced to see if paths starting from the virtual camera and starting at the eye have a clear line of sight. The testing determines either the nearest intersection (in order to determine what is visible from the eye) or traces rays from the surface of an object toward a light source to determine if there is anything intervening that would block the transmission of light to that point in space. Because the rays are similar to the rays of light in reality, they make available a number of realistic effects that are not possible using the raster based real time 3D graphics technology that has been implemented over the last thirty years. Because each illuminating ray from each light source within the scene is evaluated as it passes through each object in the scene, the resulting images can appear as if they were photographed in reality. Accordingly, these ray tracing methods have long been used in professional graphics applications such as design and film, where they have come to dominate over raster-based rendering.

[0019] The main challenge with ray tracing has generally been speed. Ray tracing requires the graphics system to compute and analyze, for each frame, each of many millions of light rays impinging on (and potentially reflected by) each surface making up the scene. In the past, this enormous amount of computation complexity was impossible to perform in real time.

[0020] One reason modern GPU 3D graphics pipelines are so fast at rendering shaded, texture-mapped surfaces is that they use coherence efficiently. In conventional scan conversion, everything is assumed to be viewed through a common window in a common image plane and projected down to a single vantage point. Each triangle or other primitive is sent through the graphics pipeline and covers some number of pixels. All related computations can be shared for all pixels rendered from that triangle. Rectangular tiles of pixels corresponding to coherent lines of sight passing through the window may thus correspond to groups of threads running in lock-step in the same streaming processor. All the pixels falling between the edges of the triangle are assumed to be the same material running the same shader and fetching adjacent groups of texels from the same textures. In ray tracing, in contrast, rays may start or end at a common point (a light source, or a virtual camera lens) but as they propagate through the scene and interact with different materials, they quickly diverge. For example, each ray performs a search to find the closest object. Some caching and sharing of results can be performed, but because each ray potentially can hit different objects, the kind of coherence that GPU’s have traditionally taken advantage of in connection with texture mapped, shaded triangles is not present (e.g., a common vantage point, window and image plane are not there for ray tracing). This makes ray tracing much more computationally challenging than other graphics approaches–and therefore much more difficult to perform on an interactive basis.

[0021] Much research has been done on making the process of tracing rays more efficient and timely. See e.g., Glassner, An Introduction to Ray Tracing (Academic Press Inc., 1989). Because each ray in ray tracing is, by its nature, evaluated independently from the rest, ray tracing has been called “embarrassingly parallel.” See e.g., Akenine-Moller et al., Real Time Rendering at Section 9.8.2, page 412 (Third Ed. CRC Press 2008). As discussed above, ray tracing involves effectively testing each ray against all objects and surfaces in the scene. An optimization called “acceleration data structure” and associated processes allows the graphics system to use a “divide-and-conquer” approach across the acceleration data structure to establish what surfaces the ray hits and what surfaces the ray does not hit. Each ray traverses the acceleration data structure in an individualistic way. This means that dedicating more processors to ray tracing gives a nearly linear performance increase. With increasing parallelism of graphics processing systems, some began envisioning the possibility that ray tracing could be performed in real time. For example, work at Saarland University in the mid-2000’s produced an early special purpose hardware system for interactive ray tracing that provided some degree of programmability for using geometry, vertex and lighting shaders. See Woop et al., “RPU: A Programmable Ray Processing Unit for Real Time Ray Tracing” (ACM 2005). As another example, Advanced Rendering Technology developed “RenderDrive” based on an array of AR250/350 rendering processors derived from ARM1 and enhanced with custom pipelines for ray/triangle intersection and SIMD vector and texture math but with no fixed-function traversal logic. See e.g., http://www.graphicshardware.org/previous/www_2001/presentations/Hot3D_Dan- iel_Hall.pdf

[0022] Then, in 2010, NVIDIA took advantage of the high degree of parallelism of NVIDIA GPUs and other highly parallel architectures to develop the OptiX.TM. raytracing engine. See Parker et al., “OptiX: A General Purpose Ray Tracing Engine” (ACM Transactions on Graphics, Vol. 29, No. 4, Article 66, July 2010). In addition to improvements in API’s (application programming interfaces), one of the advances provided by OptiX.TM. was improving the acceleration data structures used for finding an intersection between a ray and the scene geometry. Such acceleration data structures are usually spatial or object hierarchies used by the ray tracing traversal algorithm to efficiently search for primitives that potentially intersect a given ray. OptiX.TM. provides a number of different acceleration structure types that the application can choose from. Each acceleration structure in the node graph can be a different type, allowing combinations of high-quality static structures with dynamically updated ones.

[0023] The OptiX.TM. programmable ray tracing pipeline provided significant advances, but was still generally unable by itself to provide real time interactive response to user input on relatively inexpensive computing platforms for complex 3D scenes. Since then, NVIDIA has been developing hardware acceleration capabilities for ray tracing. See e.g., U.S. Pat. Nos. 9,582,607; 9,569,559; US20160070820; and US20160070767.

[0024] Given the great potential of a truly interactive real time ray tracing graphics processing system for rendering high quality images of arbitrary complexity in response for example to user input, further work is possible and desirable.

[0025] Some Ray Processes Can Provide Incorrect Results or May be Limited to Software Implementations

[0026] Ray tracing needs to resolve ray intersections when a ray passes at edges and/or vertices shared by triangles. At these locations incorrect results may be provided when, for example, an intersection is not identified or multiple incorrect intersections are identified. A Ray/Triangle Test data pipe that attempts to resolve ray intersections with edges shared between triangles is known. See e.g., Woop et al, “Watertight Ray/Triangle Intersection”, Journal of Computer Graphics Techniques Vol. 2, No. 1, 2013 (2013), which is hereby incorporated by reference. Such implementations were designed for software ray tracing on a CPU, and fall back to double precision whenever one of the barycentric coordinates is zero-valued. For example, single precision is used to obtain first results and if the first result could be considered to not be correct or precise then a second calculation using double precision is performed. The technique of Woop et al. also yields multiple reported intersections when ray passes exactly through an edge or a vertex, which is traditionally avoided in rasterization-based graphics. However, accelerator hardware may not have double precision computation capabilities. Therefore, further improvements are possible and desirable.

BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG. 1 illustrates an example non-limiting ray tracing graphics system.

[0028] FIG. 2A shows an example specular object.

[0029] FIG. 2B shows the example object within a bounding volume.

[0030] FIG. 2C shows an example volumetric subdividing of the FIG. 2B bounding volume.

[0031] FIGS. 2D, 2E and 2F show example further levels of volumetric subdivision of the bounding volume to create a bounding volume hierarchy (BVH).

[0032] FIG. 2G shows an example portion of the object comprised of primitive surfaces, in this case triangles.

[0033] FIGS. 3A-3C show example simplified ray tracing tests to determine whether the ray passes through a bounding volume containing geometry and whether the ray intersects geometry.

[0034] FIG. 4 illustrates an example ray tracing flowchart.

[0035] FIGS. 5A-5C show example different ray-primitive intersection scenarios.

[0036] FIGS. 6A and 6B show an example of how texture mapping can impact ray-primitive intersection results.

[0037] FIGS. 7A and 7B illustrate ray instance transforms.

[0038] FIG. 8A illustrates an example non-limiting bounding volume hierarchy (BVH).

[0039] FIG. 8B shows an example acceleration data structure in the form of a graph or tree.

[0040] FIG. 9 shows a simplified example non-limiting traversal co-processor comprising a tree traversal unit (TTU).

[0041] FIG. 10A illustrates an example non-limiting ray tracing shading pipeline flowchart.

[0042] FIGS. 10B and 10C illustrate more detailed ray tracing pipelines.

[0043] FIG. 11A illustrates a ray going exactly through an edge (in post-shear coordinates) without entering a box.

[0044] FIG. 11B illustrates an example non-limiting method for determining ray-primitives intersections.

[0045] FIG. 12 shows an example non-limiting process for determining intersections between a ray and one or more triangles.

[0046] FIG. 13 shows a single-precision floating-point representation and expanded exponent single-precision floating-point representation.

[0047] FIG. 14 illustrates an example non-limiting ray-triangle intersection test providing a single hit return for a closed mesh.

[0048] FIG. 15 illustrates an example non-limiting ray-triangle test and transform unit.

[0049] FIG. 16A illustrates a tie-breaking scheme obtained from following hit patterns for edges.

[0050] FIG. 16B illustrates a triangle fan with vertex shared at the origin by a plurality of triangles.

[0051] FIG. 17 illustrates an example flowchart for generating an image.

[0052] FIG. 18 illustrates an example parallel processing unit (PPU).

[0053] FIG. 19 illustrates an example memory partition unit.

[0054] FIG. 20 illustrates an example general processing cluster (GPC) within the parallel processing unit of FIG. 18.

[0055] FIG. 21 is a conceptual diagram of a graphics processing pipeline implemented by the GPC of FIG. 20.

[0056] FIGS. 22 and 23 illustrate an example streaming multi-processor.

[0057] FIG. 24 is a conceptual diagram of a processing system implemented using PPUs of FIG. 18.

[0058] FIG. 25 expands FIG. 24 to show additional interconnected devices.

DETAILED DESCRIPTION OF NON-LIMITING EMBODIMENTS

[0059] The technology herein provides hardware capabilities that accelerate ray tracing to such an extent that it brings the power of ray tracing to games and other interactive real time computer graphics, initially enabling high effect quality in shadows and reflections and ultimately global illumination. In practice, this means accelerating ray tracing by a factor of up to an order of magnitude or more over what would be possible in software on the same graphics rendering system.

[0060] In more detail, the example non-limiting technology provides dedicated hardware to accelerate ray tracing. In non-limiting embodiments, a hardware co-processor (herein referred to as a “traversal coprocessor” or in some embodiments a “tree traversal unit” or “TTU”) accelerates certain processes supporting interactive ray tracing including ray-bounding volume intersection tests, ray-primitive intersection tests and ray “instance” transforms.

[0061] In some non-limiting embodiments, the traversal co-processor performs queries on an acceleration data structure for processes running on potentially massively-parallel streaming multiprocessors (SMs). The traversal co-processor traverses the acceleration data structure to discover information about how a given ray interacts with an object the acceleration data structure describes or represents. For ray tracing, the traversal coprocessors are callable as opposed to e.g., fixed function units that perform an operation once between logical pipeline stages running different types of threads (e.g., vertex threads and pixel threads).

[0062] In some non-limiting embodiments, the acceleration data structure comprises a hierarchy of bounding volumes (bounding volume hierarchy or BVH) that recursively encapsulates smaller and smaller bounding volume subdivisions. The largest volumetric bounding volume may be termed a “root node.” The smallest subdivisions of such hierarchy of bounding volumes (“leaf nodes”) contain items. The items could be primitives (e.g., polygons such as triangles) that define surfaces of the object. Or, an item could be a sphere that contains a whole new level of the world that exists as an item because it has not been added to the BVH (think of the collar charm on the cat from “Men in Black” which contained an entire miniature galaxy inside of it). If the item comprises primitives, the traversal co-processor tests rays against the primitives to determine which object surfaces the rays intersect and which object surfaces are visible along the ray.

[0063] The traversal co-processor performs a test of each ray against a wide range of bounding volumes, and can cull any bounding volumes that don’t intersect with that ray. Starting at a root node that bounds everything in the scene, the traversal co-processor tests each ray against smaller (potentially overlapping) child bounding volumes which in turn bound the descendent branches of the BVH. The ray follows the child pointers for the bounding volumes the ray hits to other nodes until the leaves or terminal nodes (volumes) of the BVH are reached. Once the traversal co-processor traverses the acceleration data structure to reach a terminal or “leaf” node that contains a geometric primitive, it performs an accelerated ray-primitive intersection test that determines whether the ray intersects that primitive (and thus the object surface that primitive defines). The ray-primitive test can provide additional information about primitives the ray intersects that can be used to determine the material properties of the surface required for shading and visualization. Recursive traversal through the acceleration data structure enables the traversal co-processor to discover all object primitives the ray intersects, or the closest (from the perspective of the viewpoint) primitive the ray intersects (which in some cases is the only primitive that is visible from the viewpoint along the ray).

[0064] The traversal co-processor also accelerates the transform of each ray from world space into object space to obtain finer and finer bounding box encapsulations of the primitives and reduce the duplication of those primitives across the scene. Objects replicated many times in the scene at different positions, orientations and scales can be represented in the scene as instance nodes which associate a bounding box and leaf node in the world space BVH with a transformation that can be applied to the world-space ray to transform it into an object coordinate space, and a pointer to an object-space BVH. This avoids replicating the object space BVH data multiple times in world space, saving memory and associated memory accesses. The instance transform increases efficiency by transforming the ray into object space instead of requiring the geometry or the bounding volume hierarchy to be transformed into world (ray) space and is also compatible with additional, conventional rasterization processes that graphics processing performs to visualize the primitives.

[0065] The presently disclosed non-limiting embodiments thus provide a traversal co-processor, a new subunit of one or a group of streaming multiprocessor SMs of a 3D graphics processing pipeline. In order to understand where the traversal co-processor fits in the overall picture, it may be helpful to understand a few fundamentals of the algorithm employed by most or all modern ray tracers. But it should be pointed out that the technology herein provides a generic capability to determine, for a thread running in a GPU, what the nearest visible thing is from a given point along a specified direction, or if anything lies between two points. A common use case for such capability will be in processes that start tracing rays from points that have already been rasterized on triangles using conventional scan conversion techniques. The disclosed technology can but does not necessarily replace or substitute for scan conversion technology, and may often augment it and be used in conjunction with scan conversion techniques to enhance images with photorealistic reflections, shadows and other effects.

[0066] Ray Tracing Techniques

[0067] Generally, ray tracing is a rendering method in which rays are used to determine the visibility of various elements in the scene. Ray tracing can be used to determine if anything is visible along a ray (for example, testing for occluders between a shaded point on a geometric primitive and a point on a light source) and can also be used to evaluate reflections (which may for example involve performing a traversal to determine the nearest visible surface along a line of sight so that software running on a streaming processor can evaluate a material shading function corresponding to what was hit–which in turn can launch one or more additional rays into the scene according to the material properties of the object that was intersected) to determine the light returning along the ray back toward the eye. In classical Whitted-style ray tracing, rays are shot from the viewpoint through the pixel grid into the scene, but other path traversals are possible. Typically, for each ray, the closest object is found. This intersection point can then be determined to be illuminated or in shadow by shooting a ray from it to each light source in the scene and finding if any objects are in between. Opaque objects block the light, whereas transparent objects attenuate it. Other rays can be spawned from an intersection point. For example, if the intersecting surface is shiny or specular, rays are generated in the reflection direction. The ray may accept the color of the first object intersected, which in turn has its intersection point tested for shadows. This reflection process is recursively repeated until a recursion limit is reached or the potential contribution of subsequent bounces falls below a threshold. Rays can also be generated in the direction of refraction for transparent solid objects, and again recursively evaluated. See Akenine-Moller et al., cited above. Ray tracing technology thus allows a graphics system to develop physically correct reflections and shadows that are not subject to the limitations and artifacts of scan conversion techniques.

[0068] Traversal Coprocessor

[0069] The basic task the traversal coprocessor performs is to test a ray against all primitives (commonly triangles in one embodiment) in the scene and report either the closest hit (according to distance measured along the ray) or simply the first (not necessarily closest) hit encountered, depending upon use case. The naive algorithm would be an O(n) brute-force search. By pre-processing the scene geometry and building a suitable acceleration data structure in advance, however, it is possible to reduce the average-case complexity to O(log n). In ray tracing, the time for finding the closest (or for shadows, any) intersection for a ray is typically order O(log n) for n objects when an acceleration data structure is used. For example, bounding volume hierarchies (BVHs) of the type commonly used for modern ray tracing acceleration data structures typically have an O(log n) search behavior.

[0070] Bounding Volume Hierarchies

[0071] The acceleration data structure most commonly used by modern ray tracers is a bounding volume hierarchy (BVH) comprising nested axis-aligned bounding boxes (AABBs). The leaf nodes of the BVH contain the primitives (e.g., triangles) to be tested for intersection. The BVH is most often represented by a graph or tree structure data representation. In such instances, the traversal coprocessor may be called a “tree traversal unit” or “TTU”.

[0072] Given a BVH, ray tracing amounts to a tree search where each node in the tree visited by the ray has a bounding volume for each descendent branch or leaf, and the ray only visits the descendent branches or leaves whose corresponding bound volume it intersects. In this way, only a small number of primitives must be explicitly tested for intersection, namely those that reside in leaf nodes intersected by the ray. In the example non-limiting embodiments, the traversal coprocessor accelerates both tree traversal (including the ray-volume tests) and ray-primitive tests. As part of traversal, the traversal coprocessor can also handle “instance transforms”–transforming a ray from world-space coordinates into the coordinate system of an instanced mesh (object space) e.g., in order to avoid the computational complexity of transforming the primitive vertices into world space. It can do so in a MIMD (multiple-instruction, multiple data) fashion, meaning that the rays are handled independently once inside the traversal coprocessor.

[0073] Example Non-Limiting Real Time Interactive Ray Tracing System

[0074] FIG. 1 illustrates an example real time ray interactive tracing graphics system 100 for generating images using three dimensional (3D) data of a scene or object(s). System 100 includes an input device 110, a processor(s) 120, a graphics processing unit(s) (GPU(s)) 130, memory 140, and a display(s) 150. The system shown in FIG. 1 can take on any form factor including but not limited to a personal computer, a smart phone or other smart device, a video game system, a wearable virtual or augmented reality system, a cloud-based computing system, a vehicle-mounted graphics system, a system-on-a-chip (SoC), etc.

[0075] The processor 120 may be a multicore central processing unit (CPU) operable to execute an application in real time interactive response to input device 110, the output of which includes images for display on display 150. Display 150 may be any kind of display such as a stationary display, a head mounted display such as display glasses or goggles, other types of wearable displays, a handheld display, a vehicle mounted display, etc. For example, the processor 120 may execute an application based on inputs received from the input device 110 (e.g., a joystick, an inertial sensor, an ambient light sensor, etc.) and instruct the GPU 130 to generate images showing application progress for display on the display 150.

[0076] Based on execution of the application on processor 120, the processor may issue instructions for the GPU 130 to generate images using 3D data stored in memory 140. The GPU 130 includes specialized hardware for accelerating the generation of images in real time. For example, the GPU 130 is able to process information for thousands or millions of graphics primitives (polygons) in real time due to the GPU’s ability to perform repetitive and highly-parallel specialized computing tasks such as polygon scan conversion much faster than conventional software-driven CPUs. For example, unlike the processor 120, which may have multiple cores with lots of cache memory that can handle a few software threads at a time, the GPU 130 may include hundreds or thousands of processing cores or “streaming multiprocessors” (SMs) 132 running in parallel.

[0077] In one example embodiment, the GPU 130 includes a plurality of programmable streaming multiprocessors (SMs) 132, and a hardware-based graphics pipeline including a graphics primitive engine 134 and a raster engine 136. These components of the GPU 130 are configured to perform real-time image rendering using a technique called “scan conversion rasterization” to display three-dimensional scenes on a two-dimensional display 150. In rasterization, geometric building blocks (e.g., points, lines, triangles, quads, meshes, etc.) of a 3D scene are mapped to pixels of the display (often via a frame buffer memory).

[0078] The GPU 130 converts the geometric building blocks (i.e., polygon primitives such as triangles) of the 3D model into pixels of the 2D image and assigns an initial color value for each pixel. The graphics pipeline may apply shading, transparency, texture and/or color effects to portions of the image by defining or adjusting the color values of the pixels. The final pixel values may be anti-aliased, filtered and provided to the display 150 for display. Many software and hardware advances over the years have improved subjective image quality using rasterization techniques at frame rates needed for real-time graphics (i.e., 30 to 60 frames per second) at high display resolutions such as 4096.times.2160 pixels or more on one or multiple displays 150.

[0079] Traversal Coprocessor Addition to Architecture

[0080] To enable the GPU 130 to perform ray tracing in real time in an efficient manner, the GPU is provided with traversal coprocessor 138 coupled to one or more SMs 132. The traversal coprocessor 138 includes hardware components configured to perform operations commonly utilized in ray tracing algorithms. A goal of the traversal coprocessor 138 is to accelerate operations used in ray tracing to such an extent that it brings the power of ray tracing to real-time graphics application (e.g., games), enabling high-quality shadows, reflections, and global illumination. As discussed in more detail below, the result of the traversal coprocessor 138 may be used together with or as an alternative to other graphics related operations performed in the GPU 130.

[0081] In the example architecture shown, the new hardware component called a “traversal coprocessor” 138 is used to accelerate certain tasks including but not limited to ray tracing. Ray tracing refers to casting a ray into a scene and determining whether and where that ray intersects the scene’s geometry. This basic ray tracing visibility test is the fundamental primitive underlying a variety of rendering algorithms and techniques in computer graphics. For example, ray tracing can be used together with or as an alternative to rasterization and z-buffering for sampling scene geometry. It can also be used as an alternative to (or in combination with) environment mapping and shadow texturing for producing more realistic reflection, refraction and shadowing effects than can be achieved via texturing techniques or other raster “hacks”. To overcome limitations in image quality that can be achieved with rasterization, system 100 can also generate entire images or parts of images using ray tracing techniques. Ray tracing may also be used as the basic primitive to accurately simulate light transport in physically-based rendering algorithms such as path tracing, photon mapping, Metropolis light transport, and other light transport algorithms.

[0082] More specifically, SMs 132 and the traversal coprocessor 138 may cooperate to cast rays into a 3D model and determine whether and where that ray intersects the model’s geometry. Ray tracing directly simulates light traveling through a virtual environment or scene. The results of the ray intersections together with surface texture, viewing direction, and/or lighting conditions are used to determine pixel color values. Ray tracing performed by SMs 132 working with traversal coprocessor 138 allows for computer-generated images to capture shadows, reflections, and refractions in ways that can be indistinguishable from photographs or video of the real world. Since ray tracing techniques are even more computationally intensive than rasterization due in part to the large number of rays that need to be traced, the traversal coprocessor 138 is capable of accelerating in hardware certain of the more computationally-intensive aspects of that process.

[0083] In the example non-limiting technology herein, traversal coprocessor 138 accelerates both ray-box tests and ray-primitive tests. As part of traversal, it can also handle at least one level of instance transforms, transforming a ray from world-space coordinates into the coordinate system of an instanced mesh. In the example non-limiting embodiments, the traversal coprocessor 138 does all of this in MIMD fashion, meaning that rays are handled independently once inside the traversal coprocessor.

[0084] In the example non-limiting embodiments, the traversal coprocessor 138 operates as a servant (coprocessor) to the SMs (streaming multiprocessors) 132. In other words, the traversal coprocessor 138 in example non-limiting embodiments does not operate independently, but instead follows the commands of the SMs 132 to perform certain computationally-intensive ray tracing related tasks much more efficiently than the SMs 132 could perform themselves.

[0085] In the examples shown, the traversal coprocessor 138 receives commands via SM 132 instructions and writes results back to an SM register file. For many common use cases (e.g., opaque triangles with at most one level of instancing), the traversal coprocessor 138 can service the ray tracing query without further interaction with the SM 132. More complicated queries (e.g., involving alpha-tested triangles, primitives other than triangles, or multiple levels of instancing) may require multiple round trips. In addition to tracing rays, the traversal coprocessor 138 is capable of performing more general spatial queries where an AABB or the extruded volume between two AABBs (which we call a “beam”) takes the place of the ray. Thus, while the traversal coprocessor 138 is especially adapted to accelerate ray tracing related tasks, it can also be used to perform tasks other than ray tracing.

[0086] In addition to the traversal coprocessor 138, the example non-limiting technology used to support the system 100 of FIG. 1 provides additional accelerated ray tracing enhancements to a number of units as well as a substantial effort devoted to BVH construction. BVH construction need not be hardware accelerated (although it may be in some non-limiting embodiments) but could instead be implemented using highly-optimized software routines running on SMs 132 and/or CPU 120 and/or other development systems e.g., during development of an application. The following exposition describes, among other things, software-visible behavior of the traversal coprocessor 138, interfaces to surrounding units (SMs 132 and the memory subsystem), and additional features that are part of a complete ray-tracing solution such as certain enhancements to the group of SMs 132 and the memory caching system.

[0087] In example non-limiting embodiments, the traversal coprocessor 138 allows for quick traversal of an accelerated data structure (e.g., a BVH) to determine which primitives (e.g., triangles used for generating a scene) in the data structure are intersected by a query data structure (e.g., a ray). The traversal coprocessor 138 provides solutions for problems related to the practice of TTU-accelerated ray-traced rendering in a software application running on a GPU. Exemplary non-limiting embodiments of this disclosure, may extend, for example, the Tree Traversal Unit (TTU) described in U.S. Pat. No. 9,582,607 and the beam tracing method described in U.S. Pat. No. 9,569,559 to support mid-traversal ray/triangle intersection such that triangles can be handled directly by the TTU without requiring SM intervention, thus avoiding frequent costly synchronizations which may be needed without exemplary non-limiting embodiments of this disclosure. U.S. Pat. Nos. 9,582,607 and 9,569,559 are incorporated by reference.

[0088] In the example non-limiting embodiments, the TTU provides many novel aspects of the design of this datapath to ensure that the ray-triangle intersections and the resulting shaded images are robust in the face of numerically challenging inputs. To deliver a robust solution, the designs set out to provide: [0089] Watertight edge test–a ray cannot miss a pair of triangles that share a common edge by passing between them “through” the shared edge. [0090] Single hit–a ray that approaches an edge shared by two triangles should not accidentally report an intersection with both triangles. [0091] Watertight vertex test–a ray cannot miss a plurality of triangles that share a common vertex by passing between them “through” the shared vertex. [0092] Vertex single hit–a ray that approaches a vertex shared by a plurality of triangles should not accidentally report an intersection with multiple triangles sharing the vertex.

[0093] Exemplary non-limiting embodiments of this disclosure improve ray tracing performance without compromising image quality. By supporting accelerated ray/triangle intersection during traversal, the TTU is able to offload a substantial amount of computation from the streaming processors and avoid an expensive synchronization performance penalty. Because real-time ray tracing must provide as good or better image quality and stability as real-time rasterization, it is important that the TTU’s ray-triangle intersection exhibits the properties of watertightness, conservatism, and single-hit. Exemplary non-limiting embodiments of this disclosure allow for real-time ray tracing to be applied in real-time games employing millions of triangles.

[0094] Traversing an Acceleration Data Structure

[0095] A good way to accelerate ray tracing is to use an acceleration data structure. The acceleration data structure represents the 3D model of an object or a scene in a manner that will help assist in quickly deciding which portion of the object a particular ray is likely to intersect and quickly rejecting large portions of the scene the ray will not intersect. A bounding volume hierarchy (BVH) data structure is one type of acceleration data structure which can help reduce the number of intersections to test. The BVH data structure represents a scene or object with a bounding volume and subdivides the bounding volume into smaller and smaller bounding volumes terminating in leaf nodes containing geometric primitives. The bounding volumes are hierarchical, meaning that the topmost level encloses the level below it, that level encloses the next level below it, and so on. In one embodiment, leaf nodes can potentially overlap other leaf nodes in the bounding volume hierarchy.

[0096] To illustrate how a bounding volume hierarchy works, FIGS. 2A-2G show a teapot recursively subdivided into smaller and smaller hierarchical bounding volumes. FIG. 2A shows a teapot object, and FIG. 2B shows a bounding volume 202 (in this case a box, cube or rectangular parallelepiped) enclosing the whole teapot. The bounding volume 202, which can be efficiently defined by its vertices, provides an indication of the spatial location of the object and is typically dimensioned to be just slightly larger than the object.

[0097] The first stage in acceleration structure construction acquires the bounding boxes of the referenced geometry. This is achieved by executing for each geometric primitive in an object a bounding box procedure that returns a conservative axis-aligned bounding box for its input primitive such as box 202 shown in FIG. 2B. Using these bounding boxes as elementary primitives for the acceleration structures provides the necessary abstraction to trace rays against arbitrary user-defined geometry (including several types of geometry within a single structure). Because in FIG. 2B the bounding volume 202 is larger than and completely contains the teapot, a ray that does not intersect bounding volume cannot intersect the teapot, although a ray that does intersect the bounding volume may or may not intersect the teapot. Because the bounding volume 202 is readily defined by the x,y,z coordinates of its vertices in 3D space and a ray is defined by its x,y,z coordinates in 3D space, the ray-bounding volume test to determine whether a ray intersects the bounding volume 202 is straightforward (although some transform may be used to adjust to different coordinate systems, as will be explained below).

[0098] FIG. 2C, shows the bounding volume 202 subdivided into smaller contained bounding volumes. While the subdivision scheme shown here for purposes of illustration is a so-called 8-ary subdivision or “octree” in which each volume is subdivided into eight smaller volumes of uniform size, many other spatial hierarchies and subdivision schemes are known such as a binary tree, a four-ary tree, a k-d tree, a binary space partitioning (BSP) tree, and a bounding volume hierarchy (BVH) tree. See e.g., U.S. Pat. No. 9,582,607.

[0099] Each of the subdivided bounding volumes shown in FIG. 2C can be still further subdivided. FIG. 2D shows one of the subdivided volumes 204 of FIG. 2C being further subdivided to provide additional subdivided encapsulated bounding volumes. As shown in FIG. 2D, some of the subdivided bounding volumes include portions of the teapot and some do not. Volumes that do not contain a portion of the teapot are not further subdivided because the further subdivisions provide no further spatial information about the teapot. Already subdivided bounding volumes that do include at least one portion of the teapot can be still further recursively subdivided–like the emergence of each of a succession of littler and littler cats from the hats of Dr. Seuss’s `The Cat In The Hat Comes Back (1958). The portions of the space within bounding volume 202 that contain geometry are recursively subdivided to permit the traversal coprocessor 138 to use the volumetric subdivisions to efficiently discover where the geometry is located relative to any given ray. It can be noted that while a spatial or active subdivision of the volume is possible, many implementations will create the hierarchical structure defining volumes and subvolumes ahead of time. In such cases, the builder may often build the hierarchy up from individual triangles and not down from the whole scene. Building up means you do not need to determine if some subdivided volume contains anything since by definition it contains what is below it in a hierarchy of volumetric subdivisions.

[0100] FIG. 2E shows a further such subdivision of bounding volume 204 into a further smaller contained bounding volume 206 containing in this example just the spout of the teapot plus another surface on the wall of the teapot, and FIG. 2F shows an additional subdivision of bounding volume 206 into still smaller contained subdivision 208 encapsulating the end of the teapot’s spout. Depending on the way the BVH is constructed, bounding volume 208 can be further and further subdivided as desired–and traversal coprocessor 138 enables the FIG. 1 system 100 to efficiently traverse the BVH down to any arbitrary subdivision level. The number and configurations of recursive subdivisions will depend on the complexity and configuration of the 3D object being modeled as well as other factors such as desired resolution, distance of the object from the viewpoint, etc.

[0101] At some level of subdivision (which can be different levels for different parts of the BVH), the traversal coprocessor 138 encounters geometry making up the encapsulated object being modeled. Using the analogy of a tree, the successive volumetric subdivisions are the trunk, branches, boughs and twigs, and the geometric is finally revealed at the very tips of the tree, namely the leaves. In this case, FIG. 2G shows the surface of the teapot’s spout defined by an example mesh of geometric primitives. The geometric primitives shown are triangles but other geometric primitives, such as quads, lines, rectangles, quadrics, patches, or other geometric primitives known to those familiar with the state of the art, may be used (in one embodiment, such other types of primitives may be expressed as or converted into triangles). The geometric primitives in the mesh represent the shape of the 3D surface of the object being modeled. The example shown here is a mesh, but bounded geometry can include discontinuous geometry such as particles that may not be connected. In the example non-limiting embodiments, the traversal coprocessor 138 also accelerates ray intersection tests with this geometry to quickly determine which triangles are hit by any given ray. Determining ray-primitive intersections involves comparing the spatial xyz coordinates of the vertices of each primitive with the xyz coordinates of the ray to determine whether the ray and the surface the primitive defines occupy the same space. The ray-primitive intersection test can be computationally intensive because there may be many triangles to test. For example, in the mesh shown in FIG. 2G, the spout of the teapot alone is made up of over a hundred triangles–although it may be more efficient in some implementations to further volumetrically subdivide and thereby limit the number of triangles in any such “leaf node” to something like 16 or fewer.

[0102] As discussed above, ray tracing procedures determine what geometric primitives of a scene are intersected by a ray. However, due to the large number of primitives in a 3D scene, it may not be efficient or feasible to test every geometric primitive for an intersection. Acceleration data structures, such as BVH, allow for quick determination as to which bounding volumes can be ignored, which bounding volumes may contain intersected geometric primitives, and which intersected geometric primitives matter for visualization and which do not.

[0103] Ray Intersection Testing

[0104] FIGS. 3A-3C illustrate ray tracing applied to the FIG. 2G bounding volume 208 including triangle mesh 320. FIG. 3A shows a ray 302 in a virtual space including bounding volumes 310 and 315. To determine whether the ray 302 intersects one or more triangles in the mesh 320, each triangle could be directly tested against the ray 302. But to accelerate the process (since the object could contain many thousands of triangles), the ray 302 is first tested against the bounding volumes 310 and 315. If the ray 302 does not intersect a bounding volume, then it does not intersect any triangles inside of the bounding volume and all triangles inside the bounding volume can be ignored for purposes of that ray. Because in FIG. 3A the ray 302 misses bounding volume 310, the triangles of mesh 320 within that bounding volume need not be tested for intersection. While bounding volume 315 is intersected by the ray 302, bounding volume 315 does not contain any geometry and so no further testing is required.

[0105] On the other hand, if a ray such as ray 304 shown in FIG. 3B intersects a bounding volume 310 that contains geometry, then the ray may or may not intersect the geometry inside of the bounding volume so further tests need to be performed on the geometry itself to find possible intersections. Because the rays 304, 306 in FIGS. 3B and 3C intersect a bounding volume 310 that contains geometry, further tests need to be performed to determine whether any (and which) of the primitives inside of the bounding volume are intersected. In FIG. 3B, further testing of the intersections with the primitives would indicate that even though the ray 304 passes through the bounding volume 310, it does not intersect any of the primitives the bounding volume encloses (alternatively, as mentioned above, bounding volume 310 could be further volumetrically subdivided so that a bounding volume intersection test could be used to reveal that the ray does not intersect any geometry or more specifically which primitives the ray may intersect).

[0106] FIG. 3C shows a situation in which the bounding volume 310 intersected by ray 306 and contains geometry that ray 306 intersects. Traversal coprocessor 138 tests the intersections between the ray 306 and the individual primitives to determine which primitives the ray intersects.

[0107] Ray Tracing Operations

[0108] FIG. 4 is a flowchart summarizing example ray tracing operations the traversal coprocessor 138 performs as described above in cooperation with SM(s) 132. The FIG. 4 operations are performed by traversal coprocessor 138 in cooperation with its interaction with an SM 132. The traversal coprocessor 138 may thus receive the identification of a ray from the SM 132 and traversal state enumerating one or more nodes in one or more BVH’s that the ray must traverse. The traversal coprocessor 138 determines which bounding volumes of a BVH data structure the ray intersects (the “ray-complet” test 512) and subsequently whether the ray intersects one or more primitives in the intersected bounding volumes and which triangles are intersected (the “ray-primitive test” 520). In example non-limiting embodiments, “complets” (compressed treelets) specify root or interior nodes (i.e., volumes) of the bounding volume hierarchy with children that are other complets or leaf nodes of a single type per complet.

[0109] First, the traversal coprocessor 138 inspects the traversal state of the ray. If a stack the traversal coprocessor 138 maintains for the ray is empty, then traversal is complete. If there is an entry on the top of the stack, the traversal co-processor 138 issues a request to the memory subsystem to retrieve that node. The traversal co-processor 138 then performs a bounding box test 512 to determine if a bounding volume of a BVH data structure is intersected by a particular ray the SM 132 specifies (step 512, 514). If the bounding box test determines that the bounding volume is not intersected by the ray (“No” in step 514), then there is no need to perform any further testing for visualization and the traversal coprocessor 138 can return this result to the requesting SM 132. This is because if a ray misses a bounding volume (as in FIG. 3A with respect to bounding volume 310), then the ray will miss all other smaller bounding volumes inside the bounding volume being tested and any primitives that bounding volume contains.

[0110] If the bounding box test performed by the traversal coprocessor 138 reveals that the bounding volume is intersected by the ray (“Yes” in Step 514), then the traversal coprocessor determines if the bounding volume can be subdivided into smaller bounding volumes (step 518). In one example embodiment, the traversal coprocessor 138 isn’t necessarily performing any subdivision itself. Rather, each node in the BVH has one or more children (where each child is a leaf or a branch in the BVH). For each child, there is a bounding volume and a pointer that leads to a branch or a leaf node. When a ray processes a node using traversal coprocessor 138, it is testing itself against the bounding volumes of the node’s children. The ray only pushes stack entries onto its stack for those branches or leaves whose representative bounding volumes were hit. When a ray fetches a node in the example embodiment, it doesn’t test against the bounding volume of the node–it tests against the bounding volumes of the node’s children. The traversal coprocessor 138 pushes nodes whose bounding volumes are hit by a ray onto the ray’s traversal stack in an order determined by ray configuration. For example, it is possible to push nodes onto the traversal stack in the order the nodes appear in memory, or in the order that they appear along the length of the ray, or in some other order. If there are further subdivisions of the bounding volume (“Yes” in step 518), then those further subdivisions of the bounding volume are accessed and the bounding box test is performed for each of the resulting subdivided bounding volumes to determine which subdivided bounding volumes are intersected by the ray and which are not. In this recursive process, some of the bounding volumes may be eliminated by test 514 while other bounding volumes may result in still further and further subdivisions being tested for intersection by traversal coprocessor 138 recursively applying steps 512-518.

[0111] Once the traversal coprocessor 138 determines that the bounding volumes intersected by the ray are leaf nodes (“No” in step 518), the traversal coprocessor performs a primitive (e.g., triangle) intersection test 520 to determine whether the ray intersects primitives in the intersected bounding volumes and which primitives the ray intersects. The traversal coprocessor 138 thus performs a depth-first traversal of intersected descendent branch nodes until leaf nodes are reached. The traversal coprocessor 138 processes the leaf nodes. If the leaf nodes are primitive ranges, the traversal coprocessor 138 tests them against the ray. If the leaf nodes are instance nodes, the traversal coprocessor 138 applies the instance transform. If the leaf nodes are item ranges, the traversal coprocessor 138 returns them to the requesting SM 132. In the example non-limiting embodiments, the SM 132 can command the traversal coprocessor 138 to perform different kinds of ray-primitive intersection tests and report different results depending on the operations coming from an application (or an software stack the application is running on) and relayed by the SM to the TTU. For example, the SM 132 can command the traversal coprocessor 138 to report the nearest visible primitive revealed by the intersection test, or to report all primitives the ray intersects irrespective of whether they are the nearest visible primitive. The SM 132 can use these different results for different kinds of visualization. Once the traversal coprocessor 138 is done processing the leaf nodes, there may be other branch nodes (pushed earlier onto the ray’s stack) to test.

[0112] Multiple Intersections

[0113] In more detail, as shown in FIG. 3C, any given ray may intersect multiple primitives within a bounding volume. Whether the ray intersection within a given primitive matters for visualization depends on the properties and position of that primitive as well as the visualization procedures the SM 132 is performing. For example, primitives can be opaque, transparent or partially transparent (i.e., translucent). Opaque primitives will block a ray from passing through the primitive because the eye cannot see through the primitive’s opaque surface. Transparent primitives will allow the ray to pass through (because the eye can see through the transparent primitive) but the situation may be more complex. For example, transparent primitives may have specular properties that cause some portion of the ray to reflect (think of reflection from a window pane) and the rest of the ray to pass through. Other transparent primitives are used to provide a surface onto which a texture is mapped. For example, each individual leaf of a tree may be modeled by a transparent primitive onto which an image of the leaf is texture mapped.

[0114] FIGS. 5A-5C illustrate some of these scenarios using an example of three triangles assumed to be in the same bounding volume and each intersected by a ray. FIG. 5A illustrates a ray directed towards these three triangles, with the first triangle the ray encounters relative to the viewpoint being opaque. Because the “front” (from the standpoint of the direction of the ray from the eye) intersected triangle is opaque, that triangle will block the ray so the ray will not reach the other triangles even through it spatially intersects them. In this example, the triangles “behind” the opaque triangle from the viewpoint can be ignored (culled) after the intersection of the opaque triangle is identified because the “front”, opaque triangle hides the other triangles from the user’s view along the ray. Culling is indicated by dotted lines in FIGS. 5A-5C. In this case, the traversal coprocessor 138 may only need to report the identification of the first, opaque triangle to the SM 132.

[0115] FIG. 5B illustrates a ray directed towards the same three triangles but now the nearest visible triangle is partially transparent rather than opaque. Because the nearest visible intersected triangle is at least partially transparent, the ray may pass through it to hit the opaque triangle behind it. In this case, the opaque triangle will be visible through the partially transparent triangle but will block the user’s view of the third triangle along the ray. Here, the traversal coprocessor 138 may report the identification of both front triangles to the SM 132 but not report the third, culled triangle even though the ray spatially intersects that third triangle. Order of discovery may matter here. In the case of an alpha and opaque triangle, if the opaque was found first, the traversal coprocessor 138 returns the opaque triangle to the SM 132 with traversal state that will resume testing at the alpha triangle. While there is an implication here that the alpha means transparent, it really means “return me to the SM 132 and let the SM determine how to handle it.” For example, an alpha triangle might be trimmed according to a texture or function so that portions of the triangle are cut away (i.e., absent, not transparent). The traversal coprocessor 138 does not know how the SM 132 will handle the alpha triangles (i.e., it does not handle transparent triangles differently from trimmed triangles). Thus, alpha triangles may or may not block or tint the light arriving from points beyond them along the ray, and in example embodiments, they require SM 132 intervention to handle/determine those things.

[0116] FIG. 5C illustrates a scenario in which the first two triangles the ray encounters are partially transparent. Because the first and second intersected triangles are at least partially transparent, the ray will pass through the first and second triangles to impinge upon the also-intersecting third opaque triangle. Because third intersected triangle is opaque, it will block the ray, and the ray will not impinge upon any other triangles behind the third triangle even though they may be spatially intersected by it. In this case, the traversal coprocessor 138 may report all three triangles to the SM 132 but need not report any further triangles behind the opaque triangle because the opaque triangle blocks the ray from reaching those additional triangles.

[0117] In some modes, however, the SM 132 may need to know the identities of all triangles the ray intersects irrespective of whether they are opaque or transparent. In those modes, the traversal coprocessor 138 can simply perform the intersection test and return the identities of all triangles the ray spatially intersects (in such modes, the traversal coprocessor will return the same intersection results for all three scenarios shown in FIGS. 5A-5C) and allow the SM 132 to sort it out–or in some cases command the traversal coprocessor 138 to do more tests on these same triangles.

[0118] As will be discussed in more detail below, when a ray intersects an opaque triangle, the traversal coprocessor 138 can in certain operations be programmed to reduce the length of the ray being tested to the location of the opaque triangle intersection so it will not report any triangles “behind” the intersected triangle. When a partially transparent triangle is determined to be intersected by a ray, the traversal coprocessor 138 will return a more complete list of triangles the ray impinges upon for purposes of visualization, and the requesting SM 132 may perform further processing to determine whether, based for example any texture or other properties of the triangle, the ray will be blocked, passed or partially passed and partially reflected. In example embodiments, the traversal coprocessor 138 does not have access to texture properties of triangles and so does not attempt to determine visualization with respect to those properties.

[0119] Textures or Other Surface Modifications

[0120] For example, FIGS. 6A and 6B show a transparent triangle 610 with a texture 615 of a leaf applied to the triangle. One could think of a triangle made of Plexiglas with a decal of a leaf applied to it. As shown in FIG. 6A, the ray 620 intersects the transparent triangle 610 at a point that is outside the applied texture 615. Because the ray 620 intersects the triangle outside the applied texture 615, the texture will not block the ray 620 and the ray will pass through the transparent triangle 610 without obstruction. This is like being able to see through the parts of the Plexiglas triangle that are not covered by the leaf decal. Note that in one example embodiment, the SM 132 makes the visibility determination since the traversal coprocessor 138 does not necessarily have access to information concerning the leaf decal. The traversal coprocessor 138 helps the SM 132 by returning to the SM the identification of the triangle that the ray intersects along with information concerning the properties of that triangle.

[0121] In FIG. 6B, the ray 630 intersects the transparent triangle where the texture 615 is applied. SM 132 will determine whether subsequent traversal by the traversal coprocessor 138 is necessary or not based on whether the texture 615 will block the ray 630 or allow the ray 630 to pass through. If the ray 630 is blocked by the texture 615, other triangles behind the transparent triangle 610, which may have otherwise been intersected by the ray 630, will be obstructed by the texture and not contribute to visualization along the ray. In the example non-limiting embodiments herein, the traversal coprocessor 138 does not have access to texture information and so it does not attempt to accelerate this determination. Traversal coprocessor 138 may for example return to the requesting SM 132 all intersections between the ray and the various triangles within the object, and the SM may then use the graphics primitive engine 134 to make further ray tracing visualization determinations. In other example embodiments, traversal coprocessor 138 could accelerate some or all of these tests by interacting with the texture mapping unit and other portions of the 3D graphics pipeline within graphics primitive engine 134 to make the necessary visualization determinations.

[0122] Coordinate Transforms

[0123] FIGS. 2A-3C involve only a single object, namely a teapot. Just as the room you are in right now contains multiple objects, most 3D scenes contain many objects. For example, a 3D scene containing a teapot will likely also contain a cup, a saucer, a milk pitcher, a spoon, a sugar bowl, etc. all sitting on a table. In 3D graphics, each of these objects is typically modelled independently. The graphics system 100 then uses commands from the processor 120 to put all the models together in desired positions, orientations and sizes into the common scene for purposes of visualization (just as you will set and arrange the table for serving tea). What this means is that the SM 132 may command traversal processor 138 to analyze the same ray with respect to multiple objects in the scene. However, the fact that each of these objects will be transformed in position, orientation and size when placed into the common scene is taken into account and accelerated by the traversal coprocessor 138. In non-limiting example embodiments, the transform from world-to-object space is stored in the world space BVH along with a world-space bounding box. The traversal coprocessor 138 accelerates the transform process by transforming the ray from world (scene) space into object space for purposes of performing the tests shown in FIG. 4. In particular, since the transformation of the geometry from object space into world (scene) space is computationally intensive, that transformation is left to the graphics pipeline graphics primitive engine 134 and/or raster engine 136 to perform as part of rasterization. The traversal coprocessor 138 instead transforms a given ray from world space to the coordinate system of each object defined by an acceleration data structure and performs its tests in object space.

[0124] FIGS. 7A and 7B illustrates how the traversal coprocessor 138 transforms the same ray into three different object spaces. FIG. 7A shows three objects on a table: a cup, a teapot and a pitcher. These three objects and a table comprise a scene, which exists in world space. A ray that also is defined in world space emanates from the viewpoint and intersects each of the three objects.

[0125] FIG. 7B shows each of the three objects as defined in object spaces. Each of these three objects is defined by a respective model that exists in a respective object space. The traversal coprocessor 138 in example non-limiting embodiments transforms the ray into the object space of each object before performing the intersection tests for that object. This “instance transform” saves the computational effort of transforming the geometry of each object and the associated volumetric subdivisions of the acceleration data structure from object space to world space for purposes of the traversal coprocessor 138 performing intersection tests.

[0126] The requesting SM 132 keeps track of which objects are in front of which other objects with respect to each individual ray and resolves visibility in cases where one object hides another object, casts a shadow on another object, and/or reflects light toward another object. The requesting SM 132 can use the traversal processor 138 to accelerate each of these tests.

[0127] Example Tree BVH Acceleration Data Structure

[0128] FIGS. 8A and 8B show a recursively-subdivided bounding volume of a 3D scene (FIG. 8A) and a corresponding tree data structure (FIG. 8B) that may be accessed by the traversal coprocessor 138 and used for hardware-accelerated operations performed by traversal coprocessor. The division of the bounding volumes may be represented in a hierarchical tree data structure with the large bounding volume shown in FIG. 2B represented by a parent node of the tree and the smaller bounding volumes represented by children nodes of the tree that are contained by the parent node. The smallest bounding volumes are represented as leaf nodes in the tree and identify one or more geometric primitives contained within these smallest bounding volumes.

[0129] The tree data structure may be stored in memory outside of the traversal coprocessor 138 and retrieved based on queries the SMs 132 issue to the traversal coprocessor 138. The tree data structure includes a plurality of nodes arranged in a hierarchy. The root nodes N1 of the tree structure correspond to bounding volume N1 enclosing all of the triangles O1-O8. The root node N1 may identify the vertices of the bounding volume N1 and children nodes of the root node.

[0130] In FIG. 8A, bounding volume N1 is subdivided into bounding volumes N2 and N3. Children nodes N2 and N3 of the tree structure of FIG. 8B correspond to and represent the bounding volumes N2 and N3 shown in FIG. 8A. The children nodes N2 and N3 in the tree data structure identify the vertices of respective bounding volumes N2 and N3 in space. Each of the bounding volumes N2 and N3 is further subdivided in this particular example. Bounding volume N2 is subdivided into contained bounding volumes N4 and N5. Bounding volume N3 is subdivided into contained bounding volumes N6 and N7. Bounding volume N7 include two bounding volumes N8 and N9. Bounding volume N8 includes the triangles O7 and O8, and bounding volume N9 includes leaf bounding volumes N10 and N11 as its child bounding volumes. Leaf bounding volume N10 includes a primitive range (e.g., triangle range) O10 and leaf bounding volume N11 includes an item range O9. Respective children nodes N4, N5, N6, N8, N10 and N11 of the FIG. 8B tree structure correspond to and represent the FIG. 8A bounding volumes N4, N5, N6, N8, N10 and N11 in space.

[0131] The FIG. 8B tree is only three to six levels deep so that volumes N4, N5, N6, N8, N10 and N11 constitute “leaf nodes”–that is, nodes in the tree that have no child nodes. FIG. 8A shows that each of leaf node bounding volumes N4, N5, N6, and N8, contains two triangles of the geometry in the scene. For example, volumetric subdivision N4 contains triangles O1 & O2; volumetric subdivision N5 contains triangles O3 & O4; volumetric subdivision N6 contains trials O5 & O6; and volumetric subdivision N8 contains triangles O7 & O8. The tree structure shown in FIG. 8B represents these leaf nodes N4, N5, N6, and N7 by associating them with the appropriate ones of triangles O1-O8 of the scene geometry. To access this scene geometry, the traversal coprocessor 138 traverses the tree data structure of FIG. 8B down to the leaf nodes. In general, different parts of the tree can and will have different depths and contain different numbers of triangles. Leaf nodes associated with volumetric subdivisions that contain no geometry need not be explicitly represented in the tree data structure (i.e., the tree is “trimmed”).

[0132] According to some embodiments, the subtree rooted at N7 may represent a set of bounding volumes or BVH that is defined in a different coordinate space than the bounding volumes corresponding to nodes N1-N3. When bounding volume N7 is in a different coordinate space from its parent bounding volume N3, an instance node N7’ which provides the ray transformation necessary to traverse the subtree rooted at N7, may connect the rest of the tree to the subtree rooted at N7. Instance node N7’ connects the bounding volume or BVH corresponding to nodes N1-N3, with the bounding volumes or BVH corresponding to nodes N7 etc. by defining the transformation from the coordinate space of N1-N3 (e.g., world space) to the coordinate space of N7 etc. (e.g., object space).

[0133] The Internal Structure and Operation of Traversal Coprocessor 138

[0134] FIG. 9 shows an example simplified block diagram of traversal coprocessor 138 including hardware configured to perform accelerated traversal operations as described above (a still more detailed implementation of this traversal coprocessor 138 is described below). Because the traversal coprocessor 138 shown in FIG. 9 is adapted to traverse tree-based acceleration data structures such as shown in FIGS. 8A, 8B, it may also be called a “tree traversal unit” or “TTU” 700 (the 700 reference number is used to refer to the more detailed non-limiting implementation of traversal coprocessor 138 shown in FIG. 1). Tree traversal operations may include, for example, determining whether a ray intersects bounding volumes and/or primitives of a tree data structure (e.g., a BVH tree), which tests may involve transforming the ray into object space.

[0135] The TTU 700 includes dedicated hardware to determine whether a ray intersects bounding volumes and dedicated hardware to determine whether a ray intersects primitives of the tree data structure. In some embodiments, the TTU 700 may perform a depth-first traversal of a bounding volume hierarchy using a short stack traversal with intersection testing of supported leaf node primitives and mid-traversal return of alpha primitives and unsupported leaf node primitives (items). The intersection of primitives will be discussed with reference to triangles, but other geometric primitives may also be used.

[0136] In more detail, TTU 700 includes an intersection management block 722, a ray management block 730 and a stack management block 740. Each of these blocks (and all of the other blocks in FIG. 9) may constitute dedicated hardware implemented by logic gates, registers, hardware-embedded lookup tables or other combinatorial logic, etc.

[0137] The ray management block 730 is responsible for managing information about and performing operations concerning a ray specified by an SM 132 to the ray management block. The stack management block 740 works in conjunction with traversal logic 712 to manage information about and perform operations related to traversal of a BVH acceleration data structure. Traversal logic 712 is directed by results of a ray-complet test block 710 that tests intersections between the ray indicated by the ray management block 730 and volumetric subdivisions represented by the BVH, using instance transforms as needed. The ray-complet test block 710 retrieves additional information concerning the BVH from memory 140 via an L0 complet cache 752 that is part of the TTU 700. The results of the ray-complet test block 710 informs the traversal logic 712 as to whether further recursive traversals are needed. The stack management block 740 maintains stacks to keep track of state information as the traversal logic 712 traverses from one level of the BVH to another, with the stack management block pushing items onto the stack as the traversal logic traverses deeper into the BVH and popping items from the stack as the traversal logic traverses upwards in the BVH. The stack management block 740 is able to provide state information (e.g., intermediate or final results) to the requesting SM 132 at any time the SM requests.

[0138] The intersection management block 722 manages information about and performs operations concerning intersections between rays and primitives, using instance transforms as needed. The ray-primitive test block 720 retrieves information concerning geometry from memory 140 on an as-needed basis via an L0 primitive cache 754 that is part of TTU 700. The intersection management block 722 is informed by results of intersection tests the ray-primitive test and transform block 720 performs. Thus, the ray-primitive test and transform block 720 provides intersection results to the intersection management block 722, which reports geometry hits and intersections to the requesting SM 132.

[0139] A Stack Management Unit 740 inspects the traversal state to determine what type of data needs to be retrieved and which data path (complet or primitive) will consume it. The intersections for the bounding volumes are determined in the ray-complet test path of the TTU 700 including one or more ray-complet test blocks 710 and one or more traversal logic blocks 712. A complet specifies root or interior nodes of a bounding volume. Thus, a complet may define one or more bounding volumes for the ray-complet test. The ray-complet test path of the TTU 700 identifies which bounding volumes are intersected by the ray. Bounding volumes intersected by the ray need to be further processed to determine if the primitives associated with the intersected bounding volumes are intersected. The intersections for the primitives are determined in the ray-primitive test path including one or more ray-primitive test and transform blocks 720 and one or more intersection management blocks 722.

[0140] The TTU 700 receives queries from one or more SMs 132 to perform tree traversal operations. The query may request whether a ray intersects bounding volumes and/or primitives in a BVH data structure. The query may identify a ray (e.g., origin, direction, and length of the ray) and a BVH data structure and traversal state (e.g., short stack) which includes one or more entries referencing nodes in one or more Bounding Volume Hierarchies that the ray is to visit. The query may also include information for how the ray is to handle specific types of intersections during traversal. The ray information may be stored in the ray management block 730. The stored ray information (e.g., ray length) may be updated based on the results of the ray-primitive test.

[0141] The TTU 700 may request the BVH data structure identified in the query to be retrieved from memory outside of the TTU 700. Retrieved portions of the BVH data structure may be cached in the level-zero (L0) cache 750 within the TTU 700 so the information is available for other time-coherent TTU operations, thereby reducing memory 140 accesses. Portions of the BVH data structure needed for the ray-complet test may be stored in a L0 complet cache 752 and portions of the BVH data structure needed for the ray-primitive test may be stored in an L0 primitive cache 754.

[0142] After the complet information needed for a requested traversal step is available in the complet cache 752, the ray-complet test block 710 determines bounding volumes intersected by the ray. In performing this test, the ray may be transformed from the coordinate space of the bounding volume hierarchy to a coordinate space defined relative to a complet. The ray is tested against the bounding boxes associated with the child nodes of the complet. In the example non-limiting embodiment, the ray is not tested against the complet’s own bounding box because (1) the TTU 700 previously tested the ray against a similar bounding box when it tested the parent bounding box child that referenced this complet, and (2) a purpose of the complet bounding box is to define a local coordinate system within which the child bounding boxes can be expressed in compressed form. If the ray intersects any of the child bounding boxes, the results are pushed to the traversal logic to determine the order that the corresponding child pointers will be pushed onto the traversal stack (further testing will likely require the traversal logic 712 to traverse down to the next level of the BVH). These steps are repeated recursively until intersected leaf nodes of the BVH are encountered

……
……
……

本文链接：https://patent.nweon.com/16648

Nvidia Patent | Watertight ray triangle intersection

您可能还喜欢...

分类目录

最新AR/VR行业分享

最新AR/VR论文

最新AR/VR行业招聘

Nvidia Patent | Watertight ray triangle intersection

您可能还喜欢...

Nvidia Patent | Saccadic Redirection for Virtual Reality Locomotion

Nvidia Patent | Method For Overlapping Memory Accesses

Nvidia Patent | System And Method For Foveated Image Generation Using An Optical Combiner

分类目录

最新AR/VR行业分享

最新AR/VR论文

最新AR/VR行业招聘