Learn how ray tracing impacts game development by elevating the photorealism of a scene
Have you ever wondered how lighting works in the latest games? Thanks to the famous RTX, today we have more immersion when playing, if we have access to this technology.
Nvidia, along with Ray Tracing technology, has given new appreciative contours to gamers who own a branded GPU. But lighting implementations, which are so much discussed these days, have been present in games for a long time. Of course, with the advent of RTX, everything became more evident and curiosity about the subject grew.
That’s why Nvidia decided to talk a little more about how lighting is used to enrich the visuals of the games we dive into.
What is direct and indirect lighting?
Light falling on an object is called direct illumination.
It determines the color and amount of light that hits a surface from one light source, but ignores all light that might reach the surface from any other source, such as reflection or refraction. Direct lighting also determines the amount of light that is absorbed and reflected by the surface itself.
Light reflected off a surface, illuminating other objects is called indirect lighting. It gets to the surfaces of everything except light sources. In other words, indirect lighting determines the color and amount of all other light reaching a surface. Most commonly, indirect light is reflected from one surface to other surfaces.
Indirect lighting generally tends to be more difficult and expensive to calculate than direct lighting. This is because there are substantially more paths that light can take between the light emitter and the observer.
What is Global Illumination?
Global lighting is the process of calculating the color and amount of all light – direct and indirect – that is on visible surfaces in a scene.
Accurately simulating all types of indirect light is extremely difficult, especially if the scene includes complex materials like glass, water and shiny metals – or if the scene has scattering in clouds, smoke, fog or other elements known as volumetric media.
As a result, real-time graphics solutions for global illumination are typically limited to calculating a subset of indirect light – usually for surfaces with diffuse materials (also known as mattes).
How is direct and indirect lighting calculated?
Many algorithms can be used to calculate direct lighting, all with strengths and weaknesses. For example, if the scene has a single light and no shadows, direct lighting is trivial to calculate, but it won’t look very realistic. On the other hand, when a scene has multiple light sources, processing them all for each surface can become difficult.
To solve these problems, optimized algorithms and shading techniques have been developed, such as deferred shading or clustered shading. These algorithms reduce the number of surface and light interactions to be computed.
Shadows can be added using a variety of techniques, including shadow maps, stencil shadow volumes, and ray tracing.
Shadow mapping has two steps. First, the scene is rendered from the light point of view on a special texture called a shadow map. Then the shadow map is used to test whether the surfaces visible on the screen are also visible from a light point of view. Shadow maps come with many limitations and artifacts, and they quickly become expensive as the number of lights in the scene increases.
Stencil shadow volumes are based on extruding scene geometry away from the light and rendering that extruded geometry into the stencil buffer. The content of the stencil buffer is used to determine whether a given surface on the screen is in shadow or not. Stencil shadows are always sharp, unnaturally, but don’t suffer from common shadow map issues.
Until the introduction of NVIDIA RTX technology, ray tracing was very expensive to use when calculating shadows. Ray tracing is a rendering method that simulates the physical behavior of light. Tracing rays from a surface on the screen to a light allows for the calculation of shadows, but this can be challenging if the light comes from a point. Also, ray tracing shadows can get expensive quickly if there are too many lights in the scene.
More efficient sampling methods have been developed to reduce the number of rays needed to calculate soft shadows from various lights. One example is an algorithm called ReSTIR, which calculates the direct lighting of millions of Ray Traced lights and shadows at interactive frame rates.
What is Path Tracing?
For indirect lighting and global lighting, there are even more methods. The most straightforward is called Path Tracing, where random light paths are simulated for each visible surface. Some of these paths reach lights and contribute to the finished scene, while others do not.
Path Tracing is the most accurate method capable of producing results that fully represent the lighting in a scene, matching the accuracy of mathematical models for materials and lights. Path Tracing can be very expensive to calculate, but it is considered the “holy grail” of real-time graphing.
How does direct and indirect lighting affect graphics?
Direct lighting provides the basic look of realism and indirect lighting makes scenes richer and more natural.
One way that indirect lighting has been used in many video games is through the ubiquitous ambient light. This type of light can be constant or vary spatially over light probes arranged in a grid pattern. It can also be rendered into a texture that wraps static objects in a scene – this method is known as a “light map”.
In most cases, ambient light is shaded by a function of the geometry around the surface called ambient occlusion, which helps to increase the realism of the image.
Examples of direct lighting, indirect lighting and global lighting
Direct and indirect lighting have been present, in some form, in almost every 3D game since the 1990s. Below are some milestones of how lighting has been implemented in popular titles:
- 1993: DOOM presented one of the first examples of dynamic lighting. The game could vary the light intensity by sector, which made textures lighter or darker, and was used to simulate dark and bright areas or flashing lights.
- 1995: Quake introduced light maps, which were pre-computed for every level in the game. Light maps can modulate ambient light intensity.
- 1997: Quake II added color to light maps, as well as dynamic lighting from projectiles and explosions.
- 2001: Silent Hill 2 featured per-pixel lighting and shadow mapping. Shrek used deferred lighting and stencil shadows.
- 2007: Crysis showed dynamic screen-space ambient occlusion, which uses pixel depth to give a sense of changing lighting.
- 2008: Quake Wars: Ray Traced became the first gaming tech demo to use Ray Traced-based reflections.
- 2011: Crysis 2 became the first game to include screen space reflections, which is a very popular technique to this day, which reuses data present in screen space to calculate reflections.
- 2016: Rise of the Tomb Raider became the first game to use voxel-based ambient occlusion.
- 2018: Battlefield V became the first commercial game to use Ray Tracing-based reflections.
- 2019: Q2VKPT became the first game to implement Path Tracing, which was later refined in Quake II RTX.
- 2020: Minecraft with RTX is the first commercial game to use Path Tracing.
What’s next for real-time graphics lighting?
Real-time graphics are moving towards more complete simulation of light in scenes with increasing complexity.
ReSTIR dramatically expands the possibilities for artists to use multiple lights in games. Its latest variant, ReSTIR GI, applies the same ideas regarding global illumination, allowing Path Tracing with more jumps and fewer zooms. It can also render images with less noise faster. More algorithms are being developed to make Path Tracing faster and more accessible.
Using a full simulation of Ray Traced lighting effects also means that the rendered images may contain some noise. Eliminating this noise, or “denoising”, is another area of active research.
More technologies are being developed to help games effectively eliminate lighting noise in complex, highly detailed scenes with lots of motion at real-time frame rates. This challenge is being approached from two ends: advanced sampling algorithms, which generate less noise, and advanced denoisers that can handle increasingly difficult situations, such as the NRD, NVIDIA Real-Time Denoiser.
All information about lighting and RTX was released by NVidia with the intention of making clear the aspects and particularities of these technologies and how they interact with the development of the game. A lot of research is ongoing and technology continues to evolve, which can bring spectacular innovations to games.
A monster RTX 3060: is it worth the more expensive model?
Let’s test the performance of ASUS ROG STRIX Gaming OC