Nvidia’s RTX 2000 series of graphics cards brought along several hardware changes that enabled Ray Tracing support in games. The new Turing based graphics cards packed special cores in them which were dedicated to this process and were known as RT Cores. The purpose of the RT cores was to specifically handle all the graphical computation that was required to make real-time Ray Tracing possible in games. Nvidia also supplemented the cards with extra CUDA cores to increase the raw power of the cards, while also adding a new set of cores known as Tensor Cores. These cores were meant to aid in deep-learning and AI applications such as a new form of upscaling technique known as Deep Learning Super Sampling. We have already discussed Deep Learning Super Sampling or DLSS in detail in this article, where you can learn more about the AI-powered upscaling technique.
Ray Tracing is Not New
While at first glance it might seem like Ray Tracing is a new technology that is pioneered by Nvidia, the truth is actually far from it. Yes, Nvidia was the first company to implement support for real-time Ray Tracing in games, but that does not mean that Ray Tracing has not existed before the RTX series. You’ve probably been enjoying it without knowing it for years if you have watched any recent movie that features CGI Effects. The implementation in movies is admittedly a little different and much more intensive than the gaming version. Big budget productions have the luxury to be able to spend a large amount of money and time on rendering those scenes. Popular animated movies have been reported to have used about 1000 supercomputers to render the entire movie with Ray Tracing effects over a span of one month. Such large-scale rendering processes are of course not feasible or possible for the average gamer looking to play some games with some updated visuals, thus the Ray Tracing version that is present in modern games is quite a bit different in application. Still, Ray Tracing is a feature that is present in many areas of production outside of gaming, with movies being one of the more prominent ones. Productivity softwares used by professionals to work on graphically intensive scenes, such as Blender, also support Ray Tracing features. These computer graphics and rendering software use different levels of ray tracing application to produce photorealistic visuals in still renders and 3D animations.
What is Rasterization?
So why was it deemed necessary by Nvidia to implement such a complex process into traditional games? Is there any difference in the process of Ray Tracing in games to make it more optimized for the workload? To understand the mechanism behind Ray Tracing, first, we need to understand the mechanism by which the games are traditionally rendered. This will help us to understand why Ray Tracing is considered an improvement and a big leap forward in graphical fidelity. The technique that is currently used for rendering is known as “Rasterization”. In this technique, the game code directs the GPU to draw a 3D scene using polygons. These 2D shapes (mostly triangles) make up most of the visual elements that are displayed on the screen. After a scene is drawn, it gets translated or “rasterized” into individual pixels which then get processed by a dedicated shader. The shader adds colors, textures, and lighting effects on a per-pixel basis to produce a fully rendered frame. This technique needs to be repeated around 30-60 times a second to produce 30FPS or 60FPS visuals in games.
Limitations of Rasterization
While rasterization has been the default mode of rendering in games for quite some time now, the inherent process behind rasterization has some limitations. The main problem with rasterization is that this technique has a difficult time tracking exactly how the light in a scene should travel and interact with other elements of the scene. Rasterized rendering does not produce the same results as Ray Traced rendering when it comes to the lighting effects and overall illumination of a particular scene. Rasterized rendering can also sometimes produce somewhat inaccurate visuals in relation to lighting which can really damage the immersion in a particular game. This is why Ray Tracing is considered a superior form of rendering when it comes to graphical fidelity especially in relation to lighting.
What exactly is Ray Tracing?
Now that we have discussed the traditional form of rasterized rendering, let’s discuss the new application of real-time Ray Tracing in modern games. Ray Tracing is a rendering technique that creates an image based on virtual light and how that light source interacts with all the objects inside the virtual scene. Ray Tracing can create a far more life-like depiction of scenes that take advantage of the interaction of light with the objects inside the scene to give a sense of realism. In simple words, Ray Tracing is a technique that makes light behave in videogames as it does in real life.
The mechanism behind Ray Tracing
The mechanism behind Ray Tracing in games is inherently different from the other forms of Ray Tracing already found in other industries like movies. Instead of tracking all the millions of rays that come from each light source, consumer-grade ray tracing lessens the computational load by instead tracing a path from the camera that represents the user’s perspective, through a single pixel, then to whatever object is behind that pixel and then finally back to the light source of the scene in question. This technique of Ray Tracing can also produce multiple effects like absorption, reflection, refraction, and diffusion of light as determined by the object that was interacting with the light in the scene. The Ray Tracing algorithm can also take resultant rays into account so that any reflection effects or shadows are displayed accurately.
Different forms of Ray Tracing
Not all implementations of Ray Tracing are the same. The variety of games that support Ray Tracing each implement the feature in a somewhat different way. This is up to the developer of the game to increase or decrease the complexity of Ray Tracing in the game so that the game delivers the perfect balance of performance and visual quality. As of 2020, most games that support Ray Tracing usually only use Ray Tracing for one aspect of a scene as opposed to rendering the whole scene using Ray Tracing itself. It is possible, but the computational costs of full-scene Ray Tracing are astronomical as compared to the other approaches and thus not worth the effort at least right now. As of the time of writing, the different implementations of Ray Tracing currently being used in games are:
Shadows: The simplest and the least intensive Ray Tracing implementation is arguably related to the shadows. Here, Ray Tracing is used to perfectly render the shadows in a scene based on the origin of the light from the light source, and the position of the object itself. This technique is most notably used in “Shadow of the Tomb Raider” to produce a more detailed shadow map that responds to the changes in the environment around the objects that are producing the shadows. Most notably, the movement and angle of the light source can now incur the same changes in the resulting shadows as we observe in real life.Reflections: Reflections are quite a bit more computationally intensive to render using Ray Tracing, however, Ray Traced reflections look phenomenal in modern games and are probably the most noteworthy graphical improvement that can be obtained using Ray Tracing. Reflections use the source of the light in a scene to accurately render the reflections from reflective objects such as glass and water. One of the most popular games that use Ray Traced reflections is “Control”.Ambient Occlusion: This is also related to shadows and is more-or-less tied into the same basic process. Ambient Occlusion uses Ray Tracing to predict the angle and the intensity of shadows based on the position and placement of objects within a scene. When done right, Ambient Occlusion can add some amazing detail and realism to a game.Global Illumination: Probably the most computationally intensive form of Ray Tracing implementation in modern games, Global Illumination uses Ray Tracing to accurately depict the world lighting. This provides a much more realistic sense of lighting when it is turned on, but it also has a massive hit to performance due to the sheer amount of data being processed. “Metro Exodus” uses Ray Tracing to provide a much more realistic form of Global Illumination.Full Path Tracing: Finally, we are also seeing some games emerge that are fully path traced, which essentially means that everything is Ray Traced. Now granted, these games are somewhat simpler and smaller than the other games which were more or less AAA titles from big companies, but this does not mean that they don’t look impressive. In fact, some might argue that these games with full path tracing look better than all the other Ray Tracing implementations. “Minecraft RTX” and “Quake RTX” are two of the titles that are fully path traced available at the time of writing.
What do I need for Ray Tracing?
As mentioned before, Ray Tracing is a very computationally intensive task so it requires some decidedly high-end hardware to perform well. As of the time of writing, there are several graphics cards from both AMD and Nvidia which support hardware-accelerated Ray Tracing. Even the consoles from Sony and Microsoft support this feature. That expands the list of supported hardware a bit:
Nvidia GeForce RTX 2000 seriesNvidia GeForce RTX 3000 seriesAMD Radeon RX 6000 seriesMicrosoft Xbox Series XSony PlayStation 5
Bear in mind that if AMD handles Ray Tracing a bit differently than Nvidia, so there is a somewhat bigger performance penalty that is observed when you are using AMD cards for Ray Tracing. Also, if you want to experience the improved performance using Deep Learning Super Sampling, that feature is also only available on Nvidia’s RTX cards. AMD is supposedly working on a DLSS-like feature for their RX 6000 series cards, but currently, it is still under development at the time of writing. Nvidia has also coined the term “Giga Rays” to give users an idea about the relative Ray Tracing capabilities of its RTX Graphics Cards. Nvidia says that 5 Giga Rays per second is the minimum amount of virtual light ideally required to fully illuminate a typical room in a video game environment. The GeForce RTX 2070 offers 5 Giga Rays/sec, while the RTX 2080 offers 8 Giga Rays per second. The RTX 2080Ti offers a whopping 10 Giga Rays/sec. It is a somewhat arbitrary unit though so it should only be used generally to show relative performance expectations.
Performance Loss and DLSS
As is evident by now, the biggest disadvantage of Ray Tracing is the hit to performance due to the large amount of dedicated computation that needs to be done in the process. In some games, the performance hit is so large that it can take the game to a framerate which is no longer considered playable. The performance hit is even larger in games that use more complex implementations of Ray Tracing such as Reflections, Global Illumination, or Full Path Tracing. Of course, Nvidia thought of this performance penalty situation, and also developed a new compensation technique known as Deep Learning Super Sampling. This technique called DLSS was released alongside Nvidia’s RTX 2000 series back in 2018. We have already explored DLSS in detail in this article, but the essence of this technology is that it renders the image at a lower resolution and then smartly and methodically upscales the image to match the output resolution to provide vastly superior performance to native rendering. DLSS is an excellent compensation mechanism for the performance loss of Ray Tracing, but it can also be used without Ray Tracing to provide even higher framerates and a much better experience. The biggest advantage of DLSS is that it uses Deep Learning and AI to upscale the image so that there is little-to-no visual clarity difference between the native and the upscaled image. Nvidia uses the Tensor cores on its RTX series of cards to accelerate the DLSS process so that this upscaling computation can be done at the pace of the game that is being rendered. This is a truly exciting technology that we would like to see develop further and get better than it is now.
Future of Ray Tracing
Ray Tracing in games is just beginning and we can say for sure that it is here to stay. AMD has just released their first lineup of cards that support full real-time Ray Tracing with the RX 6000 series, and the PlayStation 5 and Xbox Series X also have support for Ray Tracing. The current hurdles that need to be overcome include the performance loss and the low number of games that support it. The current games that support Ray Tracing at the time of writing include:
Amid EvilBattlefield VBright MemoryCall of Duty: Modern Warfare (2019)Call Of Duty: Black Ops Cold WarControlCrysis RemasteredDeliver Us The MoonFortniteGhostrunnerJusticeMechwarrior V: MercenariesMetro ExodusMinecraftMoonlight BladePumpkin JackQuake II RTXShadow of the Tomb RaiderStay in the LightWatch Dogs LegionWolfenstein: Youngblood
Meanwhile, Nvidia has confirmed that the following titles will also support Ray Tracing once they come out:
Atomic HeartCyberpunk 2077 (launch)Dying Light 2Doom EternalEnlisted (November closed beta)JX3Mortal Shell (November)Observer: System ReduxReady Or Not (early access launch)Ring Of Elysium (launch)Synced: Off-PlanetThe Witcher IIIVampire: The Masquerade – Bloodlines 2World Of Warcraft: Shadowlands (November)Xuan-Yuan Sword VII (launch)
While these may not seem like many games, it represents a start towards a direction where the predominant form of rendering could very well be Ray Tracing. Now as far as performance goes, it is really hard to predict whether or not the performance hit from Ray Tracing will be lowered a bit. What is reasonable to expect though, is for DLSS to get better and offer sufficient compensation for the performance loss that is incurred by turning Ray Tracing on. As of the time of writing, the list of games that support DLSS is not expansive by any means, but it is a good start keeping in mind that Nvidia has announced DLSS support for several upcoming games as well. Here are all the games that currently support Deep Learning Super Sampling:
AnthemBattlefield VBright MemoryCall Of Duty: Black Ops Cold WarControlDeath StrandingDeliver Us The MoonF1 2020Final Fantasy XVFortniteGhostrunnerJusticeMarvel’s AvengersMechwarrior V: MercenariesMetro ExodusMinecraftMonster Hunter: WorldShadow of the Tomb RaiderWatch Dogs LegionWolfenstein Youngblood
As you might have noticed, most of the games that support DLSS are titles that have some form of Ray Tracing support as well. This offers further confirmation to the theory that DLSS has been developed and released mainly as a compensation technology to alleviate the immense loss of performance in Ray Tracing. DLSS is a seriously impressive technology though, as Nvidia has explained that it uses a supercomputer to perform complex calculations which trains the algorithm that the Tensor cores inside the Nvidia GPUs follow. Just like Ray Tracing, DLSS is expected to be coming to more games as well:
Amid EvilAtomic HeartBoundaryCyberpunk 2077 (launch)Edge Of Eternity (November)JX3Mortal Shell (November)Mount & Blade II Bannerlord (November)Ready Or Not (early access launch)ScavengersVampire: The Masquerade – Bloodlines 2Xuan-Yuan Sword VII (launch)
DLSS, combined with Ray Tracing seems to be the future of the gaming industry as of 2020.
Conclusion
Rasterization is the technique that has been used to convert a 2D plane of polygons into a 3D image on the screen in games for a long time now. In 2018, Nvidia introduced the RTX 2000 series of graphics cards with full support for real-time Ray Tracing in games, a technique that uses complex computations to trace the rays of light in a scene to create accurate depictions of how the light would interact with the objects in a scene. This took the world of gaming by an unexpected storm and the whole industry put Ray Tracing as their primary focus going forward. As of the time of writing, Nvidia has released another generation of graphics cards that further improve their Ray Tracing performance while both AMD and the consoles have also announced full support for the feature. Nvidia has also improved their Deep Learning Super Sampling technique which uses AI and Deep Learning to smartly upscale the image which was rendered at a lower resolution in order to compensate for the performance loss due to Ray Tracing. It seems like Ray Tracing is here to stay, and while the initial number of titles that support the feature is not expansive, more and more titles are being announced which have complete support for real-time Ray Tracing going forward. It is now up to the developers to fine-tune the Ray Tracing features in their upcoming games, and also to increase the number of titles that support this feature. Nvidia and AMD also have a responsibility to make their hardware optimized for this feature so that gamers would not have to experience devastating performance loss whenever they want to turn Ray Tracing on.
AMD’s Next-Gen RDNA 3 GPUs to Feature Enhanced Ray-Tracing Capabilities Using A…Innosilicon’s Future GPU Lineup Set to Include Ray Tracing and MoreIntel Arc A770 Tested In Blender Cycles With Ray TracingSkull and Bones PC Features Trailer Reveals Support for DLSS, Ray Tracing, FSR…
