Path tracing is a surprisingly simple technique to render realistic images. This would be my definition if you are unfamiliar with the term. But if you already have experience with various ray tracing techniques, I would probably say that path tracing is a remarkably elegant solution to the rendering equation. You can implement a toy path tracer in a weekend or, if you’ve already done it a few times before, within 25 minutes.

Recently I was documenting myself on path tracing, and some of the techniques that can be used, like next event estimation, bidirectional path tracing, Russian roulette, etc. This is a case where ShaderToy can be an invaluable source of examples and information, and so I was browsing path tracing shaders there. As the number of open tabs was starting to get impractical, I decided to use the “playlist” feature of ShaderToy to bookmark them all.

The examples of path tracers listed include very naive implementations, hacky ones, rendering features like advanced BRDF, volumetric lighting or spectral rendering, or various noise reduction techniques such as next event estimation, bidirectional path tracing, multiple importance sampling, accumulation over frames with temporal reprojection, screen space blue noise, or convolutional neural network based denoising.

Some of those shaders are meant to be artworks, but even the technical experimentation ones look nice, because the global illumination inherent to path tracing tends to generate images that are pretty.

About a year ago at SIGGRAPH Asia 2021 (which took place as a hybrid conference both online and on site at the Tokyo International Forum) one of the technical papers that caught my attention was the publication by Šárka Sochorová and Ondřej Jamriška on color mixing.

Color mixing in most digital painting tools is infamously unsatisfying, often limited to a linear interpolation in RBG space, resulting in unpleasing gradients very different from what one would expect. Ten years ago I mentioned this article that presented the color mixing of the application Paper, which tried to solve this very problem.

This time, the core idea is to model colors as pigments: estimate the pigment concentration based on the color, so in a way, move from RGB space to “pigment space”, and interpolate the pigment concentration, before converting back to RGB space.

The paper uses the Kubelka-Munk model for estimating colors from pigment concentration. The problem however is to find a transformation between the two spaces. A first assumption is made on the available pigments: essentially restricting them to CMYK. Then two problems are addressed: RGB colors that cannot be represented with those pigments, and likewise pigment colors that cannot be represented in RGB. The paper proposes a remapping that enables a transform and its inverse, thus allowing to move from RGB space to pigment space, interpolate in pigment space, and move back to RGB space.

You could argue this is therefore a physically based diffuse color mixing.

Soon after showcasing his recent rendering results which left industry veterans impressed and causing many of us to start documenting ourselves about ReSTIR, professional madman Tomasz Stachowiak showed a new demonstration of the global illumination capabilities of his pet project.

But more importantly, he took the time to describe the techniques used to get such results. The writing is fairly high level, and assumes the reader to be familiar with several advanced topics, but it comes with clear illustrations at least for some parts. It also mentions the various ways in which ReSTIR is leveraged to support the techniques used. Finally, it doesn’t try to hide the parts where the techniques fall short, quite the opposite.

In very brief, the rendering combines a geometry pass, from which a ReSTIR pass is done to compute the first bounce rays, in combination with a sparse voxel grid based irradiance cache for the rest of the light paths, which also relies on ReSTIR, and a few clever tricks to handle various corner cases, as well as denoising and temporal anti-aliasing to smooth things out.

Recently a short video from dark magic programmer Tomasz Stachowiak made the rounds in the graphics programming community, at the sound of jaws hitting the floor in its wake. It shows his recent progress on in his renderer pet project: beautiful real-time global illumination with fast convergence and barely any noise, in a static environment with dynamic lighting.

ReSTIR stands for “Reservoir-based Spatio-Temporal Importance Resampling” and is a sampling technique published at SIGGRAPH 2020 and getting refined since.

Reframing light transport for real-time (video, slides) This keynote given at HPG 2020 by Chris Wyman, who is a co-author of ReSTIR, gives another perspective on the technique, through the prism of using statistics to evaluate an unknown distribution.

Improvements over the original publication

After the initial publication, NVidia published a refined version producing images with less noise at a lower cost, which they call “RTXDI” (for RTX Direct Illumination).

Rearchitecting Spatiotemporal Resampling for Production (video, slides) Both presentations explain the same thing, but with small differences that sometimes are clearer in one or the other. They explain again the foundations of the technique, then detail where the improvements lie (use fewer more relevant samples, avoid wasting work, and using a more cache friendly approach).

To be clear, right now, ReSTIR is a box of razor blades without handles (or a box of unlabeled knobs). It’s extremely powerful, but you have to know what you’re doing. It is not intuitive, if your existing perspective is traditional Monte Carlo (or real-time) sampling techniques.

People sometimes think SIGGRAPH paper = solved. Nope. We’ve learned a lot since the first paper, and our direct lighting is a lot more stable with that knowledge. We’re still learning how to do it well on full-length paths.

And there’s a bunch of edge cases, even in direct lighting, that we know how to solve but haven’t had time to write them up, polish, and demo.

We haven’t actually tried to solve the extra noise at disocclusions in (what I think of as) a very principled way. Right now a world-space structure is probably the best way. I’m pretty sure it can be done without a (formal) world-space structure, just “more ReSTIR.”

I previously showed the derivation of how to determine the intersection of a plane and a cone. At the time I had to solve that equation, so after doing so I decided to publish it for anyone to use. Given the positive feedback, it seems this was useful, so I might as well continue with a few more.

Here is probably the most basic intersection: a ray and a plane. Solving it is straightforward, which I hope can be seen below. Like last time, I am using vector notation.

We define a ray with its origin $O$ and its direction as a unit vector $\hat{D}$. Any point $X$ on the ray at a signed distance $t$ from the origin of the ray verifies: $\vec{X} = \vec{O} + t\vec{D}$. When $t$ is positive $X$ is in the direction of the ray, and when $t$ is negative $X$ is in the opposite direction.

We define a plane with a point $S$ on that plane and the normal unit vector $\hat{N}$, perpendicular to the plane. The distance between any point $X$ and the plane is $d = \lvert (\vec{X} – \vec{S}) \cdot \vec{N} \rvert$. If this equality is not obvious for you, you can think of it as the distance between $X$ and $S$ along the $\vec{N}$ direction. When $d=0$, it means $X$ is on the plane itself.

We define $P$ the intersection or the ray and the plane, and which we are interested in finding.

Since $P$ is both on the ray and on the plane, we can write: $$ \left\{ \begin{array}{l} \vec{P}=\vec{O} + t\vec{D} \\ \lvert (\vec{P} – \vec{S}) \cdot \vec{N} \rvert = 0 \end{array} \right. $$ Because the distance $d$ from the plane is $0$, the absolute value is irrelevant here. We can just write: $$ \left\{ \begin{array}{l} \vec{P}=\vec{O} + t\vec{D} \\ (\vec{P} – \vec{S}) \cdot \vec{N} = 0 \end{array} \right. $$ All we have to do is replace $P$ with $\vec{O} + t\vec{D}$ in the second equation, and reorder the terms to get $t$ on one side. $$ (\vec{O} + t\vec{D} – \vec{S}) \cdot \vec{N} = 0 $$ $$ \vec{O} \cdot \vec{N} + t\vec{D} \cdot \vec{N} – \vec{S} \cdot \vec{N} = 0 $$ $$ t\vec{D} \cdot \vec{N} = \vec{S} \cdot \vec{N} – \vec{O} \cdot \vec{N} $$ $$ t = \frac{(\vec{S} – \vec{O}) \cdot \vec{N}}{ \vec{D} \cdot \vec{N} } $$

A question to ask ourselves is: what about the division by $0$? Looking at the diagram, we can see that $\vec{D} \cdot \vec{N} = 0$ means the ray is parallel to the plane, and there is no solution unless $O$ is already on the plane. Otherwise, the ray intersects the plane for the value of $t$ written above. That’s it, we’re done.

Note: There are several, equivalent, ways of representing a plane. If your plane is not defined by a point $S$ and a normal vector $\hat{N}$, but rather with a distance to the origin $s$ and a normal vector $\hat{N}$, you can notice that $s = \vec{S} \cdot \vec{N}$ and simplify the result above, which becomes: $$ t = \frac{s – \vec{O} \cdot \vec{N}}{ \vec{D} \cdot \vec{N} } $$

Signed distance to a plane

For the sake of simplicity, in the above we defined the distance to the plane as an absolute value. It is possible however to define it as a signed value: $d = (\vec{X} – \vec{S}) \cdot \vec{N}$. In this case $d>0$ means $X$ is somewhere on the side of the plane pointed by $\vec{N}$, while $d<0$ means $X$ is on the opposite side of the plane.

Distances that can be negative are called signed distances, and they are a foundation of Signed Distance Fields (SDF).

Over the last few months I have been trying to push my understanding of Physically Based Shading, by actively exploring every corner and turning over every stone, to uncover any area where I lack knowledge. Although this is still an ongoing process and I still have a lot to do, I thought I could already share some of what I have learned in the process.

Last weekend the Easter demoparty event Revision took place, as an online version due to the current pandemic situation. There, I presented a talk on Physically Based Shading, in which I went into electromagnetism, existing models, and an brief overview of a prototype I am working on.

The presentation goes into a lot of detail about interaction of light with matter from a physics point of view, then builds its way up to the Cook-Torrance specular BRDF model. The diffuse BRDF and the Image Based Lighting were skipped due to time constraints. I am considering doing a Part 2 to address those topics, but I haven’t decided anything yet.

In the mean time, please leave a comment or contact me if you notice any mistake or inaccuracy.

Abstract

How do you implement a Physically Based Shading for your demos yet keep the possibility to try something completely different without having to rewrite everything? In this talk we will first get an intuitive understanding of what makes matter look the way it looks, with as much detail as we can given the time we have. We will then see how this is modeled by a BRDF (Bidirectional Reflectance Distribution Function) and review some of the available models. We will also see what makes it challenging for design and for real-time implementation. Finally we will discuss a possible implementation that allows to experiment with different models, can work in a variety of cases, and remains compatible with size coding constraints.

And finally here is the recording of the talk, including a quick demonstration of the prototype:

Interference shader

Here is the shader used during the presentation to illustrate light interaction at the interface between to media:

Acknowledgements

Thanks again to Alan Wolfe for reviewing the text, Alkama for the motivation and questions upfront and help in the video department, Scoup and the Revision crew for organizing the seminars, Ronny and Siana for the help in the sound department, and everyone who provided feedback on my previous article on Physically Based Shading.

FWIW – I think the model of refraction by the electromagnetic field causing electrons to oscillate is the better one. This explains not only refraction but reflection as well, and even total internal reflection. Feynman does out the wave calculations: https://feynmanlectures.caltech.edu/II_33.html

It also explains better IMO why a light wave keeps its direction in a material. If an atom absorbs and re-emits the photon there is no reason why it should be going in the same direction as before (conservation of momentum is maintained if the atom recoils). Besides which, the lifetime of an excited atomic state is many orders of magnitude longer than the time needed for a light wave to propagate across the diameter of the atom (even at an IOR-reduced speed).

Moreover, in the comments of the shader above, CG researcher Fabrice Neyret mentioned a presentation of his from 2019, which lists interactions of light with matter: Colors of the universe. Quoting his summarized comment:

In short: the notion of photons (and their speed) in matter is a macroscopic deceiving representation, since it’s about interference between incident and reactive fields (reemitted by the dipoles, at least for dielectrics).

The first four videos show how to use boundary conditions to deduce the relationship between the electromagnetic field on both sides of a surface (or interface between two different media).

The rest of the series then dives into other topics like thin film interference.

The series assumes the viewer to be already familiar with the Maxwell equations, so it can be helpful to first see the explanation by Grant Sanderson of 3Blue1Brown on Maxwell’s equations.