Reading list on ReSTIR

Recently a short video from dark magic programmer Tomasz Stachowiak made the rounds in the graphics programming community, at the sound of jaws hitting the floor in its wake. It shows his recent progress on in his renderer pet project: beautiful real-time global illumination with fast convergence and barely any noise, in a static environment with dynamic lighting.

In a Twitter thread where he discussed some details, one keyword in particular caught my attention: ReSTIR.

ReSTIR stands for “Reservoir-based Spatio-Temporal Importance Resampling” and is a sampling technique published at SIGGRAPH 2020 and getting refined since.

The original publication

Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting
The publication page includes the recording of the SIGGRAPH presentation, with a well articulated explanation of the technique by main author Benedikt Bitterli.
(same publication hosted on the NVidia website).

Explanations of ReSTIR

Improvements over the original publication

After the initial publication, NVidia published a refined version producing images with less noise at a lower cost, which they call “RTXDI” (for RTX Direct Illumination).

Other limitations

When discussing on Twitter some of the limitations of ReSTIR, Chris Wyman made the following remarks:

To be clear, right now, ReSTIR is a box of razor blades without handles (or a box of unlabeled knobs). It’s extremely powerful, but you have to know what you’re doing. It is not intuitive, if your existing perspective is traditional Monte Carlo (or real-time) sampling techniques.

People sometimes think SIGGRAPH paper = solved. Nope. We’ve learned a lot since the first paper, and our direct lighting is a lot more stable with that knowledge. We’re still learning how to do it well on full-length paths.

And there’s a bunch of edge cases, even in direct lighting, that we know how to solve but haven’t had time to write them up, polish, and demo.

We haven’t actually tried to solve the extra noise at disocclusions in (what I think of as) a very principled way. Right now a world-space structure is probably the best way. I’m pretty sure it can be done without a (formal) world-space structure, just “more ReSTIR.”

GDC 2015 presentations

The Game Developers Conference took place last week in San Francisco. As I am starting to see more speakers publish their slides, I am creating this post to keep track of some them (this list is not meant to be exhaustive).

For a more extensive list, Cédric Guillemet has been garnering links to GDC 2015 papers on his blog.

Statistics of the gaming hardware and software

If you need figures or maybe just an idea of what hardware and software the gaming public is using, it’s all out there, gathered and published by the big boys:

FaceWorks demonstration at GTC

Geeks3D mentioned this keynote at the GPU Technology Conference, where NVIDIA’s CEO shows their technology called FaceWorks. After talking about the uncanny valley and avatar rendering uses, came the live demonstration, which seriously raises the bar in terms of face rendering and animation.

The quality is incredible, the gap from photo realism is getting very narrow, and some expressions are really convincing. The transitions and frozen expressions feel weird though, so I am wondering how it would look running freely for a moment, with all the rapid subtle moves we show even when staying idle. The avatar as a mean of communication is certainly very appealing. It would be interesting to see if when facing this rendering, we would react to the expressions displayed.

The demo itself starts after 8 minutes.