COMBINING REPROJECTION AND ADAPTIVE SAMPLING FOR REAL-TIME PATH TRACING ON THE GPU
By Olaf Schalk
Supervisors:
Dr. Ing. J. Bikker, H. van Summeren and Prof. Dr. M.J. van Kreveld.
Abstract
Real-time path tracing becomes more and more realistic and is usable for all kinds of graphical applications. Still most implementations of a path tracer tend to see each pixel as equal and they discard all the received results when there is a movement or change in the scene. With our adaptive sampling method, we show that each pixel can be treated differently, based on the variance per pixel received by the path tracer. By using a probability, which indicates how complex a pixel is compared to others, we distribute samples across all the pixels in the screen. Our reprojection method can reuse pixels from previous frames. Not all pixels can be reused, because of material properties, which are dependent on the view position. Therefore, we created an error function, based on the material property and the incoming light energy, which can determine if a reprojection will exceed a specified maximum error. By combining both the methods we can give more priority to pixels with fewer samples and to pixels that are more complex. We demonstrate that our methods can improve a default implementation of a path tracer within the first milliseconds, which is an advantage for real-time applications.
Results
We tested our adaptive sampling method in combination with our reprojection method against a default path tracer. Both methods are build on top of the default path tracer, which contains some variance reduction techniques. Our tests are done in three different scenes who are well known in graphical research:
- Cornell box scene
- Sponza scene
- San Miguel scene
The image below shows a reference image of the Cornell box scene, which has taken 4096 samples per pixel. The reference images with the output of our methods are used to calculate a mean squared error. A mean squared error is a measurement to see how far the output deviates from the reference image.
The graph below shows that the combination and reprojection method do not give very different results. In the figure underneath the graph, an image comparison after 100 milliseconds is visualized. Here is a small noise reduction visible between the combination and reprojection method on the right border of the image. In this part of the image, a new part of the scene is introduced that does not have any samples yet that can be reprojected. The combination method uses adaptive sampling to focus on such parts of an image.
For the first 700 milliseconds the combination method seems to give a better result instead of the reprojection method, within the Sponza scene as we can see below the reference image in the result graph. This is possible, because the Sponza scene is more complex and needs more time per frame, where the adaptive sampling in the combination method gives a reduction in frame time, by choosing pixels that need more samples.
In the comparison images of the Sponza scene in the figure below, which are taken after 200 milliseconds, we see a noise reduction between the default path tracer and the reprojection method. However, when we compare the reprojection method with the combination method, an improvement is hardly visible. Only above the blue curtain on the right, a small noise reduction can be noticed.
The comparison of the San Miguel scene gives the same results as in the Sponza scene, as we can see in the graph below the reference image of the San Miguel scene. Here the combination method improves more than the reprojection method. However, this is only for a short time, which is the same on all other results on the San Miguel scene.
In the next figure, the improvement is hardly visible between the combination method and reprojection method. Still the dark areas get a reduction in noise compared to the default path tracer.
Conclusion
The goal of the thesis is to improve the efficiency of a real-time path tracer using reprojection and/or adaptive sampling, without exceeding a specified maximum error. Within the same amount of time as the default path tracer, which uses variance reduction techniques, the reprojection method improves the efficiency after a movement occurs in the first milliseconds. Here we chose a maximum reprojection error of 1\256, because it is the smallest integer value difference that can be visualized on most of the standard monitors. We calculate the reprojection error with the reflection term of Walter et al. (2007), which takes the roughness of the materials into account. For glass or highly reflective materials we do not reproject samples, because such materials are very dependent on the view position.
Using adaptive sampling on top of the default path tracer yields a better efficiency, if we compare the variance of the distance to the reference pixels, with the default path tracer. The amount of efficiency depends on the minimum and maximum amount of samples we take, where in between samples are distributed by a decreasing threshold based on the variance of the pixels. The amount of samples a pixel gets between the minimum and maximum amount of samples determines a probability, which is used to distribute new samples. By scaling the variance with the luminance of a pixel, our adaptive sampling method focuses more on dark areas of a scene.
Combining the methods gives adaptive sampling more focus on the pixels that are new or did not get a sample from the reprojection method. Therefore, it quickly reduces the largest amount of noise which is useful for other techniques like filtering. The combination is a small improvement on the reprojection method when used in larger scenes. Different scene properties like: lighting, materials and amount of triangles, influences both the adaptive sampling as the reprojection method. This makes the amount of efficiency that both methods provide scene dependent.
Currently the methods are only available for static scenes. For future work we would like to see an implementation of the methods in scenes with moving objects and lights. Future work for the adaptive sampling method, is finding a way to dynamically increasing or decrease the probability of sampling a pixel. This can improve the adaptive sampling method over time. We would also like to see the methods in combination with a real-time filter, which may remove all the noise in the first milliseconds.
Contact
For more information, the complete thesis can be downloaded here. If you have questions or want to know more, you can send a mail to: olafschalk at live dot nl.