Beyond Light Transport

Differentiable Monte Carlo Ray Tracing through Edge Sampling

Abstract: Gradient-based methods are becoming increasingly important for computer graphics, machine learning, and computer vision. The ability to compute gradients is crucial to optimization, inverse problems, and deep learning. In rendering, the gradient is required with respect to variables such as camera parameters, light sources, scene geometry, or material appearance. However, computing the gradient of rendering is challenging because the rendering integral includes visibility terms that are not differentiable. Previous work on differentiable rendering has focused on approximate solutions. They often do not handle secondary effects such as shadows or global illumination, or they do not provide the gradient with respect to variables other than pixel coordinates. We introduce a general-purpose differentiable ray tracer, which, to our knowledge, is the first comprehensive solution that is able to compute derivatives of scalar functions over a rendered image with respect to arbitrary scene parameters such as camera pose, scene geometry, materials, and lighting parameters. The key to our method is a novel edge sampling algorithm that directly samples the Dirac delta functions introduced by the derivatives of the discontinuous integrand. We also develop efficient importance sampling methods based on spatial hierarchies. Our method can generate gradients in times running from seconds to minutes depending on scene complexity and desired precision. We interface our differentiable ray tracer with the deep learning library PyTorch and show prototype applications in inverse rendering and the generation of adversarial examples for neural networks.

Authors/Presenter(s): Tzu-Mao Li, MIT CSAIL, United States of America
Miika Aittala, MIT CSAIL, United States of America
Fredo Durand, MIT CSAIL, United States of America
Jaakko Lehtinen, Aalto University, NVIDIA, Finland

Selective guided sampling with complete light transport paths

Abstract: Finding good global importance sampling strategies for Monte Carlo light transport is challenging. While estimators using local methods (such as BSDF sampling or next event estimation) often work well in the majority of a scene, small regions in path space can be sampled insufficiently (e.g. a reflected caustic). We propose a novel data-driven guided sampling method which selectively adapts to such problematic regions and complements the unguided estimator. It is based on complete transport paths, i.e. is able to resolve the correlation due to BSDFs and free flight distances in participating media. It is conceptually simple and places anisotropic truncated Gaussian distributions around guide paths to reconstruct a continuous probability density function (guided PDF). Guide paths are iteratively sampled from the guided as well as the unguided PDF and only recorded if they cause high variance in the current estimator. While plain Monte Carlo samples paths independently and Markov chain-based methods perturb a single current sample, we determine the reconstruction kernels by a set of neighbouring paths. This enables local exploration of the integrand without detailed balance constraints or the need for analytic derivatives. We show that our method can decompose the path space into a region that is well sampled by the unguided estimator and one that is handled by the new guided sampler. In realistic scenarios, we show 4× speedups over the unguided sampler.

Authors/Presenter(s): Florian Reibold, Karlsruhe Institute of Technology, Germany
Johannes Hanika, Karlsruhe Institute of Technology, Germany
Alisa Jung, Karlsruhe Institute of Technology, Germany
Carsten Dachsbacher, Karlsruhe Institute of Technology, Germany

Light in Power: A General and Parameter-free Algorithm for Caustic Design

Abstract: We present in this paper a generic and parameter-free algorithm to efficiently build a wide variety of optical components, such as mirrors or lenses, that satisfy some light energy constraints. In all of our problems, one is given a collimated or point light source and a desired illumination after reflection or refraction and the goal is to design the geometry of a mirror or lens which transports exactly the light emitted by the source onto the target. We first propose a general framework and show that eight different optical component design problems amount to solving a Light Energy Conservation equation that involves the computation of Visibility diagrams. We then show that these diagrams all have the same structure and can be obtained by intersecting a 3D Power diagram with a planar or spherical domain. This allows us to propose an efficient and fully generic algorithm capable to solve these eight optical component design problems. The support of the prescribed target illumination can be a set of directions or a set of points located at a finite distance. Our solutions can satisfy design constraints such as convexity or concavity and are always graphs over the plane or the sphere. We show the effectiveness of our algorithm on numerous simulated and fabricated examples.

Authors/Presenter(s): Jocelyn Meyron, Université Grenoble-Alpes, France
Quentin Mérigot, Université Paris-Sud, France
Boris Thibert, Université Grenoble-Alpes, France

A radiative transfer framework for non-exponential media

Abstract: We develop a new theory of volumetric light transport for media with non-exponential free-flight distributions. Recent insights from atmospheric sciences and neutron transport demonstrate that such distributions arise in the presence of correlated scatterers, which are naturally produced by processes such as cloud condensation and fractal-pattern formation. Our theory formulates a non-exponential path integral as the result of averaging stochastic classical media, and we introduce practical models to solve the resulting averaging problem efficiently. Our theory results in a generalized path integral which allows us to handle non-exponential media using the full range of Monte Carlo rendering algorithms while enriching the range of achievable appearance. We propose parametric models for controlling the statistical correlations by leveraging work on stochastic processes, and we develop a method to combine such unresolved correlations (and the resulting non-exponential free-flight behavior) with explicitly modeled macroscopic heterogeneity. This provides a powerful authoring approach where artists can freely design the shape of the attenuation profile separately from the macroscopic heterogeneous density, while our theory provides a physically consistent interpretation in terms of a path space integral. We address important considerations for graphics including reciprocity and bidirectional rendering algorithms, all in the presence of surfaces and correlated media.

Authors/Presenter(s): Benedikt M. Bitterli, Dartmouth College, United States of America
Srinath Ravichandran, Dartmouth College, United States of America
Thomas Mueller, Disney Research, ETH Zurich, Switzerland
Magnus Wrenninge, Pixar Animation Studios, United States of America
Jan Novák, Disney Research, Switzerland
Steve Marschner, Cornell University, United States of America
Wojciech Jarosz, Dartmouth College, United States of America