Recursive Control Variates for Inverse Rendering

Baptiste Nicolet, Fabrice Rousselle, Jan Novák, Alexander Keller, Wenzel Jakob, Thomas Müller

Abstract

We present a method for reducing errors - variance and bias - in physically based differentiable rendering (PBDR). Typical applications of PBDR repeatedly render a scene as part of an optimization loop involving gradient descent. The actual change introduced by each gradient descent step is often relatively small, causing a significant degree of redundancy in this computation. We exploit this redundancy by formulating a gradient estimator that employs an unbiased recursive control variate, which leverages information from previous optimization steps. The control variate reduces variance in gradients, and, perhaps more importantly, alleviates issues that arise from differentiating loss functions with respect to noisy inputs, a common cause of drift to bad local minima or divergent optimizations. We experimentally evaluate our approach on a variety of path-traced scenes containing surfaces and volumes and observe that primal rendering efficiency improves by a factor of up to 10.

Heterogeneous Volume Reconstruction

We apply our technique to the problem of heterogeneous volume reconstruction building on the adjoint pass of Differential Ratio Tracking [Nimier-David et al. 2022], which acts as our baseline. Experiments in the original article relied on the L1 loss function, which made it necessary to render primal images with a relatively large sample count to reduce the impact of loss function bias. Variance reduction from our recursive control variate alleviates the need for this high sample count, which improves the reconstruction quality at equal time and significantly reduces primal rendering overheads for when targeting an equal level of quality. We demonstrate both equal time and equal quality results on the following 3 scenes.