IM–Material

Spectral Painting Reproduction via Multi-layer, Custom-Ink Printing

Abstract: We propose a workflow for spectral reproduction of paintings, which captures a painting's spectral color, invariant to illumination, and reproduces it using multi-material 3D printing. We take advantage of the current 3D printers' capabilities of combining highly concentrated inks with a large number of layers, to expand the spectral gamut of a set of inks. We use a data-driven method to both predict the spectrum of a printed ink stack and optimize for the stack layout that best matches a target spectrum. We model the bidirectional mapping using a pair of neural networks, which are learned through a carefully designed multi-objective loss function. Our loss function helps find the best possible ink layout resulting in the balance between spectral reproduction and colorimetric accuracy under a multitude of illuminants. In addition, we introduce a novel spectral vector error diffusion algorithm based on combining color contoning and halftoning, which simultaneously solves the layout discretization and color quantization problems, accurately and efficiently. Our workflow outperforms the state-of-the-art models for spectral prediction and layout optimization. We demonstrate the reproduction of a number of real paintings and historically important pigments using our prototype implementation that uses 10 custom inks with varying spectra and a resin-based 3D printer.

Authors/Presenter(s): Liang Shi, Massachusetts Institute of Technology, United States of America
Vahid Babaei, Massachusetts Institute of Technology, MPI Informatik; Saarland University, MMCI, Germany
Changil Kim, Massachusetts Institute of Technology, United States of America
Michael Forshey, Massachusetts Institute of Technology, United States of America
Yuanming Hu, MIT CSAIL, United States of America
Pitchaya Sitthi-Amorn, Chulalongkorn University, Thailand
Szymon Rusinkiewicz, Princeton University, United States of America
Wojciech Matusik, Massachusetts Institute of Technology, United States of America

Acquiring spatially varying appearance of printed holographic surfaces

Abstract: We present two novel and complimentary approaches to measure diffraction effects in commonly found planar spatially varying holographic surfaces. Such surfaces are increasingly found in various decorative materials such as gift bags, holographic papers, clothing and security holograms, and produce impressive visual effects that have not been previously acquired for realistic rendering. Such holographic surfaces are usually manufactured with one dimensional diffraction gratings that are varying in periodicity and orientation over an entire sample in order to produce a wide range of diffraction effects such as gradients and kinematic (rotational) effects. Our proposed methods estimate these two parameters and allow an accurate reproduction of these effects in real-time. The first method simply uses a point light source to recover both the grating periodicity and orientation in the case of regular and stochastic textures. Under the assumption that the sample is made of the same repeated diffractive tile, good results can be obtained using just one to five photographs on a wide range of samples. The second method is based on polarization imaging and enables an independent high resolution measurement of the grating orientation and relative periodicity at each surface point. The method requires a minimum of four photographs for accurate results, does not assume repetition of an exemplar tile, and can even reveal minor fabrication defects. We present point light source renderings with both approaches that qualitatively match photographs, as well as real-time renderings under complex environmental illumination.

Authors/Presenter(s): Antoine Toisoul, Imperial College London, United Kingdom
Daljit Singh J. Dhillon, Imperial College London, United Kingdom
Abhijeet Ghosh, Imperial College London, United Kingdom

Connecting Measured BRDFs to Analytic BRDFs by Data-Driven Diffuse-Specular Separation

Abstract: The bidirectional reflectance distribution function (BRDF) is crucial for modeling the appearances of real-world materials. In production rendering, analytic BRDF models are often used to approximate the surface reflectances since they are compact and flexible. On the other hand, measured BRDFs, which are constructed from real-world measurements, usually have more realistic appearances, but consume much more storage and are hard to modify. In this paper, we propose a novel framework for connecting measured and analytic BRDFs. First, we develop a robust method for separating a measured BRDF into diffuse and specular components, as is commonly done in analytic models but has been difficult previously to do explicitly for data-driven reflectances. This diffuse-specular separation further allows novel measured BRDF editing on the diffuse and specular parts separately. In addition, we conduct analysis on each part of the measured BRDF, and demonstrate a more intuitive and lower-dimensional PCA model than Nielsen et al. [2015]. In fact, our measured BRDF model has the same number of parameters (8 parameters) as the commonly used analytic models, such as the GGX model. Finally, we visualize the analytic and measured BRDFs in the same space, and directly demonstrate their similarities and differences. We also design an analytic fitting algorithm for two-layer materials, which is more robust, efficient and simple, compared to previous non-convex optimization-based analytic fitting methods.

Authors/Presenter(s): Tiancheng Sun, University of California, San Diego, United States of America
Henrik Wann Jensen, University of California, San Diego, United States of America
Ravi Ramamoorthi, University of California, San Diego, United States of America

An Adaptive Parameterization for Efficient Material Acquisition and Rendering

Abstract: One of the key ingredients of any physically based rendering system is a detailed specification characterizing the interaction of light and matter of all materials present in a scene, typically via the Bidirectional Reflectance Distribution Function (BRDF). Despite their utility, access to real-world BRDF datasets remains limited: this is because measurements involve scanning a four-dimensional domain at sufficient resolution, a tedious and often infeasibly time-consuming process. We propose a new parameterization that automatically adapts to the behavior of a material, warping the underlying 4D domain so that most of the volume maps to regions where the BRDF takes on non-negligible values, while irrelevant regions are strongly compressed. This adaptation only requires a brief 1D/2D measurement of the material's retro-reflective properties. Our parameterization is unified in the sense that it combines several steps that previously required intermediate data conversions: the same mapping can simultaneously be used for BRDF acquisition, storage, and it supports efficient Monte Carlo sample generation. We observe that the above desiderata are satisfied by a core operation present in modern rendering systems, which maps uniform variates to directions samples that are proportional to an analytic BRDF. Based on this insight, we define our adaptive parameterization as an invertible, retro-reflectively driven mapping between the parametric and directional domains. We are able to create noise-free renderings of existing BRDF datasets after conversion into our representation with the added benefit that the warped data is significantly more compact, requiring 16KiB and 544KiB per spectral channel for isotropic and anisotropic specimens, respectively. Finally, we show how to modify an existing gonio-photometer to provide the needed retro-reflection measurements. Acquisition then proceeds within a 4D space that is warped by our parameterization. We demonstrate the efficacy of this scheme by acquiring the first set of spectral BRDFs of surfaces exhibiting arbitrary roughness, including anisotropy.

Authors/Presenter(s): Jonathan Dupuy, Unity Technologies, France
Wenzel Jakob, EPFL, Switzerland