Intrinsic decomposition and inverse rendering for indoor scenes remain significant challenges in computer vision, primarily due to two distinct data gaps: evaluation data and training data. To bridge the evaluation data gap, this dissertation introduces the Measured Albedo in the Wild (MAW) dataset, comprising 888 images with physical albedo measurements, alongside complementary metrics for assessing albedo intensity, chromaticity, and texture beyond the traditional Weighted Human Disagreement Rate (WHDR).

To address the training data gap in challenging indoor scenes, the thesis introduces GaNI, a novel photometric stereo inverse rendering framework designed to effectively handle global illumination effects. GaNI employs a three-stage approach that facilitates the accurate capture and reconstruction of indoor scene properties, which enables collection of near ground-truth inverse rendering training data with photometric stereo.

These contributions collectively advance intrinsic decomposition and inverse rendering, paving the way for enhanced photorealistic applications in virtual reality, augmented reality, robotics, and computational photography.