Image Analysis using the Physics of Light Scattering

Sammanfattning: Any generic computer vision algorithm must be able to copewith the variations in appearance of objects due to differentillumination conditions. While these variations in the shadingof a surface may seem a nuisance, they in fact containinformation about the world. This thesis tries to provide anunderstanding what information can be extracted from theshading in a single image and how to achieve this. One of thechallenges lies in finding accurate models for the wide varietyof conditions that can occur.Frequency space representations are powerful tools foranalyzing shading theoretically. Surfaces act as low-passfilters on the illumination making the reflected lightband-limited. Hence, it can be represented by a finite numberof components in the Fourier domain, despite having arbitraryillumination. This thesis derives a basis for shading byrepresenting the illumination in spherical harmonics and theBRDF in a basis for isotropic reflectance. By analyzing thecontributing variance of this basis it is shown how to createfinite dimensional representations for any surface withisotropic reflectance.The finite representation is used to analytically derive aprincipal component analysis (PCA) basis of the set of imagesdue to the variations in the illumination and BRDF. The PCA isperformed model-based so that the variations in the images aredescribed by the variations in the illumination and the BRDF.This has a number of advantages. The PCA can be performed overa wide variety of conditions, more than would be practicallypossible if the images were captured or rendered. Also, thereis an explicit mapping between the principal components and theillumination and BRDF so that the PCA basis can be used as aphysical model.By combining a database of captured illumination and adatabase of captured BRDFs a general basis for shading iscreated. This basis is used to investigate materialclassification from a single image with known geometry butarbitrary unknown illumination. An image is classified byestimating the coecients in this basis and comparing them to adatabase. Experiments on synthetic data show that materialclassification from reflectance properties is hard. There aremis-classifications and the materials seem to cluster intogroups. The materials are grouped using a greedy algorithm.Experiments on real images show promising results.Keywords:computer vision, shading, illumination,reflectance, image irradiance, frequency space representations,spherical harmonics, analytic PCA, model-based PCA, materialclassification, illumination estimation