Handheld Reflectance Acquisition of Paintings

Niranjan Thanikachalam   Loïc Baboulaz   Damien Firmenich   Sabine Süsstrunk   Martin Vetterli

IEEE Transactions on Computational Imaging, 2017

Abstract : Relightable photographs are alternatives to traditional photographs as they provide a richer viewing experience. However, the complex acquisition systems of existing techniques have restricted its usage to specialized setups. We introduce an easy-to-use and affordable solution for using smartphones to acquire the reflectance of paintings and similar almost-planar objects like tablets, engravings and textile. Our goal is to enable interactive relighting of such artifacts by everyone. In our approach, we non-uniformly sample the reflectance functions by moving the LED light of a smartphone and simultaneously tracking the position of the smartphone by using its camera. We then propose a compressive-sensing based approach for reconstructing the light transport matrix from the non-uniformly sampled data. As shown with experiments, we accurately reconstruct the light transport matrix that can then be used to create relightable photographs.

Full Text

This site serves as a web based gallery for the relightable photographs acquired using our mobile setup. Click on an object to start relighting it.

Scene with Toys

Oil painting of Notre Dame, Paris

A specular patch of oil painting

A glossy Iconograph

Oil painting by Hohmann

Wheat Field with Cypresses

Potrait of Cornelis van der Geest