Light-Path Triangulation

 
Overview
 
We investigate the feasibility of reconstructing an arbitrarily-shaped specular scene (refractive or mirror-like) from one or more viewpoints. By reducing shape recovery to the problem of reconstructing individual 3D light paths that cross the image plane, we obtain three key results. First, we show how to compute the depth map of a specular scene from a single viewpoint, when the scene redirects incoming light just once. Second, for scenes where incoming light undergoes two refractions or reflections, we show that three viewpoints are sufficient to enable reconstruction in the general case. Third, we show that it is impossible to reconstruct individual light paths when light is redirected more than twice. Our analysis assumes that, for every point on the image plane, we know at least one 3D point on its light path. This leads to reconstruction algorithms that rely on an "environment matting" procedure to establish pixel-to-point correspondences along a light path. Preliminary results for a variety of scenes (mirror, glass, etc) are also presented.
 
People
 
Eron Steger (University of Toronto)
Nigel J. Morris (University of Toronto)
Kiriakos N. Kutulakos (University of Toronto)
 
Related Publications
 
Eron Steger and Kiriakos N. Kutulakos, A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation. Int. J. Computer Vision, vol. 76, no. 1, pp. 13-29, 2008. Winner, David Marr Prize Honorable Mention.   PDF (2354KB)    SpringerLink entry
Eron Steger and Kiriakos N. Kutulakos, A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation. Proc. 10th Int. Conf. on Computer Vision, Beijing, China, pp. 1448-1455, 2005. Winner, David Marr Prize Honorable Mention.   PDF (886KB)    IEEEXplore entry
Eron Steger and Kiriakos N. Kutulakos, A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation. Microsoft Research Tech. Rep., no. MSR-TR-2005-12, 2005. This is a slightly-extended version of the ICCV paper.   PDF (1047KB)   
Nigel J. Morris and Kiriakos N. Kutulakos, Dynamic Refraction Stereo. Proc. 10th Int. Conf. on Computer Vision, Beijing, China, pp. 1448-1455, 2005.   PDF (985KB)    IEEEXplore entry
 
Supplementary Material
 

Supplementary material submitted to ICCV 2005 (2.2MB zip file)

Dynamic refraction stereo project page

 
Acknowledgements
 
This work was supported in part by the National Science Foundation under Grant No. IRI-9875628, by the Natural Sciences and Engineering Research Council of Canada under the RG-PIN and PGS-M programs, by a fellowship from the Alfred P. Sloan Foundation, by an Ontario Premier's Research Excellence Award and by Microsoft Research.
 
Site last modified on Wednesday, April 24, 2013
Send questions or comments about this page to Kyros Kutulakos