Abstract: Factorization using Singular Value Decomposition (SVD) is often used for recovering 3D shape and motion from feature correspondences across multiple views. However, this is the correct error to minimize only when the x and y positional errors in the features are uncor- related and identically distributed. But this is rarely the case in real data, where uncertainty in feature position depends on the underlying spatial intensity structure in the image, which has strong directionality to it. The proper measure to minimize is covariance--weighted squared -error (or the Mahalanobis distance). In this talk, I will describe a new approach to covariance--weighted factorization, which can factor noisy feature correspondences with high degree of directional uncertainty into structure and motion. Our approach is based on transforming the raw-data into a covariance-weighted data space, where the components of noise in the different directions are uncorrelated and identically. We empirically show that our method does not degrade with increase in directionality of uncertainty, even in the extreme when only "normal -flow" data is available. It thus provides a unified approach for treating corner-like points together with points along linear structures in the image.

Abstract: Although digital still and video cameras are becoming ubiquitous, the user's experience with visual media is still largely passive. Yet, they play an ever increasing role in social interactions and in providing visual experiences of places and objects. In this talk, I will outline ongoing research at MSR that is aimed at enabling interaction with visual media and people to people interactions via visual media. Our technologies support image and video based 3D scene modeling, video processing and enhancement, personal photo and video sharing, annotation, and organization, video understanding, and novel adapative approaches for document layout and for presentations.