Non-Rigid Structure from Locally-Rigid Motion

We introduce locally-rigid motion, a general framework for solving the M-point, N-view structure-from-motion problem for unknown bodies deforming under orthography. The key idea is to first solve many local 3-point, N-view rigid problems independently, providing a "soup" of specific, plausibly rigid, 3D triangles. The main advantage here is that the extraction of 3D triangles requires only very weak assumptions: (1) deformations can be locally approximated by near-rigid motion of three points (i.e., stretching not dominant) and (2) local motions involve some generic rotation in depth. Triangles from this soup are then grouped into bodies, and their depth flips and instantaneous relative depths are determined. Results on several sequences, both our own and from related work, suggest these conditions apply in diverse settings---including very challenging ones (e.g., multiple deforming bodies). Our starting point is a novel linear solution to 3-point structure from motion, a problem for which no general algorithms currently exist.
Allan D. Jepson (University of Toronto)
Jonathan Taylor (University of Toronto)
Kiriakos N. Kutulakos (University of Toronto)
Related Publications
Jonathan Taylor, Allan D. Jepson and Kiriakos N. Kutulakos, Non-Rigid Structure from Locally-Rigid Motion. Proc. Computer Vision and Pattern Recognition Conf., San Francisco, CA, 2010.   PDF (5070KB)    IEEEXplore entry
Supplementary Material
  • Derivations and details on edge functional and alignment costs. [pdf]

  • Code and videos/results will be posted here soon.
This work was supported in part by the Natural Sciences and Engineering Research Council of Canada under the RGPIN, Discovery and PGS-D programs.
Site last modified on Wednesday, September 25, 2019
Send questions or comments about this page to Kyros Kutulakos