Abstract: In this talk, I describe a parametric model-based framework for incremental object shape estimation and tracking in monocular image sequences. Model-based structure and motion estimation approaches usually assume a fixed class of shape representations (splines, deformable superquadrics, etc) that are initialized prior to tracking. Since the model shape coverage is fixed a-priori, incremental discovery of structure is decoupled from tracking, thereby limiting both processes in their scope and robustness. I present a tracking framework that supports the automatic detection and integration of geometric primitives (lines and points) incrementally during tracking. Such primitives are not explicitly captured in the initial model, but they are moving consistently with its image motion. The consistency tests used to reveal new structure are based on trinocular relationships between geometric primitives. The method allows us to increase both the model's scope and, ultimately, its higher-level shape coverage and it improves tracking robustness and accuracy by directly employing the newly recovered features in both model forward prediction and reconstruction. The formulation represents a step towards automating model shape acquisition and tracking, since it allows weaker assumptions on the availability of a prior shape representation. The method is also robust and unbiased. We demonstrate the proposed approach in an image-based tracking domain involving complex 3D object structure and motion.

(joint research with Dimitris Metaxas and Sven Dickinson)