Shanon Xuan Ju, Michael J. Black, Allan D. Jepson, Skin and Bones: Multi-layer, Locally Affine, Optical Flow and Regularization with Transparency, Proc. Computer Vision and Pattern Recognition Conference (CVPR'96), San Francisco, CA, June, 1996. Postscript. © IEEE

Abstract: This paper describes a new method for estimating optical flow that strikes a balance between the flexibility of local dense computations and the robustness and accuracy of global parameterized flow models. The approach assumes that image motion can be represented by an affine flow model within a local image patch. Since some image regions may not have sufficient information to estimate an affine motion model robustly, we define a spatial smoothness constraint on the affine flow parameters of neighboring patches. We refer to this as a ``Skin and Bones'' model in which the affine patches can be thought of as rigid bones connected by a flexible skin. Since local image patches may contain multiple motions we use a layered representation for the affine bones. With the possibility of mulitple motions at a given point, standard regularization schemes cannot be used to smooth the multiple sets of affine parameters. We therefore develop a new framework for {\em regularization with transparency} that can applied to produce a smoothed layered motion representation. The motion estimation problem, with layered locally affine bones and transparent regularization, is formulated as an objective function that is minimized using an EM-algorithm. Numerous experiments with synthetic and natural images are provided throughout the paper to illustrate the method.