Modeling Appearance Change in Image Sequences
Research Overview
Most methods for estimating optical flow assume that image appearance
of the moving objects remains more or less constant. The most common
assumption, referred to as brightness constancy, posits that the light
reflected from a moving surface toward the camera (surface irradiance)
remains constant from on frame to the next.
Unfortunately there are many cases where this assumption is violated,
causing poor motion estimates; examples include occlusions, changing
orientations of surfaces with respect to light sources, specularities,
and shadows.
To deal with complex forms of appearance changes, Michael Black, Yaser
Yacoob and I proposed a formulation in which appearance changes were
modeled by a linear mixture of causes. These included photometric causes
such as lighting variations and specularities, as well as object-specific
variations in appearance caused by complex local patterns of motion
and occlusion. This research represents an encouraging first step toward
the inference of the different physical causes of intensity variations in
image sequences, but there exist many open problems in this area.
In subsequent work, Horst Haussecker and I derived a linear formulation
of a more restricted class of appearance changes based on physical
models of image formation and photometry.
Such physical models include changes in surface reflection caused by
changes in surface orientation with respect to a direction light source,
and thermal diffusion in infrared (IR) imaging. The latter formulation
was used for tracking unmarked paper in a copier with laser-induced
thermal emittance patterns on the paper surface.
Related Publications
- Haussecker, H.W. and Fleet, D.J. (2001) Estimating optical flow with
physical models of brightness variation. IEEE Transactions on Pattern
Analysis and Machine Intelligence 23(6):661-673
(abstract)
(pdf)
© IEEE
- Black, M.J., Fleet, D.J., and Yacoob, Y. (2000) Robustly
estimating changes in image appearance. Computer Vision and
Image Understanding 78(1):8-31
(abstract)
- Haussecker, H. and Fleet, D.J. (2000) Computing optical flow with
physical models of brightness variation. IEEE Conference on Computer
Vision and Pattern Recognition,
Hilton Head, South Carolina, Vol. II, pp. 760-767
(pdf)
© IEEE
- Black, M.J., Fleet, D.J., and Yacoob, Y. (1998) A framework for
modeling appearance change in image sequences. IEEE International
Conference on Computer Vision, Mumbai, India, January, pp. 660-667
(pdf)
© IEEE
- Black, M.J., Yacoob, Y., and Fleet, D.J. (1997) Modelling appearance
change in image sequences. International Workshop on Visual Form,
Capri, Italy, May. World Scientific Pub., C. Arcelli, L.P. Cordella, and
G.S. di Baja, Eds., pp. 11-20
Return to David Fleet's home page.