Introduction
Most algorithms for tracking objects in video consist of two components: a model of the dynamics of the object being tracked, and a model of its appearance. Often the appearance model is constructed before tracking, perhaps from training images, and then used as-is when tracking test sequences.
What if the test sequence contains appearances of the object, or lighting conditions, that don't exactly match those of the training data? Typically, trackers with fixed appearance models will perform poorly under these circumstances.
In this project we make use of the new appearance information that comes available during tracking to incrementally improve a subspace appearance model of the target. The key to this algorithm is a novel incremental algorithm for PCA, allowing for efficient subspace updates.
Papers
- (latest)
Incremental Learning for Robust Visual Tracking
David Ross, Jongwoo Lim, Ruei-Sung Lin, Ming-Hsuan Yang.
In the International Journal of Computer Vision, Special Issue: Learning for Vision, 2007. [PS.GZ] [PDF] - Incremental Learning for Visual Tracking
Jongwoo Lim, David Ross, Ruei-Sung Lin, Ming-Hsuan Yang
In L. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, MIT Press, 2005. [PS.GZ] [PDF] - Adaptive Probabilistic Visual Tracking with Incremental
Subspace Update
David Ross, Jongwoo Lim, Ming-Hsuan Yang
In T. Pajdla and J. Matas, editors, Proc. Eighth European Conference on Computer Vision (ECCV 2004), Springer, 2004. [PS.GZ] [PDF]
Videos
- Here are the video results from the IJCV article. First the
performance of our algorithm on its own:
-
Next a comparison with two other trackers: WSL (highlighted ellipse)
and Comaniciu's mean-shift tracker (green dotted rectangle):
- Videos from the "Incremental Learning for Visual Tracking" NIPS04 paper can be found, along with a description, on Jongwoo's website.
- Videos corresponding to the "Adaptive Probabilistic Visual
Tracking with Incremental Subspace Update" ECCV04 paper.
- Tracking Results: David [avi/divx 2mb] Dog [avi/divx 5mb] Ming-Hsuan [avi/divx 6mb]
- Submitted Videos: David [avi/divx 3mb] Ming-Hsuan [avi/divx 10mb]
- Trouble viewing the videos? Please email me if there are any problems viewing the videos. The VLC media player should be able to show them on all platforms. On a Mac, installing Perian will let you view them in Quicktime.
Code and Data
Matlab souce code for the algorithm can be obtained here. ivt-2008-05-04.zip
The raw data sequences are also available. Feel free to use them to test your algorithms. The data used in the IJCV paper are available below as .mat files, for Matlab version 7 and above, and for Matlab 6. In two cases, the original color AVI videos are avaialble as well.
- Dudek [Matlab 7+, 133mb] [Matlab 6]
- Plush Toy (aka Sylvester) [Matlab 7+, 56mb] [Matlab 6]
- David Indoor [Matlab 7+, 24mb] [Matlab 6] [Original AVI]
- Trellis [Matlab 7+, 23mb] [Matlab 6] [Original AVI]
- Car 4 [Matlab 7+, 135mb] [Matlab 6] [Original AVI]
- Car 11 [Matlab 7+, 17mb] [Matlab 6] [Original AVI]
- Fish Fish [Matlab 7+, 67mb] [Matlab 6]
The raw videos used for the earlier papers are also available. All of the test videos are provided as as .pgm image files (one image per frame). If you don't see the sequence you're looking for, please send us an email.
- David [pgm 21mb]
- Ming-Hsuan [pgm 72mb]
- Dog [pgm 55mb]
- David Indoor [pgm 33mb] Note: Feel free to use this sequence in your own publications. If you do, please consider citing our papers, and refer to it as the "Handsome Fellow" sequence
- Sylvester (aka Plush Toy)[pgm 58mb]
- Dudek Face Sequence (also used in Jepson, Fleet, and El-Maraghi's WSL tracker) [link to webpage]
- Matlab script to read in the data load_data.m
Presentations
- Machine Learning Group Meeting given by David at the University of Toronto, January 19, 2003. [PDF]