an occluded basketball


We propose a new model for the probabilistic estimation of continuous state variables from a sequence of observations, such as tracking the position of an object in video. This mapping is modeled as a product of dynamics experts (features relating the state at adjacent time-steps) and observation experts (features relating the state to the image sequence). Individual features are flexible in that they can switch on or off at each time-step depending on their inferred relevance (or on additional side information), and discriminative in that they need not model the full generative likelihood of the data. When trained conditionally, this permits the inclusion of a broad range of rich features (for example, features relying on observations from multiple time-steps), and allows the relevance of features to be learned from labeled sequences.



Here are the results of applying our model to the problem of tracking a basketball. The etimated location of ball is given in blue, with the size of the ellipse indicating the uncertainty in position. The predicted locations of the basketball, according to each of the individual observation features, are given by yellow boxes (if our model turns the feature on during that timestep) and red circles (if the feature is turned off).