TensorFaces - Multilinear Tensor Decomposition of Image Ensembles

The goal of machine vision is automated image understanding and object recognition by a computer. Recent events have redoubled interest in biometrics and the application of computer vision technologies to non-obtrusive identification, surveillance, tracking, etc. Face recognition is a difficult problem for computers. This is due largely to the fact that images are the composite consequence of multiple factors relating to scene structure (i.e., the location and shapes of visible objects), illumination (i.e., the location and types of light sources), and imaging (i.e., viewpoint, viewing direction and camera characteristics). Multiple factors can confuse and mislead an automated recognition system. In addressing this problem, we take advantage of the assets of multilinear algebra, the algebra of higher-order tensors, to obtain a parsimonious representation that separates the various constituent factors. Our new representation of facial images, called TensorFaces, leads to improved recognition algorithms for use in the aforementioned applications.

Proprietary Position
US Patent Application No. 60/383,300 filed March 23, 2002.

Human Motion Signatures - 3-Mode Factor Analysis

Given motion capture samples of Charlie Chaplin's walk, is it possible to synthesize other motions---say, ascending or descending stairs---in his distinctive style? More generally, in analogy with handwritten signatures, do people have characteristic motion signatures that individualize their movements? If so, can these signatures be extracted from example motions? Furthermore, can extracted signatures be used to recognize, say, a particular individual's walk subsequent to observing examples of other movements produced by this individual?

We have developed an algorithm that extracts motion signatures and uses them in the animation of graphical characters. The mathematical basis of our algorithm is a statistical numerical technique known as n-mode analysis. For example, given a corpus of walking, stair ascending, and stair descending motion data collected over a group of subjects, plus a sample walking motion for a new subject, our algorithm can synthesize never before seen ascending and descending motions in the distinctive style of this new individual.

Proprietary Position
US Patent Application No. 60/337,912 filed December 6, 2001.

Adaptive Meshes

Adaptive mesh models for the nonuniform sampling and reconstruction of visual data. Adaptive meshes are dynamic models assembled from nodal masses connected by adjustable springs. Acting as mobile sampling sites, the nodes observe interesting properties of the input data, such as intensities, depths, gradients, and curvatures. The springs automatically adjust their stiffnesses based on the locally sampled information in order to concentrate nodes near rapid variations in the input data. The representational power of an adaptive mesh is enhanced by its ability to optimally distribute the available degrees of freedom of the reconstructed model in accordance with the local complexity of the data.

We developed open adaptive mesh and closed adaptive shell surfaces based on triangular or rectangular elements. We propose techniques for hierarchically subdividing polygonal elements in adaptive meshes and shells. We also devise a discontinuity detection and preservation algorithm suitable for the model. Finally, motivated by (nonlinear, continuous dynamics, discrete observation) Kalman filtering theory, we generalize our model to the dynamic recursive estimation of nonrigidly moving surfaces.