Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model are reviewed, and two new classes of methods are presented. One new approach is to make Metropolis-Hastings updates of the indicators specifying which mixture component is associated with each observation, perhaps supplemented with a partial form of Gibbs sampling. The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters. These methods are simple to implement and are more efficient than previous ways of handling general Dirichlet process mixture models with non-conjugate priors.
Technical Report No. 9815, Dept. of Statistics, University of Toronto (September 1998), 17 pages: postscript, pdf, associated software.
Neal, R. M. (2000) ``Markov chain sampling methods for Dirichlet process mixture models'', Journal of Computational and Graphical Statistics vol. 9, pp. 249-265: abstract, associated references, associated software.The following technical report describes a more elaborate algorithm based on split-merge proposals:
Jain, S. and Neal, R. M. (2000) ``A Split-Merge Markov Chain Monte Carlo Procedure for the Dirichlet Process Mixture Model'', Technical Report No. 2003, Dept. of Statistics (July 2000), 32 pages: abstract, postscript, pdf, associated references.Some earlier work of mine on models equivalent to Dirichlet process mixtures is described in the following technical report:
Neal, R. M. (1991) ``Bayesian mixture modeling by Monte Carlo simulation'', Technical Report CRG-TR-91-2, Dept. of Computer Science, University of Toronto, 23 pages: abstract, postscript, pdf, associated references.