Radford Neal's Publications

Contents:
Book
Refereed research papers
Refereed research notes
Refereed tutorial/review papers
Refereed conference papers
Unrefereed conference papers
Book chapters
Ph.D. thesis
Technical reports
Unpublished manuscripts
References to selected papers categorized by subject can be found in the notes on my research interests.

Book

Neal, R. M. (1996) Bayesian Learning for Neural Networks, Lecture Notes in Statistics No. 118, New York: Springer-Verlag: blurb, associated references, associated software.

Refereed research papers

Shahbaba, B., Lan, S., Johnson, W. O., and Neal, R. M. (2013) ``Split Hamiltonian Monte Carlo'', Statistics and Computing, has appeared online, but not yet in a print issue, 11 pages, doi:10.1007/s11222-012-9373-1.

Tarasov, L., Dykeb, A. S., Neal, R. M., and Peltier, W. R. (2011) ``A data-calibrated distribution of deglacial chronologies for the North American ice complex from glaciological modeling'', Earth and Planetary Science Letters, available online, 5 October 2011, doi:10.1016/j.epsl.2011.09.010.

Shahbaba, B. and Neal, R. M. (2009) ``Nonlinear Models Using Dirichlet Process Mixtures'', Journal of Machine Learning Research, vol. 10, pp. 1829-1850: abstract, pdf, associated references.

Li, L. and Neal, R. M. (2008) ``Compressing parameters in Bayesian high-order models with application to logistic sequence models'', Bayesian Analysis, vol. 3, pp. 793-822: abstract, pdf.

Li, L., Zhang, J., and Neal, R. M. (2008) ``A method for avoiding bias from feature selection with application to naive Bayes classification models'', Bayesian Analysis, vol. 3, pp. 171-196: abstract, pdf.

Sun, S., Greenwood, C. M. T., and Neal, R. M. (2007) ``Haplotype inference using a Bayesian hidden Markov model'', Genetic Epidemiology, vol. 31, pp. 937-948: abstract .

Jain, S. and Neal, R. M. (2007) ``Splitting and merging components of a nonconjugate Dirichlet process mixture model'' (with discussion), Bayesian Analysis, vol. 2, pp 445-472, posted online 2007-01-23: pdf (discussion is in other URLs).

Listgarten, J., Neal, R. M., Roweis, S. T., Wong, P., and Emili, A. (2007) ``Difference detection in LC-MS data for protein biomarker discovery'', Bioinformatics, vol. 23, pp. e198-e204.

Shahbaba, B. and Neal, R. M. (2007) ``Improving classification when a class hierarchy is available using a hierarchy-based prior, Bayesian Analysis, vol. 2, pp. 221-238: pdf.

Shahbaba, B. and Neal, R. M. (2006) ``Gene function classification using Bayesian models with hierarchy-based priors'', BMC Bioinformatics, 7:448, 9 pages: abstract, pdf, html, associated references.

Jain, S. and Neal, R. M. (2004) ``A Split-Merge Markov Chain Monte Carlo Procedure for the Dirichlet Process Mixture Model'', Journal of Computational and Graphical Statistics, vol. 13, pp. 158-182: abstract, associated references.

Neal, R. M. (2003) ``Slice sampling'' (with discussion), Annals of Statistics, vol. 31, pp. 705-767: abstract, text from online journal site, associated references, associated software, more associated software.

Neal, R. M. (2001) ``Annealed importance sampling'', Statistics and Computing, vol. 11, pp. 125-139: abstract, associated references.

Diaconis, P., Holmes, S., and Neal, R. M. (2000) ``Analysis of a nonreversible Markov chain sampler'', Annals of Applied Probability, vol. 10, pp. 726-752: abstract, associated reference.

Neal, R. M. (2000) ``Markov chain sampling methods for Dirichlet process mixture models'', Journal of Computational and Graphical Statistics, vol. 9, pp. 249-265: abstract, associated references, associated software.

Moffat, A., Neal, R. M., and Witten, I. H. (1998) ``Arithmetic coding revisited'', ACM Transactions on Information Systems, vol. 16, pp. 256-294: abstract, associated reference, associated software.

Neal, R. M. and Dayan, P. (1997) ``Factor analysis using delta-rule wake-sleep learning'', Neural Computation, vol. 9, pp. 1781-1803: abstract, associated references, associated software.

MacKay, D. J. C. and Neal, R. M. (1996) ``Near Shannon limit performance of low density parity check codes'', Electronics Letters, vol. 32, pp. 1645-1646. Reprinted with printing errors corrected in vol. 33, pp. 457-458: abstract.

Neal, R. M. (1996) ``Sampling from multimodal distributions using tempered transitions'', Statistics and Computing, vol. 6, pp. 353-366: abstract, associated references.

Hinton, G. E., Dayan, P., Frey, B. J., and Neal, R. M. (1995) ``The ``wake-sleep'' algorithm for unsupervised neural networks'', Science, vol. 268, pp. 1158-1161: abstract, associated references.

Dayan, P., Hinton, G. E., Neal, R. M., and Zemel, R. S. (1995) ``The Helmholtz machine'', Neural Computation, vol. 7, pp. 1022-1037: abstract, associated reference.

Neal, R. M. (1994) ``An improved acceptance procedure for the hybrid Monte Carlo algorithm'', Journal of Computational Physics, vol. 111, pp. 194-203: abstract.

Neal, R. M. (1992) ``Connectionist learning of belief networks'', Artificial Intelligence, vol. 56, pp. 71-113: abstract, associated reference.

Witten, I. H., Birtwistle, G. M., Cleary, J. G., Hill, D. R., Levinson, D., Lomow, G. A., Neal, R., Peterson, M., Unger, B. W., and Wyvill, B. L. M. (1983) ``JADE: A distributed software prototyping environment'', ACM Operating Systems Review, vol. 7, no. 3.

Witten, I. H. and Neal, R. M. (1982) ``Using Peano curves for bilevel display of continuous-tone images'', IEEE Computer Graphics and Applications, vol. 2, no. 3.

Refereed research notes

Neal, R. M. (2000) ``On deducing conditional independence from d-separation in causal graphs with feedback'', Journal of Artificial Intelligence Research, vol. 12, pp. 87-91: abstract, postscript, pdf.

Gilks, W. R., Neal, R. M., Best, N. G., and Tan, K. K. C. (1997) ``Corrigendum: Adaptive rejection Metropolis sampling'', Applied Statistics, vol. 46, pp. 541-542. [Note: Contrary to the impression given by the journal's failure to cite the original reference, I was not an author of the paper containing the error which this note corrects.]

Neal, R. M. (1993) ``Comments on `A theoretical analysis of Monte Carlo algorithms for the simulation of Gibbs random field images''', IEEE Transactions on Information Theory, vol. 39, p. 310.

Neal, R. M. (1992) ``Asymmetric parallel Boltzmann machines are belief networks'', Neural Computation, vol. 4, pp. 832-834.

Refereed tutorial/review papers

Kass, R. E., Carlin, B. P., Gelman, A., and Neal, R. M. (1998) ``Markov Chain Monte Carlo in Practice: A Roundtable Discussion'', The American Statistician, Vol. 52, pp. 93-100.

Witten, I. H., Neal, R. M., and Cleary, J. G. (1987) ``Arithmetic coding for data compression'', Communications of the ACM, vol. 30, pp. 520-540: abstract, associated software.

Refereed conference papers

Neal, R. M. (2008) ``Computing likelihood functions for high-energy physics experiments when distributions are defined by simulators with nuisance parameters'', in the proceedings of the PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics, June 2007, CERN 2008-001, pp. 119-126: abstract, postscript, pdf.

Listgarten, J., Neal, R. M., Roweis, S. T., Puckrin, R., and Cutler, S. (2007) ``Bayesian detection of infrequent differences in sets of time series with shared structure'', in B. Scholkopf, et al (editors), Advances in Neural Information Processing Systems 19 (aka NIPS*2006), 8 pages.

Meeds, E., Ghahramani, Z., Neal, R., and Roweis, S. (2007) ``Modeling dyadic data with binary latent features'', in B. Scholkopf, et al (editors), Advances in Neural Information Processing Systems 19 (aka NIPS*2006), 8 pages.

Listgarten, J., Neal, R. M., Roweis, S. T., and Emili, A. (2005) ``Multiple alignment of continuous time series'', in L. K. Saul, et al (editors), Advances in Neural Information Processing Systems 17 (aka NIPS*2004), MIT Press, 8 pages: abstract, postscript, pdf.

Neal, R. M., Beal, M. J., and Roweis, S. T. (2004) ``Inferring state sequences for non-linear systems with embedded hidden Markov models'', in S. Thrun, et al (editors), Advances in Neural Information Processing Systems 16 (aka NIPS*2003), MIT Press, 8 pages: abstract, postscript, pdf, associated reference.

Neal, R. M. (2003) ``Density modeling and clustering using Dirichlet diffusion trees'', in J. M. Bernardo, et al. (editors) Bayesian Statistics 7, pp. 619-629: abstract, postscript, pdf, associated references, associated software.

Harvey, M. and Neal, R. M. (2000) ``Inference for Belief Networks Using Coupling From the Past'', in C. Boutilier and M. Goldszmidt (editors), Uncertainty in Artificial Intelligence: Proceedings of the Sixteenth Conference (2000), pp. 256-263: abstract, postscript, pdf, associated reference.

Neal, R. M. (1998) ``Regression and classification using Gaussian process priors'' (with discussion), in J. M. Bernardo, et al (editors) Bayesian Statistics 6, Oxford University Press, pp. 475-501: abstract, postscript (without discussion), pdf (without discussion), associated reference, associated software.

MacKay, D. J. C. and Neal, R. M. (1995) ``Good codes based on very sparse matrices'', in C. Boyd (editor) Cryptography and Coding: 5th IAM Conference, Lecture Notes in Computer Science No. 1025, pp. 100-111. Springer-Verlag: abstract.

Moffat, A., Neal, R., and Witten, I. H. (1995) ``Arithmetic coding revisited'', in J. A. Storer and M. Cohn (editors) Proceedings of the Fifth IEEE Data Compression Conference, pp. 202-211, Los Alamitos, California: IEEE Computer Society Press.

Neal, R. M. (1993) ``Bayesian learning via stochastic dynamics'', in C. L. Giles, S. J. Hanson, and J. D. Cowan (editors) Advances in Neural Information Processing Systems 5 (aka NIPS*1992), pp. 475-482, San Mateo, California: Morgan Kaufmann: abstract, postscript, pdf, associated references.

Unrefereed conference papers

Thompson, M. B. and Neal, R. M. (2010) ``Slice sampling with adaptive multivariate steps: The shrinking-rank method'', JSM 2010, Section on Statistical Computing, pp. 3890-3896: abstract, pdf.

Hinton, G. E., Dayan, P., To, A., and Neal, R. (1995) ``The Helmholtz machine through time'', in F. Fogelman-Soulie and R. Gallinari (editors) ICANN-95, pp. 483-490: abstract, associated reference.

Neal, R. M. (1992) ``Bayesian mixture modeling'', in C. R. Smith, G. J. Erickson, and P. O. Neudorfer (editors) Maximum Entropy and Bayesian Methods: Proceedings of the 11th International Workshop on Maximum Entropy and Bayesian Methods of Statistical Analysis, Seattle, 1991, pp. 197-211, Dordrecht: Kluwer Academic Publishers: abstract, associated references.

Neal, R. M., Lomow, G. A., Peterson, M., Unger, B. W., and Witten, I. H. (1984) ``Experience with an inter-process communication protocol in a distributed programming environment'', CIPS Session 84 Conference, Calgary, Alberta, May 1984.

Wyvill, B. L. M., Neal, R., Levinson, D., and Bramwell, B. (1984) ``JAGGIES: A distributed hierarchical graphics system'', CIPS Session 84 Conference, Calgary, Alberta, May 1984.

Book chapters

Neal, R. M. (2010) ``MCMC using Hamiltonian dynamics'', in the Handbook of Markov Chain Monte Carlo, S. Brooks, A. Gelman, G. L. Jones, and X.-L. Meng (editors), Chapman & Hall / CRC Press, pp. 113-162: abstract, postscript, pdf, associated software.

Neal, R. M. and Zhang, J. (2006) ``High dimensional classification with Bayesian neural networks and Dirichlet diffusion trees'', in I. Guyon, S. Gunn, M. Nikravesh, and L. A. Zadeh (editors) Feature Extraction: Foundations and Applications, Studies in Fuzziness and Soft Computing, Volume 207, Springer, pp. 265-295.

Neal, R. M. (2006) ``Classification with Bayesian neural networks'', in J. Quinonero-Candela, B. Magnini, I. Dagan, and F. D'Alche-Buc (editors) Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Textual Entailment, Lecture Notes in Computer Science no. 3944, Springer-Verlag, pp. 28-32.

Neal, R. M. (1998) ``Assessing relevance determination methods using DELVE'', in C. M. Bishop (editor), Neural Networks and Machine Learning, pp. 97-129, Springer-Verlag: abstract, associated references, postscript, pdf.

Neal, R. M. and Hinton, G. E. (1998) ``A view of the EM algorithm that justifies incremental, sparse, and other variants'', in M. I. Jordan (editor) Learning in Graphical Models, pp. 355-368, Dordrecht: Kluwer Academic Publishers: abstract, postscript, pdf.

Neal, R. M. (1998) ``Suppressing random walks in Markov chain Monte Carlo using ordered overrelaxation'', in M. I. Jordan (editor) Learning in Graphical Models, pp. 205-225, Dordrecht: Kluwer Academic Publishers: abstract, associated reference, postscript, pdf.

Ph.D. Thesis

Neal, R. M. (1994) Bayesian Learning for Neural Networks, Ph.D. Thesis, Dept. of Computer Science, University of Toronto, 195 pages: abstract, postscript, pdf, associated references, associated software.

Technical reports

Shestopaloff, A. Y. and Neal, R. M. (2013) ``On Bayesian inference for the M/G/1 queue with efficient MCMC sampling'', Technical Report, 18 pages: abstract, pdf.

Wang, C. and Neal, R. M. (2013) ``MCMC methods for Gaussian process models using fast approximations for the likelihood'', Technical Report, 21 pages: abstract, pdf.

Shestopaloff, A. Y. and Neal, R. M. (2013) ``MCMC for non-linear state space models using ensembles of latent sequences, Technical Report, 18 pages: abstract, pdf.

Wang, C. and Neal, R. M. (2012) ``Gaussian process regression with heteroscedastic or non-Gaussian residuals'', Technical Report, 19 pages: abstract, pdf.

Neal, R. M. (2012) ``How to View an MCMC Simulation as a Permutation, with Applications to Parallel Simulation and Improved Importance Sampling'', Technical Report No. 1201, Dept. of Statistics, University of Toronto, 42 pages: abstract, postscript, pdf.

Shahbaba, B., Lan, S., Johnson, W. O., and Neal, R. M. (2011) ``Split Hamiltonian Monte Carlo'', arXiv:1106.5941v1.

Neal, R. M. (2010) ``MCMC using ensembles of states for problems with fast and slow variables such as Gaussian process regression'', Technical Report No. 1011, Dept. of Statistics, University of Toronto, 24 pages: abstract, postscript, pdf, associated software.

Thompson, M. and Neal, R. M. (2010) ``Covariance-adaptive slice sampling'', Technical Report No. 1002, Dept. of Statistics, University of Toronto, 17 pages: abstract, postscript, pdf.

Shahbaba, B. and Neal, R. M. (2007) ``Nonlinear Models Using Dirichlet Process Mixtures'', Technical Report No. 0707, Dept. of Statistics, University of Toronto, 16 pages: abstract, postscript, pdf, associated references.

Li, L., Zhang, J., and Neal, R. M. (2007) ``A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models'', Technical Report No. 0705, Dept. of Statistics, University of Toronto, 21 pages: abstract, postscript, pdf.

Neal, R. M. (2006) ``Puzzles of anthropic reasoning resolved using full non-indexical conditioning'', Technical Report No. 0607, Dept. of Statistics, University of Toronto, 53 pages: abstract, postscript, pdf.

Shahbaba, B. and Neal, R. M. (2006) ``Gene function classification using Bayesian models with hierarchy-based priors'', Technical Report No. 0606, Dept. of Statistics, University of Toronto, 14 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (2005) ``Estimating ratios of normalizing constants using Linked Importance Sampling'', Technical Report No. 0511, Dept. of Statistics, University of Toronto, 37 pages: abstract, postscript, pdf, associated software.

Shahbaba, B. and Neal, R. M. (2005) ``Improving classification when a class hierarchy is available using a hierarchy-based prior'', Technical Report No. 0510, Dept. of Statistics, 11 pages: abstract, postscript, pdf.

Jain, S. and Neal, R. M. (2005) ``Splitting and merging components of a nonconjugate Dirichlet process mixture model'', Technical Report No. 0507, Dept. of Statistics, 37 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (2005) ``The short-cut Metropolis method'', Technical Report No. 0506, Dept. of Statistics, University of Toronto, 28 pages: abstract, postscript, pdf, associated software.

Neal, R. M. (2004) ``Taking bigger Metropolis steps by dragging fast variables'', Technical Report No. 0411, Dept. of Statistics, University of Toronto, 9 pages: abstract, postscript, pdf, associated software.

Neal, R. M. (2004) ``Improving asymptotic variance of MCMC estimators: Non-reversible chains are better'', Technical Report No. 0406, Dept. of Statistics, University of Toronto, 25 pages: abstract, postscript, pdf.

Neal, R. M. (2003) ``Markov chain sampling for non-linear state space models using embedded hidden Markov models'', Technical Report No. 0304, Dept. of Statistics, University of Toronto, 9 pages: abstract, postscript, pdf, associated reference.

Neal, R. M. (2002) ``Circularly-coupled Markov chain sampling'', Technical Report No. 9910 (revised), Dept. of Statistics, University of Toronto, 49 pages: abstract, postscript, pdf.

Neal, R. M. (2001) ``Transferring prior information between models using imaginary data'', Technical Report No. 0108, Dept. of Statistics, University of Toronto, 29 pages: abstract, postscript, pdf, associated software.

Neal, R. M. (2001) ``Defining priors for distributions using Dirichlet diffusion trees'', Technical Report No. 0104, Dept. of Statistics, University of Toronto, 25 pages: abstract, postscript, pdf, associated references, associated software.

Pinto, R. L. and Neal, R. M. (2001) ``Improving Markov chain Monte Carlo estimators by coupling to an approximating chain'', Technical Report No. 0101, Dept. of Statistics, University of Toronto, 13 pages: abstract, postscript, pdf.

Neal, R. M. (2000) ``Slice sampling'', Technical Report No. 2005, Dept. of Statistics, University of Toronto, 40 pages: abstract, postscript, pdf, associated references, associated software, more associated software.

Jain, S. and Neal, R. M. (2000) ``A Split-Merge Markov Chain Monte Carlo Procedure for the Dirichlet Process Mixture Model'', Technical Report No. 2003, Dept. of Statistics, 32 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (1998) ``Markov chain sampling methods for Dirichlet process mixture models'', Technical Report No. 9815, Dept. of Statistics, University of toronto, 17 pages: abstract, postscript, pdf, associated references, associated software.

Neal, R. M. (1998) ``Annealed importance sampling'', Technical Report No. 9805 (revised), Dept. of Statistics, University of Toronto, 25 pages: abstract, associated references, postscript, pdf.

Neal, R. M. (1997) ``Markov chain Monte Carlo methods based on `slicing' the density function'', Technical Report No. 9722, Dept. of Statistics, University of Toronto, 27 pages: abstract, postscript, pdf, associated references.

Diaconis, P., Holmes, S., and Neal, R. M. (1997) ``Analysis of a non-reversible Markov chain sampler'', Technical Report BU-1385-M, Biometrics Unit, Cornell University, 26 pages: abstract, associated reference, postscript, pdf.

Neal, R. M. (1997) ``Monte Carlo implementation of Gaussian process models for Bayesian regression and classification'', Technical Report No. 9702, Dept. of Statistics, University of Toronto, 24 pages: abstract, postscript, pdf, associated reference, associated software.

Neal, R. M. and Dayan, P. (1996) ``Factor analysis using delta-rule wake-sleep learning'', Technical Report No. 9607, Dept. of Statistics, University of Toronto, 23 pages: abstract, postscript, pdf, associated references, associated software.

Neal, R. M. (1995) ``Suppressing random walks in Markov chain Monte Carlo using ordered overrelaxation'', Technical Report No. 9508, Dept. of Statistics, University of Toronto, 22 pages: abstract, associated reference, postscript, pdf.

Neal, R. M. (1994) ``Sampling from multimodal distributions using tempered transitions'', Technical Report No. 9421, Dept. of Statistics, University of Toronto, 22 pages: abstract, associated references, postscript, pdf.

Neal, R. M. (1994) ``Priors for infinite networks'', Technical Report CRG-TR-94-1, Dept. of Computer Science, University of Toronto, 22 pages: abstract, postscript, pdf, associated reference.

Neal, R. M. (1993) Probabilistic Inference Using Markov Chain Monte Carlo Methods, Technical Report CRG-TR-93-1, Dept. of Computer Science, University of Toronto, 144 pages: abstract, contents, postscript, pdf.

Neal, R. M. (1992) ``Bayesian training of backpropagation networks by the hybrid Monte Carlo method'', Technical Report CRG-TR-92-1, Dept. of Computer Science, University of Toronto, 21 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (1991) ``Bayesian mixture modeling by Monte Carlo simulation'', Technical Report CRG-TR-91-2, Dept. of Computer Science, University of Toronto, 23 pages: abstract, postscript, pdf, associated references.

Neal, R. M. (1990) ``Learning stochastic feedforward networks'', Technical Report CRG-TR-90-7, Dept. of Computer Science, University of Toronto, 34 pages: abstract, postscript, pdf, associated reference.

Unpublished manuscripts

Neal, R. M. (1999) ``Erroneous results in `Marginal likelihood from the Gibbs output''', unpublished letter: abstract, postscript, pdf.

Neal, R. M. (1989) ``The computational complexity of taxonomic inference'', unpublished manuscript, 18 pages: postscript, pdf.


Back to Radford Neal's home page