For an introduction to Bayesian methods, you can look at the slides for my NIPS*2004 tutorial: Postscript or PDF.
Here are some publications of mine that touch on fundamental issues in Bayesian inference:
Shahbaba, B. and Neal, R. M. (2005) ``Improving classification when a class hierarchy is available using a hierarchy-based prior'', Technical Report No. 0510, Dept. of Statistics, 11 pages: abstract, postscript, pdf.Neal, R. M. (2001) ``Transferring prior information between models using imaginary data'', Technical Report No. 0108, Dept. of Statistics, University of Toronto, 29 pages: abstract, postscript, pdf, associated software.
Neal, R. M. (2001) ``Defining priors for distributions using Dirichlet diffusion trees'', Technical Report No. 0104, Dept. of Statistics, University of Toronto, 25 pages: abstract, postscript, pdf, associated software.
Neal, R. M. (1997) ``Monte Carlo implementation of Gaussian process models for Bayesian regression and classification'', Technical Report No. 9702, Dept. of Statistics, University of Toronto, 24 pages: abstract, postscript, pdf, associated reference, associated software.
Neal, R. M. (1996) Bayesian Learning for Neural Networks, Lecture Notes in Statistics No. 118, New York: Springer-Verlag: blurb, associated references, associated software.
Neal, R. M. (1994) ``Priors for infinite networks'', Technical Report CRG-TR-94-1, Dept. of Computer Science, University of Toronto, 22 pages: abstract, postscript, pdf, associated reference.
Neal, R. M. (1993) Probabilistic inference using Markov chain Monte Carlo methods, Technical Report CRG-TR-93-1, Dept. of Computer Science, University of Toronto, 144 pages: abstract, contents, postscript, pdf.
Neal, R. M. (1991) ``Bayesian mixture modeling by Monte Carlo simulation'', Technical Report CRG-TR-91-2, Dept. of Computer Science, University of Toronto, 23 pages: abstract, postscript, pdf, associated references.