David MacKay
.




Search :


.

David's Publications

What you can find here

This website includes My papers and software are mirrored in North America.

Master site, UK | Mirror in Canada | Mirror in USA (under construction)

  • Most of these files are ".ps.gz", i.e. postscript files (.ps) compressed with gzip (.gz).
  • Since Mon 15/7/02, I have tried to provide PDF alternatives to all postscript files.
  • Since Sun 20/10/02, DJVU alternatives are also provided.

    • If you don't have gunzip on your machine, you can find gunzip executables for various systems here; the msdos executable is also here. Some users report that gsview32 is useful for viewing the ps.gz files.
    Some users of windows machines find that if they download a ps.gz file by http then it does not unzip and display, but if they download it by ftp, it works fine. To convert an http URL to an ftp URL, please use the following pair as an example:

    ftp://www.inference.phy.cam.ac.uk/pub/www/mng10/GP/gpros.ps.gz
    http://www.inference.phy.cam.ac.uk/mng10/GP/gpros.ps.gz

    djvu files are provided for some papers. [Download djView]

There is a website devoted to this book

Information Theory, Inference, and Learning Algorithms

 (Hardback, 640 pages, Published September 2003)

Order your copy

Price: £30.00 / $50.00 from |CUP UK/USA| |amazon.co.uk/.com/.ca/.co.jp| |Barnes and Noble USA | fetchbook.info | allbookstores | biggerbooks | blackwells | kalahari.net (South Africa, R464) |

Best buys (US) - Barnes and Noble, $35.00 | amazon.com, $39.50 [amazon.co.uk, £21.00 offer has expired; now costs £29.46]

Download the book too

You may download The book in one file (640 pages):

U.K. english Canada canada South Africa south africa
PDF (A4) pdf (9M) pdf pdf
Postscript (A4) postscript (second printing, January 2004) (5M) postscript postscript
DJVU djvu file (6M) djvu file djvu file
(djvu information | Download djView)
Just the words (latex) [provided for convenient searching] (2.4M) (latex) (latex)
Individual chapters postscript and pdf available from this page mirror mirror

Notes:

  • Version 6.0 was released Thu 26/6/03; the book is finished. You are welcome to view the book on-screen. Version 6.0 was used for the first printing, published by C.U.P. September 2003.
  • Version 6.6 was released Mon 22/12/03; it will be used for the second printing, to be released January 2004. In this second printing, a small number of typographical errors were corrected, and the design of the book was altered slightly. Page-numbering generally remains unchanged, except in chapters 1, 6, and 28, where a few paragraphs, figures, and equations have moved around. All equation, section, and exercise numbers are unchanged.

Copyright issues: The book is copyright (c) Cambridge University Press. It has been available in bookstores since September 2003. The cover price is 30 pounds (UK) and $50 (USA).

Now the book is published, these files will remain viewable on this website. The same copyright rules will apply to the online copy of the book as apply to normal books. [e.g., copying the whole book onto paper is not permitted.]

History:
Draft 1.1.1 - March 14 1997.
Draft 1.2.1 - April 4 1997.
Draft 1.2.3 - April 9 1997.
Draft 1.2.4 - April 10 1997. Margins altered so as to print better on Northamerican paper
Draft 1.3.0 - December 23 1997.
Draft 1.9.0 - Feb 1 1999.
Draft 2.0.0 - Jan 6 2000. New page layout.
Draft 2.2.0 - Dec 23 2000. Fresh draft.
Draft 3.1415 - Jan 12 2003. Nearly finished.
Draft 4.0 - April 15 2003. Chapter sequence finalized.
Draft 4.1 - April 18 2003. Adding new frontmatter (Preface etc)to book. Corrected printing of letter version.
Version 6.0 - Thu 26 June 2003. Final version.
Version 6.6 - Mon 22 December 2003. Second printing.

  • Here is my method for converting to two-up under linux:
    pstops '4:0L@.67(20cm,1cm)+1L@.67(20cm,15cm),3R@.67(1cm,15.25cm)\
+2R@.67(1cm,29.25cm)' $*.ps $*.dps

Gallager codes, MN codes and review papers on sparse graph codes

mncEL.ps.gz. | PDF. | DJVU. | abstract. | ps mirror, Canada | | PDF | | DJVU |
` Near Shannon Limit Performance of Low Density Parity Check Codes '.
With Radford Neal. (Very short paper, published in Elec. Lett.)
mncN.ps.gz. (55 pages) | PDF | DJVU | abstract. | ps mirror, Canada | pdf | DJVU |
` Good Error-Correcting Codes based on Very Sparse Matrices '.
Appeared in IEEE-IT, 1999. Also, with Radford Neal, an early massively shortened version of this paper (published in the 1995 book `Cryptography and Coding' LNCS 1025): mnc4s.ps.gz. | PDF | | DJVU | | ps mirror, Canada | | pdf | DJVU |
mncNcorrns.ps.gz. (1 page) | PDF | DJVU | mncNcorrns.tex. | ps mirror, Canada | | pdf | DJVU |
` Errata for the paper:
Good Error-Correcting Codes based on Very Sparse Matrices '.
(Contains 4 corrections to the published version)
gfqoptimize.ps.gz. gfqoptimize.pdf. abstract. | <- UK | Canada -> | gfqoptimize.ps.gz. gfqoptimize.pdf. abstract.
`Optimizing Sparse Graph Codes over GF(q)'. (3 pages) (2003)
More on Gallager Codes over GF(q): see Matthew Davey's web pages
dilbert3.ps.gz | PDF | DJVU | Canada -> | dilbert3.ps.gz. pdf | | DJVU |
`Error-correcting codes and probability propagation'. (16 pages of lecture notes for the 2001 meeting on Statistics and Artificial Intelligence) | Also accumulator networks.
sparsecodes0.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | sparsecodes0.ps.gz. abstract.
` Sparse Graph Codes '. (3 page summary paper)
sparsecodes.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | sparsecodes.ps.gz. abstract.
` Gallager Codes - Recent Results '. (12 page paper)
encyclopedia.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | encyclopedia.ps.gz. abstract.
` Encyclopedia of Sparse Graph Codes '.
Encyclopedia of Sparse Graph Codes (hypertext archive, under construction)
rev.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | rev.ps.gz. abstract.
` A Revolution: Belief Propagation in Graphs with Cycles '.
Brendan J. Frey and David J.C. MacKay. Presented at NIPS 1997.
ibis.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | ibis.ps.gz. abstract.
` Relationships between Sparse Graph Codes '. This Paper appears in the (unpublished) proceedings of IBIS 2000, Japan.

I have also put a mirror of Gallager's classic book on my website, Gallager, R. G., Low Density Parity Check Codes, Monograph, M.I.T. Press, 1963.

[See also the big paper on Gallager codes and MN codes.]
mceliece.ps.gz | TBSC-Decoding.pdf (old pdf file). | PDF | | DJVU |
BSC Thresholds for Code Ensembles based on `Typical pairs' decoding.
S. Aji, H. Jin, A. Khandekar, R. J. McEliece, and D. J. C. MacKay. In proceedings of IMA meeting 1999.
theorems.ps.gz | djvu | abstract | <- UK | Canada -> | theorems.ps.gz djvu | abstract |
`On Thresholds of Codes'.

Experimental work on regular Gallager codes

[See also earlier papers on Gallager codes and MN codes.]
R3.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | R3.ps.gz. abstract.
` Two Small Gallager Codes '. with Matthew Davey
sensit.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | sensit.ps.gz. abstract.
` Sensitivity of Low Density Parity Check Codes to Decoding Assumptions '.
David J.C. MacKay and Christopher P. Hesketh.
margulis.ps.gz. margulis.pdf. | margulis.DJVU. abstract. | <- UK | Canada -> | margulis.ps.gz. margulis.pdf. | margulis.DJVU. abstract.
` Weaknesses of Margulis and Ramanujan-Margulis Low-Density Parity-Check Codes '.
David J.C. MacKay and Michael S. Postol.

High Rate Codes

seagate.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | seagate.ps.gz. abstract.
` Evaluation of Gallager Codes for Short Block Length and High Rate Applications '. with Matthew Davey.
highrate.ps.gz. highrate.pdf. abstract. | <- UK | Canada -> | highrate.ps.gz. highrate.pdf. abstract.
` Gallager Codes for High Rate Applications '.
Research note written with Ed Ratzer, Jan 2003. Looks in more detail at high rate codes.

Irregular Gallager codes and Repeat-Accumulate Codes

ldpc-irreg.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | ldpc-irreg.ps.gz. abstract.
` Comparison of Constructions of Irregular Gallager Codes '.
David MacKay, Simon Wilson and Matthew Davey.
Published in IEEE Transactions on Communications (submitted 30 July 1998) and in Proceedings of the 1998 Allerton Conference on Communication, Control, and Computing.
irreg-its.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | irreg-its.ps.gz. abstract.
` Decoding Times of Irregular Gallager Codes '.
ra-its.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | ra-its.ps.gz. abstract.
` Decoding Times of Repeat-Accumulate Codes '.
puncture.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | puncture.ps.gz. abstract.
` Punctured and Irregular High-Rate Gallager Codes '.

Turbo codes

BPTD.ps.zip (old USA site). BPTD.ps (USA master site). BPTD.ps.gz (UK mirror, not nec. up to date). BPTD.ps.gz (Canada mirror).
Turbo Decoding as an Instance of Pearl's `Belief Propagation' Algorithm.
R. J. McEliece, D. J. C. MacKay and J.-F. Cheng. In IEEE Journal on Selected Areas in Communication.
turbo-ldpc.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | turbo-ldpc.ps.gz. abstract.
` Turbo Codes are Low Density Parity Check Codes '.
This material was included in other papers, such as the big one (mncN.ps.gz).
tcc-al.ps.gz. | PDF | | DJVU | no abstract. | <- UK | Canada -> | tcc-al.ps.gz. no abstract.
` Trellis-constrained codes '.
Brendan J. Frey and David J.C. MacKay. Presented at Allerton 1997.
postscript. | <- UK | Canada -> | abstract.
` Irregular turbocodes '. with Brendan Frey. (in Proceedings of the 37th Allerton Conference on Communication, Control and Computing 1999, Allerton House, Illinois.)
G4096.ps.gz. G4096.pdf. | G4096.DJVU. abstract. | <- UK | Canada -> | G4096.ps.gz. G4096.pdf. | G4096.DJVU. abstract.
` (4096,3249) Gallager Codes compared with Tanner Product Codes '. `

Runlength limiting codes, & codes for correcting insertions and deletions

PDF | | DJVU | abstract. | <- UK |
` "Watermark codes": reliable communication over channels with insertions, deletions, and substitutions '. Matthew Davey and David MacKay. One page paper in ISIT2000.
`Reliable communication over channels with insertions, deletions, and substitutions.' Paper appeared in IEEE-IT Feb 2001. See also Matthew Davey's web pages
ERRATUM: (19th March 2001) In the published version of the above IEEE-IT paper, figure 15 contained a minor error, now corrected on the file on this website. Both the dotted `Zigangirov' curves were a tiny bit higher than they ought to be. You can see the difference between the old curves and the corrected curves in this figure and you can also obtain the corrected curves or the entire corrected page.
The error has no influence on the substance of the paper.
This error arose because our definition of the deletion rate (pd) differs by a factor (1-pi) from that of Zigangirov, where pi is the insertion rate.
Thanks to Dave Leigh.
rll01.ps.gz. rll01.pdf. | rll01.DJVU. abstract. | <- UK | Canada -> | rll.ps.gz. rll01.pdf. | rll01.DJVU. abstract.
` Almost-certainly Runlength-limiting Codes '.
in proceedings of the IMA Cryptography and Coding Conference 2001, copyright Springer
rllt.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | rllt.ps.gz. abstract.
` An Alternative to Runlength-limiting Codes: Turn Timing Errors into Substitution Errors '.
istc2000.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | istc2000.ps.gz. abstract.
` Codes for Channels with Insertions, Deletions and Substitutions '. by Ed Ratzer and David MacKay. Presented at ISTC 2000, Brest. [See also

Quantum Error-Correcting Codes

As submitted to journal: qecc.ps.gz. qecc.pdf. abstract. | <- UK | Canada -> | qecc.ps.gz. qecc.pdf. abstract.
`Sparse Graph Codes for Quantum Error-Correction'.
by David MacKay, Graeme Mitchison, and Paul McFadden.
quant-ph/0304161. IEEE Transactions on Information Theory, vol 50, no 10, in press.
Extended version: qeccE.ps.gz. qeccE.pdf. abstract. | <- UK | Canada -> | qeccE.ps.gz. qeccE.pdf. abstract.
`Sparse Graph Codes for Quantum Error-Correction'.
by David MacKay, Graeme Mitchison, and Paul McFadden.
quant-ph/0304161. Extended version of paper appearing in IEEE Transactions on Information Theory.

Early work on Cryptanalysis and Coding

fe.ps.gz. | PDF | | DJVU | abstract | ps mirror, Canada |
`A Free Energy Minimization Framework for Inference Problems in Modulo 2 Arithmetic'. Also a short version of the same paper: fes.ps.gz, | PDF | | DJVU | | ps mirror, Canada | appeared in Electronics Letters, title `A Free Energy Minimization Algorithm for Decoding and Cryptanalysis'. Or a short version, including pseudocode in appendix: fem.ps.gz. | PDF | | DJVU | | ps mirror, Canada |
Submissions to ISIT97 (all rejected)
muleisit.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | muleisit.ps.gz . abstract.
` Shortened Turbo Codes '. (A one-page summary)
scc.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | scc.ps.gz . abstract.
` Decoding Shortened Cyclic Codes by Belief Propagation '. (with Simon T. Wilson) (One-page summary)
gfq.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | gfq.ps.gz . abstract.
` Low Density Parity Check Codes over GF(q) '. (One-page summary) (with M.C. Davey)

Discussion documents

replica.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | replica.ps.gz. abstract.
` Thresholds of Low-Density Parity Check Codes (Discussion Document) '.
bethe.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | bethe.ps.gz. abstract.
` Questions about the Bethe Free Energy and Sum-Product (Discussion Document) '.
With Jonathan Yedidia, William Freeman, and Yair Weiss
xword.ps.gz. xword.pdf. abstract. | <- UK | Canada -> | xword.ps.gz. xword.pdf. abstract.
` Counting Crosswords '. by David J C MacKay and Jeremy Thorpe

Other applications of Message-Passing

Papers written by students in my group

pumphreySAT.pdf
Solving the Satisfiability Problem using Message-Passing Techniques
by Sam J Pumphrey (2001)
young-maxind.pdf
Solving Combinatorial Problems using Message-Passing
by Ben Young (2001)

PhD Thesis (Caltech 1991) and related papers:

My thesis consisted of four published papers, one unpublished chapter, and `summary' and `postscript' chapters. If you are only interested in one published chapter then you might prefer to download the single `Neural Computation' files listed after each chapter below. The thesis version is to be preferred, however, as it contains slightly more information. The whole thing, postscript | djvu | pdf
Contents
pp. i-vi (30K); |DJVU|
Chapter 1: Summary
pp. 01-06 (40K); |DJVU|
Chapter 2: Bayesian Interpolation
pp. 07-17 (86K); |DJVU|
pp. 18-25 (68K); |DJVU|
pp. 26-33 (91K); |DJVU|
or as published in Neural Computation 4 3 415-447 (188K). |DJVU|
Chapter 3: A Practical Bayesian Framework for Backprop Networks
pp. 34-43 (94K); |DJVU|
pp. 44-52 (71K); |DJVU|
or as published in Neural Computation 4 3 448-472 (144K).| |DJVU|
Chapter 4: Information-based objective functions for active data selection
pp. 53-64 (122K); |DJVU|
or as published in Neural Computation 4 4 589-603 (125K). |DJVU|
Chapter 5: The evidence framework applied to classification networks
pp. 65-69 (85K); |DJVU|
pp. 70-70 (195K); |DJVU|
pp. 71-78 (191K); |DJVU|
or as published in Neural Computation 4 5 698-714 (456K). |DJVU|
Chapter 6: Inferring input-dependent noise levels
pp. 79-81 (34K); |DJVU|
Chapter 7: Postscript
pp. 82-86 (39K); |DJVU|
Bibliography
pp. 87-92 (30K). |DJVU|

Bayesian methods for neural networks

See also my PhD thesis
network.ps.gz, | PDF | DJVU | abstract | ps mirror, Canada |
`Probable Networks and Plausible Predictions - A Review of Practical Bayesian Methods for Supervised Neural Networks'
This is the final version of a review paper based on a book chapter in a Springer publication entitled "Bayesian methods for backpropagation networks"; it appeared as a commissioned review article in Network (IOPP).
cpi_short.ps.gz | PDF | DJVU | | ps mirror, Canada |
Bayesian Non-Linear Modelling with Neural Networks
A review paper giving a basic introduction to neural networks and then describing Bayesian methods, with two case studies.
nips91.ps.gz.(9 pages) | | PDF | DJVU | | ps mirror, Canada| pdf. | DJVU. |
` Bayesian Model Comparison and Backprop Nets '.
NIPS paper reviewing Bayesian methods, and including discussion of the relationship to the `Effective VC dimension' in Structural Risk Minimization.
cpi4.ps.gz.(791K,43 pages) | PDF | DJVU | abstract. | ps mirror, Canada |
`Bayesian Methods for Neural Networks: Theory and Applications'.
Course notes for Neural Networks Summer School
hme.ps.gz up to date links are here
`Bayesian methods for Mixtures of Experts'
Steve Waterhouse, David MacKay and Tony Robinson
objective.ps.gz. | PDF | DJVU | abstract. | <- UK | Canada -> | objective.ps.gz. abstract.
` Searching for `optimal' inputs with an empirical regression model '.
nnbm.ps.gz. | PDF | DJVU |
`The Nonnegative Boltzmann Machine' with Oliver Downs and Daniel Lee. (NIPS 99)

Applications of Bayesian neural networks

pred.ps.gz, | PDF | DJVU | abstract | ps mirror, Canada |
Bayesian Non-linear Modeling for the Energy Prediction Competition
(see also the Language, Proteins, Density nets page)

Independent Component Analysis (and other latent variable models)

ica.ps.gz. | PDF | DJVU | abstract. | <- UK | Canada -> | ica.ps.gz. abstract.
` Maximum Likelihood and Covariant Algorithms for Independent Component Analysis '. (see also Barak Pearlmutter's work)
Unpublished paper.
PhD Thesis on Bayesian Independent Component Analysis
by James Miskin
icagenes.ps.gz | pdf | DJVU. abstract. | <- UK | Canada -> | icagenes.ps.gz. | pdf. | djvu. abstract.
` Latent Variable Models for Gene Expression Data '.
With James Miskin

Generalized Boltzmann machines / Contrastive Divergence

gbm.ps.gz | pdf | DJVU | abstract. | <- UK | Canada -> | gbm.ps.gz. pdf. | DJVU. abstract.
`Failures of the One-Step Learning Algorithm'.

`Boosting'

boost.ps.gz. boost.pdf. abstract. | <- UK | Canada -> | boost.ps.gz. boost.pdf. abstract.
`Likelihood-based Boosting'. With Radford Neal (Unpublished paper, 1998)

Computational Implementation of Bayesian methods

alpha.ps.gz, | PDF | DJVU | abstract | ps mirror, Canada | pdf | djvu |
Hyperparameters: optimize, or integrate out?
This paper appeared in shortened form in the 1993 Maxent proceedings. The long version underwent a lengthy refereeing process with Neural Computation, where it was published (1999) under the title Comparison of Approximate Methods for Handling Hyperparameters.
laplace.ps.gz. | PDF | DJVU | abstract. | <- UK | -> Canada | laplace.ps.gz. abstract.
`Choice of Basis for Laplace Approximation'.
To appear in Machine Learning.
delve.ps.gz. | PDF | DJVU | abstract. | <- UK | Canada -> | delve.ps.gz. abstract.
` More sensitive tests in DELVE'.

See also

Monte Carlo methods

erice.ps.gz. | PDF | DJVU | abstract. | <- UK | Canada -> | erice.ps.gz. abstract.
` Introduction to Monte Carlo methods '.
A review paper in the proceedings of an Erice summer school, ed. M.Jordan.
paris.ps.gz. | PDF | DJVU | abstract | ps mirror, Canada. abstract |
` Probabilistic Networks: New Models and New Methods '.
A paper describing an interpolation model with input-dependent noise level, implemented using `BUGS'. This paper is for ICANN95 in Paris. For more information about the BUGS program, see ftp.mrc-bsu.cam.ac.uk.
astro.ps.gz, | PDF | DJVU | astro.tar | ps mirror, Canada |
Inferring the distance to the Virgo cluster from Cepheid data using the BUGS program. (Bayesian inference using Gibbs Sampling). Currently this sketched report uses mock data.
mnr4997.pdf
Infrared magnitude-redshift relations for luminous radio galaxies
K. J. Inskip, P. N. Best, M. S. Longair, and D. J. C. MacKay
This is an electronic version of an article published in Monthly Notices of the Royal Astronomical Society: complete citation information for the final version of the paper, as published in the print edition of Monthly Notices of the Royal Astronomical Society, is available on the Blackwell Science Synergy online delivery service, accessible via the journal's Website at: http://www.blacksci.co.uk/MNR.
Exact sampling from antiferromagnetic spin systems
Ryan Patterson, Andrew Childs, Radford Neal and David MacKay. This paper was published in Phys Rev E vol 63, 036113, under the title Exact Sampling from nonattractive distributions using summary states.
slice.ps.gz | pdf | DJVU. abstract. | <- UK | Canada -> | slice.ps.gz | pdf | DJVU | abstract.
` Slice Sampling - a Binary Implementation '.
By John Skilling and David MacKay
To appear in Annals of Statistics
nested.ps | nested.pdf |
` Nested Sampling - a tutorial '.

Variational methods

ensemble.ps.gz. | PDF | DJVU | abstract. | ps mirror, Canada |
` Developments in Probabilistic Modelling with Neural Networks - Ensemble Learning '.
nips.ps.gz. | PDF | DJVU | abstract | ps mirror, Canada |
`Ensemble Learning and Evidence Maximization'.
ensemblePaper.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | ensemblePaper.ps.gz. abstract.
` Ensemble Learning for Hidden Markov Models '.
vgc.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | vgc.ps.gz. abstract.
` Variational Gaussian Process Classifiers '.
With M.N. Gibbs.
minima.ps.gz. minima.pdf. | minima.DJVU. abstract. | <- UK | Canada -> | minima.ps.gz. minima.pdf. | minima.DJVU. abstract.
` A Problem with Variational Free Energy Minimization '. (first published Sat 23/6/01)

Gaussian processes

abstract. | <- UK | Canada -> | gpros.ps.gz . abstract.
` Efficient implementation of Gaussian processes. '.
With M.N. Gibbs. Draft paper. was `Efficient methods for Interpolation with Gaussian Processes'
See also Mark's Gaussian process web site
vgc.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | vgc.ps.gz. abstract.
` Variational Gaussian Process Classifiers '.
With M.N. Gibbs.
See also Mark's Gaussian process web site
gp.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | gp.ps.gz. abstract.
` Gaussian Processes - A Replacement for Supervised Neural Networks?'.
Lecture notes for a tutorial at NIPS 1997. More about Gaussian processes.
gpB.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | gpB.ps.gz. abstract.
` Introduction to Gaussian Processes '.
This is a later version of the above lecture notes, to appear in proceedings of a NATO school.
(see also the ICA & Latent Variable Models page)

Language modelling and Hidden Markov models

lang4.ps.gz, | PDF | | DJVU | abstract | ps mirror, Canada |
`A Hierarchical Dirichlet Language Model.' with L.Peto
amino.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | amino.ps.gz. abstract.
` Models for Dice Factories and Amino Acid Probabilities. '. (Unfinished technical report from 1994; this work led to my work on density networks for protein sequences.)
Also, an unfinished addendum, amino_is.ps.gz. | PDF | | DJVU |

Hidden Markov models

jordan.ps.gz. abstract. | <- UK | Canada -> | jordan.ps.gz. | PDF | | DJVU | abstract.
` Equivalence of Linear Boltzmann Chains and Hidden Markov Models '.
ensemblePaper.ps.gz. abstract. | <- UK | Canada -> | ensemblePaper.ps.gz. | PDF | | DJVU | abstract.
` Ensemble Learning for Hidden Markov Models '.

Protein sequence modelling and Density Networks

ch_learning.ps.gz. abstract. | <- UK | Canada -> | ch_learning.ps.gz. | PDF | | DJVU | abstract.
` Bayesian Neural Networks and Density Networks '.
density.ps.gz, | PDF | | DJVU | abstract | ps mirror, Canada |
`Density Networks and their application to Protein Modelling' (this version appeared in Maximum Entropy Proceedings)
density97.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | density97.ps.gz. abstract.
D. MacKay and M. Gibbs (1997): ` Density Networks '. (this version in proceedings of Edinburgh meeting, ed. Jim Kay)

Modelling of images and radar

nn_im_decon.ps.gz, | PDF | | DJVU | abstract | ps mirror, Canada |
`Neural Network Image Deconvolution', by John E. Tansley, Martin J. Oldfield and David J.C. MacKay
radar3.ps.gz. | PDF | | DJVU | abstract | ps mirror, Canada |
`Bayesian analysis of linear phased-array radar' by A.G. Green and D.J.C. MacKay.
i3.ps.gz (494K). | PDF | | DJVU | abstract | ps mirror, Canada |
A. Barnett and D. MacKay: `Bayesian Comparison of Models for Images'.

Neuroscience Data Analysis

newint.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | newint.ps.gz. abstract.
D. MacKay and R. Takeuchi: `Interpolation models with multiple hyperparameters'. (Published in Statistics and Computing)
shuffle.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | shuffle.ps.gz. abstract.
` A Comment on Data Shuffling '.
by David J. C. MacKay, Christopher deCharms and Virginia R. de Sa.
cnssumm.ps.gz. | PDF | <- UK | USA -> | | DJVU | <- UK | USA -> | Ginny's site |
Old draft: bridge.ps.gz. abstract. | <- UK | Canada -> | bridge.ps.gz. abstract.
` Model fitting as an Aid to Capacitance Compensation and Bridge Balancing in Neuronal Recording '.
by David J. C. MacKay and Virginia R. de Sa. - Publication details:
de Sa, V.R., & MacKay, D.J.C. (2001). Model fitting as an Aid to Bridge Balancing in Neuronal Recording. Neurocomputing (special issue devoted to Proceedings of the CNS 2000 meeting) Vol38-40, 1651-1656.
  • Time-warp-invariant computation with action potentials: Deductions about the Hopfield-Brody Mouse by David MacKay and Seb Wills
    In December 2000, the Inference Group won Hopfield and Brody's `mouse brain' competition.

    • Dynamical Neural Networks

    dynet.ps.gz. | PDF | | DJVU | | <- UK | Canada -> | dynet.ps.gz. | Abstract (in Germany)
    ` A Recurrent Neural Network for Modelling Dynamical Systems '.
    by Coryn A.L. Bailer-Jones, David J.C. MacKay, Philip J. Withers

    Papers for which postscript files are not available here, but may be available in Materials Science

    H.K.D.H. Bhadeshia, D.J.C. MacKay, and L.E. Svensson.
    Impact toughness of C-MN steel arc welds - Bayesian neural network analysis.
    Materials Science and Technology, 11(10):1046-1051, 1995.

    L.Gavard, H.K. D.H. Bhadeshia, D.J.C. MacKay, and S.Suzuki.
    Bayesian neural network model for austenite formation in steels.
    Materials Science and Technology, pages 453-463, 1996.

    H.Fujii, D.J.C. MacKay, and H.K. D.H. Bhadeshia.
    Bayesian neural network analysis of fatigue crack growth rate in nickel base superalloys.
    ISIJ International, 36(11):1373-1382, 1996.

    T.Cool, H.K.D.H. Bhadeshia, and D.J.C. MacKay.
    The yield and ultimate tensile strength of steel welds.
    Materials Science and Engineering A, A223:186-200, 1997.

    J.Jones, J.King, H.K.D.H. Bhadeshia, and D.J.C. MacKay.
    Modelling the strength of Nickel base superalloys.
    3rd International Parsons Turbine Conference, 1995.

    T.Cool, H.K.D.H. Bhadeshia, and D.J.C. MacKay.
    Modelling the mechanical properties in the HAZ of power plant steels i: Bayesian neural network analysis of proof strength.
    In H.Cerjak, editor, Mathematical Modelling of Weld Phenomena 3, Materials Modelling Series, pages 403-442. Institute of Materials, London, 1997.

    D.J.C. MacKay.
    Bayesian non-linear modelling with neural networks.
    In H.Cerjak, editor, Mathematical Modelling of Weld Phenomena 3, Materials Modelling Series, pages 359-389. Institute of Materials, London, 1997.

    A.Y. Badmos, H.K.D.H. Bhadeshia, and D.J.C. MacKay.
    Neural network models for the tensile properties of mechanically alloyed ODS iron-alloys. Materials Science and Engineering A. Accepted for publication, 1997.

    F. Brun, T. Yoshida, J. D. Robson, V. Narayan, H.K.D.H. Bhadeshia and D. J. C. MacKay.
    Theoretical Design of Creep Resistant Steels. 1997 (Submitted).
    And many more....

    Decision theory

    _decision.ps | pdf | DJVU
    Decision theory - a simple example

    Fair Elections

    RFSpdf.pdf | pdf master site
    A maximum entropy approach to fair elections
    Roger Sewell, David MacKay, Iain McLean

    Human-computer Interfaces

    Dasher - a Data Entry Interface Using Continuous Gestures and Language Models

    David J Ward, Alan F Blackwell and David J C MacKay

    Existing devices for communicating information to computers are bulky, slow to use, or unreliable. Dasher is a new interface incorporating language modelling and driven by continuous two-dimensional gestures, e.g. a mouse, touchscreen, or eye-tracker. Tests have shown that this device can be used to enter text at around 25 words per minute, compared with typical ten-finger keyboard typing of 40 - 60 words per minute.

    Although the interface is slower than a conventional keyboard, it is small and simple, and could be used on personal data assistants and by handicapped computer users.

    In UIST 2000: The 13th Annual ACM Symposium on User Interface Software and Technology

    Compressed postscript file. | PDF | Mastercopy of Abstract

    The inference group Dasher site

    eyeshortpaper.ps.gz. eyeshortpaper.pdf. | eyeshortpaper.DJVU. abstract. | <- UK | Canada -> | eyeshortpaper.ps.gz. eyeshortpaper.pdf. | eyeshortpaper.DJVU. abstract.
    ` Hands-free Text Entry using Inverse Arithmetic Coding with an Eyetracker '.
    eyeflyer.ps.gz. eyeflyer.pdf. | eyeflyer.DJVU. abstract. | <- UK | Canada -> | eyeflyer.ps.gz. eyeflyer.pdf. | eyeflyer.DJVU. abstract.
    ` Hands-free Text Entry using Dasher with an Eye-tracker '.
    One-page flyer by David J. Ward and David J.C. MacKay.

    dasherButtons.ps.gz. dasherButtons.pdf. abstract. | <- UK | Canada -> | dasherButtons.ps.gz. dasherButtons.pdf. abstract.
    ` Efficient communication through one or two buttons '.
    by David MacKay, Chris Ball, and Mick Donegan.

    David's Dasher site and experimental results

    A note about the iTap / T9 text-to-numbers code

    Computational Neuroscience

    My first journal paper, Analysis of Linsker's simulations of Hebbian rules. (D. J. C. MacKay and K. D. Miller)
    Network 1, 257-298,
    is not conveniently available online.
    Here is a text-only version of this paper (postscript). Here is a tar file of gzip-ed tiff or postscript files of the figures. More information.

    The Role of Constraints in Hebbian Learning. (Kenneth D. Miller and David J.C. MacKay)
    More information and pointers to a technical report. postscript (UCSF,USA) | postscript (Cambridge,UK).
    Hyperacuity and Coarse Coding - a short paper that I never got round to publishing
    postscript | PDF | | DJVU | | dvi | ps mirror, Canada | dvi mirror, Canada |
    timing.ps.gz. | <- UK | Canada -> | | PDF | | DJVU | timing.ps.gz.
    ` Associative memory using action potential timing '.
    Roland Muller, David MacKay and Andreas Herz
    advance.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | advance.ps.gz. abstract.
    ` Time Warp Invariance by Recoding Time Delays into Time Delays '.
    Unpublished research note, March 1996.
    spike_inf.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | spike_inf.ps.gz. abstract.
    ` Hierarchical Odor or Speech Recognition - Bayesian Inference embodied in Neuronal Spikes '.
    Unpublished research note, March 1996.
    Computation with action potentials: Deductions about the Hopfield-Brody Mouse (html document)
    David MacKay and research group.
    October 2000.

    See also the Neuroscience Data Analysis page.

    Evolution

    gene.ps.gz. | PDF | | DJVU | abstract. | <- UK | Canada -> | gene.ps.gz. abstract.
    ` Rate of Information Acquisition by a Species subjected to Natural Selection '.
    or, `Why sex is good for your bits'

    Early work

    A Method of Increasing the Contextual Input to Adaptive Pattern Recognition Systems, Technical Report RIPR 1000/14/87, Research Initiative in Pattern Recognition, Royal Signals and Radar Establishment, Malvern, Worcester. WR14 3PS
    (My first paper on neural networks.)
    DJVU files: Readable, but poor quality (130K) high-resolution version (2M) zip file containing huge tif images
    connectionsM.ps.gz | PDF | |DJVU |
    My first paper on statistics and neural networks,
    Maximum Entropy Connections: Neural Networks,
    was published in Maximum Entropy and Bayesian Methods, Laramie, 1990, pages 237-244.
    I don't have the original postscript, but I have slapped together the above postscript document (which includes an unnecessary 7 page bibliography) to preserve the contents of the paper on line.

    Mathematical recreations

    rectangles.ps.gz. rectangles.pdf. html. abstract. | <- UK | Canada -> | rectangles.ps.gz. rectangles.pdf. html. abstract.
    `Simple Proofs of a Rectangle Tiling Theorem'. (2003)
    corner.ps.gz. corner.pdf. abstract. | <- UK | Canada -> | corner.ps.gz. corner.pdf. abstract.
    ` Big Things that Squeeze Round Corners '.
    conway.ps.gz. conway.pdf. abstract. | <- UK | Canada -> | conway.ps.gz. conway.pdf. abstract.
    I edited a 15-page book chapter, `The Power of Mathematics', by John Conway.

    Other Fun stuff

    Codes
    A note about the iTap / T9 text-to-numbers code
    Games
    Bao: An illustration of what research in theoretical physics is like

    Bayesian methods, quickies

    `The pope is (probably) not an alien' | <- UK | Canada -> | html
    With Sean Eddy
    euro.ps.gz. euro.pdf. | euro.DJVU. abstract. | <- UK | Canada -> | euro.ps.gz. euro.pdf. | euro.DJVU. abstract.
    Belgian euro coins: 140 heads in 250 tosses -- suspicious?

    Problems using this archive? Some questions and answers.

    Note : If you prefer to get documents by ftp rather than by clicking in a web browser, then you can get to most of these files by anonymous ftp to wol.ra.phy.cam.ac.uk, cd pub/www/mackay.

    Conversely, if you want to get a document by http rather than by ftp, change the address from ftp://www.inference.phy.cam.ac.uk/pub/www/mackay/blah to http://www.inference.phy.cam.ac.uk/mackay/blah.

    when I logged into your web address as sent, I got a bunch of nondescript characters which at least my Nestscape 1.12 was not deciphering.
    OK - that means that your Netscape doesn't know about uncompressing files ending in .ps.gz and displaying them with ghostview. The thing to do in that situation is to select "Save As file" from the "File" menu, saving to name.ps.gz, then uncompress (gunzip) it by hand and then view the postscript in your favourite manner.
  • Do you have another question about Finding things on our web server? | or any other topic?
  • I mirror my archive in Canada thus

    Not yet available

    Automatic Relevance Determination (ARD) by MacKay and Neal.
    This paper is not available, and will perhaps never be completed. But please see pred.ps and network.ps above, and buy Radford Neal's book (Springer, 1996); all the information is there.
    • The Inference Group is supported by the Gatsby Foundation
    and by a partnership award from IBM Zurich Research Laboratory
    David J.C. MacKay
    Site last modified Mon Aug 23 18:20:36 BST 2004