Below are some Java applets I've written over the years. Some of these have corresponding research articles. Enjoy!

Common Metropolis-Hastings algorithms

• Metropolis-Hastings Algorithms is an applet which showcases many common Markov chain algorithms. Click on this link to view a short description of the featured algorithms. The applet itself will appear in a separate window.
• Ever wondered what the transition kernels for the M-H algorithms above look like? Look no further, with M-H Roadmap.
• Simulated Annealing is a mutation of the Metropolis-Hastings applet which shows how to use MCMC algorithms to find the global maxima of a given function.

Coupling constructions for Markov chains

• Coupling constructions for Markov chains (in which several chains are run concurrently until and after they meet) are important for the study of convergence to equilibrium. Catalytic coupling is one such construction, based on the following paper (ps/pdf). The method is universal in that implementation requires no preliminary analytical estimates.
• Another way of understanding the coupling technique discussed above is in terms of random fields. Field couplers, which is similar to the M-H Roadmap applet, explains that point of view. It is based on another paper (ps/pdf).

Perfect simulation

• Perfect simulation is the main reason we want to couple Markov chains in this way. While the two applets above give proof-of-concept on quite general toy examples, here are two more realistic examples. The first applet, PumpSimulator, shows how to use catalytic coupling to simulate perfectly from an autogamma model, also known as the Bayesian posterior for the pump dataset. Read the accompanying paper (ps/pdf), or download the source for the second example therein.
• Suppose data is generated from a mixture of (partially) known distributions with unknown weights. This applet and the accompanying paper (ps) showcases how to sample perfectly in simple cases from the posterior distribution on weights using the Data Augmentation algorithm, with polynomial computational cost.
• The PageRank distribution originally used by Google can actually be simulated perfectly. I don't have a Java applet for this, but see here for a C++ project which performs the simulation on just under one million web pages.

Learn more!

The applets above demonstrate several algorithms located at the intersection between Computer Science and Probability Theory. You can learn more about these topics by following the links below.

• The WWW Virtual Library has a page with links to MCMC.
• An online tutorial on Metropolis-Hastings and Gibbs samplers is available from Charlie Geyer. Make sure to read also his rants about burn-in and the length of simulations.
• Molecular Monte Carlo is what physicists and chemists sometimes call these algorithms.
• Once you have written, tested and debugged your own Monte Carlo method, you may wish to download some real random numbers to drive the simulation. Don't confuse these with the pseudo-random numbers generated by your computer.
• To get up to speed on the latest research, there is David Wilson's annotated bibliography on perfect random sampling with Markov chains, and also Steve Brooks' MCMC preprint server.