Computational Modeling
Fall 2005

For today: HomeworkSeventeenPointFive (Bayesian readings)

Today's outline:

0) Bayesian inference, examples and coding


For Monday: HomeworkEighteen


Examples from Chapter 3 of MacKay, 
"Information Theory, Inference, and Learning Algorithms"

(The web page explains that "The same copyright rules will apply to
the online copy of the book as apply to normal books. [e.g., copying
the whole book onto paper is not permitted.]" 

This implies that the principle of fair use applies to the online
book; that is, I can distribute copies of a single chapter for class.

I believe that the copyright notice on the handout is intended to
refer to the whole book.)


To solve the Euro problem, we have to take a few steps.


Step 1: choosing among simple hypotheses

     Exercise 3.1, Exercise 3.4, Exercise 3.12

     Trick 1: don't bother computing denominators

              do all the normalization at the end.

     Trick 2: likelihood ratio is the product of ratios

     Trick 3: don't bother computing priors

     Often we only care about the likelihoods, or the likelihood ratio.

     Trick 4: log likelihood is the sum of log likelihoods


Step 2: inferring (continuous) parameters

     Exercise 3.3
     Exercise 3.5 (bent coin, page 52)

     Trick 5: discretize the distribution of parameters


Step 3: comparing a simple hypothesis with a compound

     Exercise 3.15 (Euro problem)

     Trick 6: the likelihood of a compound is the sum
              of the likelihoods of the components, which
	      is the normalization constant of the update.

Fun problems:

    Exercises 3.8 and 3.9 (Monty Hall problem and variations)

    Exercise 3.11 (Bayes in the courtroom)

    Trick 7: there is no Trick 7