{"id":442,"date":"2020-04-13T14:15:34","date_gmt":"2020-04-13T14:15:34","guid":{"rendered":"https:\/\/www.allendowney.com\/blog\/?p=442"},"modified":"2020-04-13T14:15:34","modified_gmt":"2020-04-13T14:15:34","slug":"bayesian-hypothesis-testing","status":"publish","type":"post","link":"https:\/\/www.allendowney.com\/blog\/2020\/04\/13\/bayesian-hypothesis-testing\/","title":{"rendered":"Bayesian hypothesis testing"},"content":{"rendered":"\n

I have mixed feelings about Bayesian hypothesis testing. On the positive side, it’s better than null-hypothesis significance testing (NHST). <\/p>\n\n\n\n

And it is probably necessary as an onboarding tool: Hypothesis testing is one of the first things future Bayesians ask about; we need to have an answer.<\/p>\n\n\n\n

On the negative side, Bayesian hypothesis testing is often unsatisfying because the question it answers is not the most useful question to ask. <\/p>\n\n\n\n

To explain, I’ll use an example from Bite Size Bayes<\/a>, which is a series of Jupyter notebooks I am writing to introduce Bayesian statistics.<\/p>\n\n\n\n

In Notebook 7<\/a>, I present the following problem from David MacKay’s book,\u00a0Information Theory, Inference, and Learning Algorithms<\/em><\/a>:<\/p>\n\n\n\n

“A statistical statement appeared in The Guardian on Friday January 4, 2002:<\/p>\n\n\n\n

“When spun on edge 250 times, a Belgian one-euro coin came up heads 140 times and tails 110. \u2018It looks very suspicious to me\u2019, said Barry Blight, a statistics lecturer at the London School of Economics. \u2018If the coin were unbiased the chance of getting a result as extreme as that would be less than 7%\u2019.”<\/p><\/blockquote>\n\n\n\n

“But [asks MacKay] do these data give evidence that the coin is biased rather than fair?”<\/p>\n\n\n\n

I start by formulating the question as an estimation problem. That is, I assume that the coin has some probability, x<\/em>, of landing heads, and I use the data to estimate it.<\/p>\n\n\n\n

If we assume that the prior distribution is uniform, which means that any value between 0 and 1 is equally likely, the posterior distribution looks like this:<\/p>\n\n\n\n

\"\"
Posterior distribution of x<\/em>, which is the probability of heads, given a uniform prior.<\/figcaption><\/figure>\n\n\n\n

This distribution represents everything we know about x<\/em> given the prior and the data. And we can use it to answer whatever questions we have about the coin.<\/p>\n\n\n\n

So let’s answer MacKay’s question: “Do these data give evidence that the coin is biased rather than fair?”<\/p>\n\n\n\n

The question implies that we should consider two hypotheses:<\/p>\n\n\n\n