{"id":468,"date":"2020-07-26T23:12:28","date_gmt":"2020-07-26T23:12:28","guid":{"rendered":"https:\/\/www.allendowney.com\/blog\/?p=468"},"modified":"2020-07-27T00:51:43","modified_gmt":"2020-07-27T00:51:43","slug":"alice-and-bob-exchange-data","status":"publish","type":"post","link":"https:\/\/www.allendowney.com\/blog\/2020\/07\/26\/alice-and-bob-exchange-data\/","title":{"rendered":"Alice and Bob exchange data"},"content":{"rendered":"\n

Two questions crossed my desktop this week, and I think I can answer both of them with a single example.<\/p>\n\n\n\n

On Twitter<\/a>, Kareem Carr asked, “If Alice believes an event has a 90% probability of occurring and Bob also believes it has a 90% chance of occurring, what does it mean to say they have the same degree of belief? What would we expect to observe about both Alice\u2019s and Bob\u2019s behavior?”<\/p>\n\n\n\n

And on Reddit<\/a>, a reader of \/r\/statistics asked, “I have three coefficients from three different studies that measure the same effect, along with their 95% CIs. Is there an easy way to combine them into a single estimate of the effect?”<\/p>\n\n\n\n

So let me tell you a story:<\/p>\n\n\n\n

One day Alice tells her friend, Bob, “I bought a random decision-making box. Every time you press this button, it says ‘yes’ or ‘no’. I’ve tried it a few times, and I think it says ‘yes’ 90% of the time.”<\/p>\n\n\n\n

Bob says he has some important decisions to make and asks if he can borrow the box. The next day, he returns the box to Alice and says, “I used the box several times, and I also think it says ‘yes’ 90% of the time.”<\/p>\n\n\n\n

Alice says, “It sounds like we agree, but just to make sure, we should compare our predictions. Suppose I press the button twice; what do you think is the probability it says ‘yes’ both times?”<\/p>\n\n\n\n

Bob does some calculations and reports the predictive probability 81.56%.<\/p>\n\n\n\n

Alice says, “That’s interesting. I got a slightly different result, 81.79%. So maybe we don’t agree after all.”<\/p>\n\n\n\n

Bob says, “Well let’s see what happens if we combine our data. I can tell you how many times I pressed the button and how many times it said ‘yes’.”<\/p>\n\n\n\n

Alice says, “That’s ok, I don’t actually need your data; it’s enough if you tell me what prior distribution you used.”<\/p>\n\n\n\n

Bob tells her he used a Jeffreys prior.<\/p>\n\n\n\n

Alice does some calculations and says, “Ok, I’ve updated my beliefs to take into account your data as well as mine. Now I think the probability of ‘yes’ is 91.67%.”<\/p>\n\n\n\n

Bob says, “That’s interesting. Based on your data, you thought the probability was 90%, and based on my data, I thought it was 90%, but when we combine the data, we get a different result. Tell me what data you saw, and let me see what I get.”<\/p>\n\n\n\n

Alice tells him she pressed the button 8 times and it always said ‘yes’.<\/p>\n\n\n\n

“So,” says Bob, “I guess you used a uniform prior.”<\/p>\n\n\n\n

Bob does some calculations and reports, “Taking into account all of the data, I think the probability of ‘yes’ is 93.45%.”<\/p>\n\n\n\n

Alice says, “So when we started, we had seen different data, but we came to the same conclusion.”<\/p>\n\n\n\n

“Sort of,” says Bob, “we had the same posterior mean, but our posterior distributions were different; that’s why we made different predictions for pressing the button twice.”<\/p>\n\n\n\n

Alice says, “And now we’re using the same data, but we have different posterior means. Which makes sense, because we started with different priors.”<\/p>\n\n\n\n

“That’s true,” says Bob, “but if we collect enough data, eventually our posterior distributions will converge, at least approximately.”<\/p>\n\n\n\n

“Well that’s good,” says Alice. “Anyway, how did those decisions work out yesterday?”<\/p>\n\n\n\n

“Mostly bad,” says Bob. “It turns out that saying ‘yes’ 93% of the time is a terrible way to make decisions.”<\/p>\n\n\n\n

If you would like to know how any of those calculations work, you can see the details in a Jupyter notebook:<\/p>\n\n\n\n