{"id":434,"date":"2020-02-20T18:02:58","date_gmt":"2020-02-20T18:02:58","guid":{"rendered":"https:\/\/www.allendowney.com\/blog\/?p=434"},"modified":"2023-10-02T13:29:46","modified_gmt":"2023-10-02T13:29:46","slug":"correlation-determination-and-prediction-error","status":"publish","type":"post","link":"https:\/\/www.allendowney.com\/blog\/2020\/02\/20\/correlation-determination-and-prediction-error\/","title":{"rendered":"Correlation, determination, and prediction error"},"content":{"rendered":"\n

This tweet<\/a> appeared in my feed recently:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

I wrote about this topic in Elements of Data Science<\/a><\/em>\u00a0Notebook 9<\/a>, where I suggest that using Pearson’s coefficient of correlation, usually denoted r, to summarize the relationship between two variables is problematic because:<\/p>\n\n\n\n

    \n
  1. Correlation only quantifies the linear relationship between variables; if the relationship is non-linear, correlation tends to underestimate it.<\/li>\n\n\n\n
  2. Correlation does not quantify the “strength” of the relationship in terms of slope, which is often more important in practice.<\/li>\n<\/ol>\n\n\n\n

    For an explanation of either of those points, see the discussion in Notebook 9<\/a>. But that tweet and the responses got me thinking, and now I think there are even more reasons correlation is not a great statistic:<\/p>\n\n\n\n

      \n
    1. It is hard to interpret as a measure of predictive power.<\/li>\n\n\n\n
    2. It makes the relationship between variables sound more impressive than it is.<\/li>\n<\/ol>\n\n\n\n

      As an example, I’ll quantify the relationship between SAT scores and IQ tests. I know this is a contentious topic; people have strong feelings about the SAT, IQ, and the consequences of using standardized tests for college admissions.<\/p>\n\n\n\n

      I chose this example because<\/em> it is a topic people care about, and I think the analysis I present can contribute to the discussion.<\/p>\n\n\n\n

      But a similar analysis applies in any domain where we use a correlation to quantify the strength of a relationship between two variables.<\/p>\n\n\n\n

      SAT scores and IQ<\/h3>\n\n\n\n

      According to Frey and Detterman, “Scholastic Assessment or g? The relationship between the Scholastic Assessment Test and general cognitive ability<\/a>“, the correlation between SAT scores and general intelligence (g<\/em>) is 0.82.<\/p>\n\n\n\n

      That’s just one study, and if you read the paper, you might have questions about the methodology. But for now I will take this estimate at face value. If you find another source that reports a different correlation, feel free to plug in another value and run my analysis again. <\/p>\n\n\n\n

      In the notebook<\/a>, I generate fake datasets with the same mean and standard deviation as the SAT and the IQ, and with a correlation of 0.82.<\/p>\n\n\n\n

      Then I use them to compute <\/p>\n\n\n\n