{"id":1465,"date":"2024-12-20T18:33:33","date_gmt":"2024-12-20T18:33:33","guid":{"rendered":"https:\/\/www.allendowney.com\/blog\/?p=1465"},"modified":"2024-12-23T16:13:43","modified_gmt":"2024-12-23T16:13:43","slug":"political-alignment-and-outlook","status":"publish","type":"post","link":"https:\/\/www.allendowney.com\/blog\/2024\/12\/20\/political-alignment-and-outlook\/","title":{"rendered":"Political Alignment and Outlook"},"content":{"rendered":"\n
\n

This is the fourth in a series of excerpts from Elements of Data Science<\/em>, now available from Lulu.com<\/a> and online booksellers. It’s from Chapter 15, which is part of the political alignment case study. You can read the complete chapter here<\/a>, or run the Jupyter notebook on Colab<\/a>.<\/p>\n<\/blockquote>\n\n\n\n

In the previous chapter<\/a>, we used data from the General Social Survey (GSS) to plot changes in political alignment over time. In this notebook, we\u2019ll explore the relationship between political alignment and respondents\u2019 beliefs about themselves and other people.<\/p>\n\n\n\n

First we\u2019ll use groupby<\/code> to compare the average response between groups and plot the average as a function of time. Then we\u2019ll use the Pandas function pivot table<\/code> to compute the average response within each group as a function of time.<\/p>\n\n\n\n

Are People Fair?<\/h2>\n\n\n\n

In the GSS data, the variable fair<\/code> contains responses to this question:<\/p>\n\n\n\n

\n

Do you think most people would try to take advantage of you if they got a chance, or would they try to be fair?<\/p>\n<\/blockquote>\n\n\n\n

The possible responses are:<\/p>\n\n\n\n

Code<\/th>Response<\/th><\/tr><\/thead>
1<\/td>Take advantage<\/td><\/tr>
2<\/td>Fair<\/td><\/tr>
3<\/td>Depends<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

As always, we start by looking at the distribution of responses, that is, how many people give each response:<\/p>\n\n\n\n

values(gss[\"fair\"])\n<\/pre>\n\n\n\n
1.0    16089\n2.0    23417\n3.0     2897\nName: fair, dtype: int64\n<\/pre>\n\n\n\n
The plurality think people try to be fair (2), but a substantial minority think people would take advantage (1). There are also a number of NaNs, mostly respondents who were not asked this question.<\/p>\n\n\n\n
gss[\"fair\"].isna().sum()\n<\/pre>\n\n\n\n
29987\n<\/pre>\n\n\n\n
To count the number of people who chose option 2<\/code>, \u201cpeople try to be fair\u201d, we\u2019ll use a dictionary to recode option 2<\/code> as 1<\/code> and the other options as 0<\/code>.<\/p>\n\n\n\n
recode_fair = {1: 0, 2: 1, 3: 0}\n<\/pre>\n\n\n\n
As an alternative, we could include option 3<\/code>, \u201cdepends\u201d, by replacing it with 1<\/code>, or give it less weight by replacing it with an intermediate value like 0.5<\/code>. We can use replace<\/code> to recode the values and store the result as a new column in the DataFrame<\/code>.<\/p>\n\n\n\n
gss[\"fair2\"] = gss[\"fair\"].replace(recode_fair)\n<\/pre>\n\n\n\n
And we\u2019ll use values<\/code> to make sure it worked.<\/p>\n\n\n\n
values(gss[\"fair2\"])\n<\/pre>\n\n\n\n
0.0    18986\n1.0    23417\nName: fair2, dtype: int64\n<\/pre>\n\n\n\n
Now let\u2019s see how the responses have changed over time.<\/p>\n\n\n\n
Fairness Over Time<\/h2>\n\n\n\n
As we saw in the previous chapter, we can use groupby<\/code> to group responses by year.<\/p>\n\n\n\n
gss_by_year = gss.groupby(\"year\")\n<\/pre>\n\n\n\n
From the result we can select fair2<\/code> and compute the mean.<\/p>\n\n\n\n
fair_by_year = gss_by_year[\"fair2\"].mean()\n<\/pre>\n\n\n\n
Here\u2019s the result, which shows the fraction of people who say people try to be fair, plotted over time. As in the previous chapter, we plot the data points themselves with circles and a local regression model as a line.<\/p>\n\n\n\n
plot_series_lowess(fair_by_year, \"C1\")\n\ndecorate(\n    xlabel=\"Year\",\n    ylabel=\"Fraction saying yes\",\n    title=\"Would most people try to be fair?\",\n)\n<\/pre>\n\n\n\n
\"_images\/c66dd4e209513c6b52923f0279d558dc7cf98d7002a9608170da7c0372146851.png\"<\/figure>\n\n\n\n
Sadly, it looks like faith in humanity has declined, at least by this measure. Let\u2019s see what this trend looks like if we group the respondents by political alignment.<\/p>\n\n\n\n
Political Views on a 3-point Scale<\/h2>\n\n\n\n
In the previous notebook, we looked at responses to polviews<\/code>, which asks about political alignment. The valid responses are:<\/p>\n\n\n\n
Code<\/th>Response<\/th><\/tr><\/thead>
1<\/td>Extremely liberal<\/td><\/tr>
2<\/td>Liberal<\/td><\/tr>
3<\/td>Slightly liberal<\/td><\/tr>
4<\/td>Moderate<\/td><\/tr>
5<\/td>Slightly conservative<\/td><\/tr>
6<\/td>Conservative<\/td><\/tr>
7<\/td>Extremely conservative<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
To make it easier to visualize groups, we\u2019ll lump the 7-point scale into a 3-point scale.<\/p>\n\n\n\n
recode_polviews = {\n    1: \"Liberal\",\n    2: \"Liberal\",\n    3: \"Liberal\",\n    4: \"Moderate\",\n    5: \"Conservative\",\n    6: \"Conservative\",\n    7: \"Conservative\",\n}\n<\/pre>\n\n\n\n
We\u2019ll use replace<\/code> again, and store the result as a new column in the DataFrame<\/code>.<\/p>\n\n\n\n
gss[\"polviews3\"] = gss[\"polviews\"].replace(recode_polviews)\n<\/pre>\n\n\n\n
With this scale, there are roughly the same number of people in each group.<\/p>\n\n\n\n
values(gss[\"polviews3\"])\n<\/pre>\n\n\n\n
Conservative    21573\nLiberal         17203\nModerate        24157\nName: polviews3, dtype: int64\n<\/pre>\n\n\n\n
Fairness by Group<\/h2>\n\n\n\n
Now let\u2019s see who thinks people are more fair, conservatives or liberals. We\u2019ll group the respondents by polviews3<\/code>.<\/p>\n\n\n\n
by_polviews = gss.groupby(\"polviews3\")\n<\/pre>\n\n\n\n
And compute the mean of fair2<\/code> in each group.<\/p>\n\n\n\n
by_polviews[\"fair2\"].mean()\n<\/pre>\n\n\n\n
polviews3\nConservative    0.577879\nLiberal         0.550849\nModerate        0.537621\nName: fair2, dtype: float64\n<\/pre>\n\n\n\n
It looks like conservatives are a little more optimistic, in this sense, than liberals and moderates. But this result is averaged over the last 50 years. Let\u2019s see how things have changed over time.<\/p>\n\n\n\n
Fairness over Time by Group<\/h2>\n\n\n\n
So far, we have grouped by polviews3<\/code> and computed the mean of fair2<\/code> in each group. Then we grouped by year<\/code> and computed the mean of fair2<\/code> for each year. Now we\u2019ll group by polviews3<\/code> and year<\/code>, and compute the mean of fair2<\/code> in each group over time.<\/p>\n\n\n\n
We could do that computation \u201cby hand\u201d using the tools we already have, but it is so common and useful that it has a name. It is called a pivot table<\/strong>, and Pandas provides a function called pivot_table<\/code> that computes it. It takes the following arguments:<\/p>\n\n\n\n