Election polling had a pretty good rep until 2016. But it turns out they weren't far off even then. It's really the media driving the narrative. Learn all about how election polling works today.
Learn more about your ad-choices at https://www.iheartpodcastnetwork.com
See omnystudio.com/listener for privacy information.
No summary for this episode yet.
This is an iHeart Podcast. Guaranteed human. Banking with Capital One helps you keep more money in your wallet with no fees or minimums on checking accounts and no overdraft fees. Just ask the Capital One Bank Guy. It's pretty much all he talks about, in a good way. He'd also tell you that this podcast is his favorite podcast too. Thanks, Capital One Bank Guy. What's in your wallet? Terms apply. See CapitalOne.com slash bank. Capital One N-A, member FDIC. It's time for Cyber Monday. Dell Technologies' biggest sale of the year. Enjoy the lowest prices of the year on select PCs like the Dell 16 Plus featuring Intel Core Ultra processors.
And with built-in advanced features, it's the PC that helps you do more faster. Plus, earn Dell rewards and enjoy many other benefits like free shipping, price match guarantee, and expert support. They also have huge deals on accessories that pair perfectly with your Dell PC and make perfect gifts for everyone on your list. Shop now at dell.com slash deals. Welcome to Stuff You Should Know, a production of iHeartRadio's How Stuff Works. Hey, and welcome to the podcast. I'm Josh Clark, and there's Charles W. Chuck Bryan over there, and we've got the scoop. Jerry's around here somewhere. And this is Stuff You Should Know, off to a great start.
She's in her office. She is. She's got this remote thing going on. Yeah. Yeah. It's like the COVID special. That's right. And this was, this has been one I've been wanting to do since 2016. Yeah, why? And it seems like the fire kind of went down on it, and now it's the fire is back up again in election season. And I thought no better time than to talk election polling and this weird sort of black magic, which is really not black magic at all. And as we'll see, the polling wasn't even really that off in 2016. No, it was great. There was a furious fear. We'll talk about it in a second. But there's a furious reaction by the media just left polling and pollsters out to dry saying, like, you, you're terrible.
Your whole craft is useless. You lied to us. As the pollsters went back after election night on 2016, which, by the way, was a bit of a surprise to everybody involved. I think so, including the president. Yeah, when the pollsters went back and looked at their stuff, they said, wait a minute, no, this is all fine. It was you guys, media. You screwed up. You don't know what polling is or what it does. Or how to talk about it, most importantly. Yeah, and then you, public, you have no idea what's going on. You just see some percentages and you automatically leap to some conclusions. And this is way off.
So it's in part that the media was misrepresenting it. Some polls weren't very good. And then the public in general just needs to be a bit more educated on statistics to understand what they're hearing. And that's what we're here for. Because I took statistics three times in college, the same course. At Georgia? At Georgia. I took one of those classes. I hated it. Intro to statistics, right? Yep, boy, I hated that class. The professor, finally, I walked up to her on the last day of the third time, was like, please. And she bumped my D up to a C, and I never looked back. She'd say, you have a one in four chance.
And you're like, but what does that mean? Right. What is four? But so if I can understand this after doing some research, then anybody can understand at least the gist of it enough to understand polling and not be taken in by bad representation of what poll results are. Yeah. So if you remember in 2016, there were pollsters saying, or I'm sorry, and I'm going to say that wrong over and over again. You had media saying that Hillary Clinton is going to win in a landslide. She's got an 85% chance to win, some said as high as 95. She's going to win the popular vote by three percentage points. All the battleground states in the Midwest, she's going to win those narrowly.
And it did not work out that way. And like you said, there was a furor over how could everyone be this wrong with the polling. And there's a man named Nate Silver who everyone probably knows at this point. Yeah. Who has made his name as a data specialist and runs the 538 blog and said, you know what? polling is flawed. And that's probably the first thing that everyone should understand is all polling is a little bit flawed. State polling is definitely a little more flawed than national polling. But here's the deal, everybody. These polls from 2016 were not only not so far off, but historically dating back to since 1972, they actually performed a little better than a lot of elections.
Yeah. And the state polling, while worse than average, wasn't that far off from the average error rate. So what do you want? So there's a lot of stuff. Like we said, there was a lot of postmortem that was done on the 2016 polls and what was gotten wrong and what was gotten right. And we'll talk about that later. But the point is that overall it wasn't that far off. And so the idea isn't that the polls fail or that there's something inherently flawed with polling or that there's even something inherently wrong with the media. Like I want to go on record here, especially in this climate, the media is not our enemy.
Like any healthy democracy needs a vital, robust, independent media as free from bias as an objective to reality and good and justice as possible. But there's also such a thing as a 24-hour news cycle and you've got to fill that. And that's given the rise of opinion news and pundits and basically trying to capture as much market share as possible, which is definitely the wrong track for media in general. But I just want to go on record while we're going to be kind of beating the media up a little bit. That does not mean that the media is inherently flawed or evil or seeks to kill you and your family and your family dog.
So Silver goes back and a bunch of people go back and look at history and kind of what went wrong here in 2016 as far as the polling goes. He says, you know what? We went back for the past 12 presidential cycles since 1972. And he said the polling error was 4.1. He said in 2016, that national polling error was 3.1. So technically by a full point, it was a full point better. He said, we predicted that she would win the popular vote by three percentage points. She actually did win the popular vote by two percentage points. The state polls were the real difference maker. They actually did underperform at a 5.
2 error rate. And that doesn't sound like that much. I think the overall error rate for state polls since 1972 was 4.8. So 4.8, 5.2 doesn't sound like much. But if you're talking about a percentage of error in just a handful of swing states that can make something look like a landslide, even though you lose a popular vote, that's exactly what happened. Right. That's exactly what happened. Because you've got to remember, Trump didn't win the popular vote. He won the electoral college and it came down to those swing states. But the fact that they were off just by 0.4 points from the average for the error rate goes to show you just how close that race actually was, which, again, is the opposite of how it was being broadcast throughout the election.
It was supposed to be a landslide. Like Hillary Clinton might as well just be like taking measurements for curtains in the Oval Office right now. Like it was just that set. So it was presented one way when in reality, if you really looked at the polls and the polling results, if you looked at them with a sober face, it was a much closer race than it appeared or than it was being broadcast. I haven't had a sober face since that night. So we should talk about the margin of error in polling. Anytime you see a poll, they talk about the margin of error. It's usually plus or minus three or four. And that is on each side.
So for each candidate's poll, in other words, it could be a potential like seven to eight point swing and still be within that margin of error. So when Trump is winning states by a 0.2 percent margin or a 0.5 or a 0.7 percent margin, that's well, well, well, well within the margin of error. Right, right. So that margin of error, by the way, is just built in. we'll talk about it a little more in a little bit, but it's like there's just no way around it. To get around any margin of error, you would have to literally go through and interview every single voter in America and then compile the evidence or their data perfectly without any miskeys or anything like that.
And it's just impossible. So everyone accepts that any poll is going to have a margin of error, but you want to keep it within plus or minus three points, maybe four. Yeah. So a little history of polling. We've always been pretty spellbound by polls in this country. We put a lot of stock in polls, especially the presidential race. The word straw poll, if you've ever heard that, that comes from the idea that you hold up a piece of straw to see which way the wind is blowing. So a straw poll is kind of like, here's how things stand today on something. Like this is the way the wind is blowing today on this matter.
Yeah, and they're just kind of informal. They used to take them like on train cars as journalists would ask people who they were on the train with, who they're going to vote for. Nothing like formal or anything, but it does kind of reveal how longstanding our fascination with polls really, really is. Yeah, it got pretty serious in the 1930s, specifically the 1936 election, where a literary digest, it was a pretty big magazine at the time, polled its subscribers. It's just kind of funny even seeing this sentence. They predicted a landslide win for Republican Alf Landon over FDR. So if you've never heard of Alf Landon, you know why.
because Alf Landon did not beat FDR. And the magazine's editor said, you know what? We didn't even think about the fact that we just polled our subscribers and that there are wealthy people, or at least wealthier on average, and they're probably going to vote Republican. So Alf Landon was their man. Right. So if you go out even today and just interview Republicans and say, hey, who you're going to vote for, and then take that result and apply it to the entire population of the nation, You've got a flawed poll, and that's what Literary Digest did. But in doing so, they established this kind of, they pointed out a real design flaw that now is just one of the first basic things that anybody conducting a poll gets rid of.
That's right. Gallup came on the scene. They galloped onto the scene. So sorry. And they were one of the first big polling companies to say, all right, we've got to get this right. We've got to get a representation of all of America here. So we're going to send our people door to door. We're going to go to every zip code in America. And they did that from 1935 to 1984 and got basically within about three percentage points, doing a pretty good job, but it was really expensive. So in the 80s, in the mid-80s, they switched to calling people on the telephone. Yeah, which, I mean, that's still today.
That is the gold standard is for a human being to dial up another human being and ask them some questions. And we'll talk a little more about it. But what Gallup does and what Pew does and what a few others do is it's called randomized sampling or probability sampling, which is where you basically leave it to chance that any voter, registered voter in America is going to get a phone call from you. So what Gallup is doing and what Pew does is called randomized sampling or probability sampling, where any voter in America has an equal chance of receiving a phone call from Gallup or from Pew and being asked these questions.
And it worked pretty well for a while when they moved from in-person over to the phone because they were still asking people questions and they could still get their answers and harass them, which is a big thing, as we'll see, about this type of sampling. The problem is when people started to use caller ID, they stopped picking up the phone as much. And so the response rate went down dramatically. Yeah, so they would call people using random digit dialing, which is a computer system, where it fed in an area code and then the first three digits, and then randomly dialed the last four. So you've got a pretty good start there on the random sampling.
But even then, they said, you know what, women tend to answer the phone more than men. So to truly randomize it, whenever, whoever picks up the phone, we have to then follow up and say, we want to talk to the person in the house who's had the most recent birthday. Right. Further randomizing, I got kind of a laugh about this because I don't know that I've ever, literally ever seen my father pick up a telephone in his life. Yeah. Or at least growing up. For the first 18 years of my life, I don't think I ever saw him answer the phone. It's all ham radio, huh? No, he went into that. But just literally, not one time, he would just let it ring if no one was around, if my mom wasn't around to answer it.
And granted, it was usually never for him. No one ever called to talk to him. But I picked up on that. And my friends used to get really frustrated back before texting that I would just never answer my phone. And I always just thought it was an option. Like when the phone rings, it doesn't mean you're obligated. It just means now you have an option. You can answer it or not. Well, technically, that's true. I mean, like it depends. You don't have to answer the phone, but it depends on, you know, who in your life could possibly be calling. Well, I didn't think it was rude or anything. I just thought it was literally like, you know, I'm going to hedge my bets here that one of my friends isn't stuck on the side of the road.
Right. They can leave a message, and if they are, I'll go get them. So what you're talking about, Chuck, is what's called a non-response. And that's factored into the response rate, which with phone polling from 1980 until the 1990s, it was manageable. I think the response rate peaked at 36% in 1997, which was good. Not bad. Now it's down to like 9% because like I said, people have caller ID and if some unknown number is calling, you typically don't answer. And that actually affects things because there is a certain kind of person who answers the phone no matter what. And they are not like every single American.
And that actually factors into the kind of poll you're conducting. Plus, also, you want like a certain amount of responses. I think out of a sample size, you want a minimum of 800 survey responses. And back in the day when you got a 36% response rate, meaning 36% of those people you called would answer the phone and go through all of the questions and answer them fully and complete the survey. since it was down to 9%, you went from having to call between 2,000 and 2,500 people to up to 9,000 people now just to get 800 surveys completed. And that made the whole thing a lot more expensive. On the one hand, because it was expensive, it meant that there were fewer and fewer companies that could conduct these polls, which meant that the polls you were seeing were more and more legitimate.
But on the other hand, it also usually decreased sample size a little bit Because as Gallup pointed out, like, you can kind of fiddle with the numbers a little bit with a smaller response rate and smaller sample size. Yeah, and it also led to robocalls because of expense, because of people not answering their phone as much. And those systems, I mean, I love how Dave Roos put it. He said they range from okay to terrible in how well they work, online polls and these other new techniques. But I think we should take a break and then talk about what I found the very interesting way that they further randomized this thing from this point forward right after this.
you know everyone living with a rare autoimmune condition has their own story to tell and that's why in season five of untold stories life with a severe autoimmune condition a ruby studio production in partnership with Argenics, you'll hear powerful real-life perspectives. That's right. This podcast explores stories of what life is really like with MG or CIDP. Host Martine Hackett sits down with people who faced it all, the early signs and symptoms, the search for answers, and the strength it takes to keep moving forward. Yeah, and this season, the stories go even deeper, showing us what resilience truly looks like through setbacks, breakthroughs, and the communities that make all the difference.
So listen to Untold Stories, Life with a Severe Autoimmune Condition, on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Attention parents and grandparents. If you're looking for a gift that's more than just a toy, give them something that inspires confidence and adventure all year long. Give them a Guardian bike, the easiest, safest, and number one kid's bike on the market. Yeah, with USA-made kids-specific frames and patented safety technology, kids are learning to ride in just one day with no training wheels needed. It's why Guardian is America's favorite kids' bike and the New York Times and Wirecutter's top pick three years in a row.
That's right. My daughter has a Guardian bike, and she loves it, and that thing was really easy to put together. And get this. This holiday season, Guardian is offering their biggest deal of the year, over 40% in savings on all bikes, plus $100 in free accessories. Guardian bikes have become one of the most sought-after gifts of the season, and inventory is going fast, so don't wait. Join over a half a million families who've discovered the magic of Guardian. Visit guardianbikes.com to shop now. Learning stuff with Joshua and Charles. Stuff you should know. All right. So we've already talked about the fact that they've randomly called someone, and then they take one further step on that call by saying, let me speak with whoever had the most recent birthday, even if it's, I guess, your three-year-old.
Right. And one other thing I kind of made mention to it that I have to interject, dude. Like harassing people, like if you've been picked by this computer, if your phone number has been picked, they're going to keep calling you and calling you. And that is because as a person who doesn't normally participate in phone surveys, you are a specific kind of person that you can't be left out of the population because you represent a large number of people and they want your opinion. So part of this phone standard of calling people is to call them over and over and over again to basically harass them into participating to get their answers for the survey because it's as important, if not more important sometimes, than the people who are like, oh, yeah, I'd love to answer this phone survey.
Two totally different kinds of people. Yeah, absolutely. And I was totally kidding, by the way, to the listener when I said they will speak to a three-year-old. They ask the most recent birthday of someone of voting age, obviously. Right. So then you've got a pretty decent random sampling to begin with. And then you have to start the process of waiting, which comes in a lot of different forms. If you want an example of like a really good political poll, it's going to be paid for by a neutral source. It's not going to be like a CNN poll or a Fox News poll or a Super PAC or anything like that. You're going to have a random sample of the public, which we just talked about.
You're going to be dialing cell phones and landlines these days. That's a big one. Also, they'll ask you if you have a cell phone and a landline, and if you say, yes, I have both, they're going to adjust your response based on the fact that you had a higher chance of being selected because you have two numbers that the computer could have picked. Right. And another thing is, like you mentioned, they're going to keep calling you. The best ones use live interviewers still. And then what they want to do, and this last one is really important, is they're going to try and improve the accuracy of the results by weighting the response to match.
What they want to do is just match a real-world demographic, age, race, your income level, your education level, and all of that stuff is factored in, and all this stuff is weighed out because, well, we'll talk about it, but, you know, there are many different kinds of Americans, and if you want a really good sampling of different kinds of Americans, you're going to, like you said, have to fidget with the numbers to make it a true representative population. Right. So because even if you just get it exactly right demographically and weighted, which, like you said, we'll talk about some more in a second, you still have that margin of error.
And again, that's that, you know, 52 percent plus or minus three points. And that means that it could be 55 percent or it could be 49 percent. They don't know. But somewhere between that, most of your answers are going to be the correct answer somewhere in there. That's what that means with that margin of error. And the reason that that's built in is because it is basically impossible to perfectly represent the larger population through random sampling. You're just not going to pick everybody correctly just by the fact that it's random and it's a sample. Yeah. And that's important because like that's why you hear so much hay being made over a double digit lead in a poll, which Biden had sort of semi recently.
I know it's gotten a lot tighter since then. But, you know, when Biden was up by, I think, like 10 percentage points, people were flipping out because, you know, like we said, it's plus or minus four for each candidate. So that's a total of eight. And so basically the press started screaming like he's outside of the margin of error, everybody. Like nothing can beat them. Right, right, yeah. But now things are back within that margin. I saw on PBS NewsHour, they interview Mark Shields and David Brooks. Brooks is a New York Times columnist, and I think Mark Shields is an independent columnist. And one of them actually said, and this was in July, America has clearly made up its mind on who's going to be the next president.
And I was like, this is July. Did you not learn anything from 2016? I couldn't believe that those words came out of their mouth. It's irresponsible. And they said it's so matter-of-factly. Yeah, it's irresponsible. And there's been studies about this, too, that have suggested that words like that, that polls that say 99% chance of winning, that this kind of stuff actually has a negative impact on the leader because it makes people think, well, I don't need to go out and vote. Everybody else is going to go vote. And the turnout might be lower than otherwise. There's also people who dispute that.
They say, yes, it makes sense intuitively and anecdotally, but we've yet to actually see genuine data that says clearly that this has this effect. But it's something that's still being studied right now, whether it actually does or not. Well, and I also saw an article the other day about the quote-unquote silent majority and that another reason those polls were so wrong back then and they're saying are probably wrong now is because there are – they say that there's a substantial block of voters who very privately and secretly vote for Trump. Yeah, the term for them among pollsters is shy Trump voters.
They won't admit that they're going to vote for Trump, but they're going to vote for Trump and that that affects polls. I saw that that's actually not been proven to actually exist. But I think it was a Pew. There's a really great Pew article. If this stuff is speaking to you at all, go check out Pew's key things to know about election polling in the U.S. And it has a bunch of great links that you should follow in there. And there's also Sciline. They have surveys and polling, which is a guide for journalists to polling. but I found out you don't actually have to be a journalist to read it online.
So if you want to go check those out, they have some great, like, just some breakdowns of some of the stuff we're talking about, but also about how to read polls and what to trust and look for in general. And a little known fact, Pew was actually originally called Pew Pew until 1976 when Star Wars came out. And they were like, we've got to change our name now, guys. Yeah. Can't do it. Man, it is. It is Dad-o-rama today with you, huh? So back to the weighting thing. And by the way, we should mention that Gallup said if they wanted to increase that sample size and actually get the margin of error down to like plus or minus two, that they could do that.
But that would be like a literal 100% increase in the cost. So like everyone just please live with plus or minus three or four points. Yeah, and everybody generally does. And Dave uses this really good example. Dave Ruse helped us out with this. And he said this margin of error is best understood where if you selected 100 marbles, there's a jar of 500 red and 500 blue marbles, and you pick out 100 of them, you might pick out 50 of each one time. Wait, 500? What? You said 500 marbles? Oh, no, I'm sorry, 1,000 marbles. I've lost my marbles. Yes, there's 1,000 marbles, okay? And 500 are red and 500 are blue.
Your task, Chuck, is to pick out 100. So you go to the trouble picking out 150, red, 50, or blue, and I say, do it again. And this time it's 47 and 53. And you keep saying, again, again. Right. And I smack my writing crop on the desk that you're sitting at. And I do it 100 times. And you get super turned on. Yeah, I do it 100 times because dear leader told me to. Right. And at the end, you get a little bell curve and basically a plus or minus four. Right. So, yeah, almost all of them. this is what's a 95% confidence interval. Almost all of them are going to fall in that bell curve. There's going to be some outliers.
There's going to be that one time where it was just absolutely insane. You actually picked 100 red marbles randomly blindfolded from this jar. That's so insignificant statistically. It's just such an outlier. But almost all of them are going to be in there. So when you're polling like this large group of people, like American voters, and 95% of them are falling within a couple of percentage points of either side of this middle, you can pretty much feel confident about that. And that is the basis of election polling, of political polling, of all polling, really, that they have this built-in margin that they know exists, but everybody can live with it.
The problem is when you're hovering around that 50% mark and you're talking about a two-party system, one of them has like 51% and the other one has 49%, but there's a plus or minus of like two points. That means flip a coin, America. It means we have no idea. And some people would say, well, why even do polling? Because what you're showing there is not who's going to win. That's not the point of polling. The point of polling is to take a snapshot of how America or whoever you're polling is feeling that moment about who to elect, about what laws to pass, about religion, about the Cleveland Indians.
It doesn't matter, right? Like the, that's what a poll does. But you can pervert polls into making them talk a different language and say, hey, look at this percentage. You take these polls, you convert them into something else. Now you have something like a 95% chance this person is going to win. Go shout that, Wolf Blitzer. And Wolf Blitzer goes and shouts it as loud as he can. So we need to talk a little bit more about waiting. I mentioned earlier that there's other things they do to sort of tip the scale. And that sounds like a bad term, so I guess I shouldn't say it that way. But things they do to make it equitable and a true representative of the American population, for instance.
African-American voters make up 12 percent of voters. So if they did a poll and in the end they only got 6 percent of respondents that were African-American, then they just double it basically. If the respondents were overwhelmingly Caucasian, they would weight that down to their true representative number, which is about, I think, 66 percent. Yeah, the electorate is white. And if 80 percent of white people respond or 80 percent of the people that respond are white, then they're going to kick that down. And again, this is just adjusting the poll to the proper weight so you have a really legitimate snapshot.
And, you know, if it sounds crazy that they're using a thousand people's responses and drawing that out to the size of the voting population of America, it is. But if you're a statistician, it isn't. You know what I mean? No, it reliably works as long as you present it with plus or minus this margin of error. It sounds crazy as just an average Joe on the street. It does. To be like, they ask a thousand people and we're supposed to know and extrapolate that. And a statistician who are number wonks and data wonks would say, yeah, that's exactly what that means. Shut up. That's really all you need.
But it really is a testimony to the power of those statistics and that data and the analysis of them. Yeah, weighting is really important. It goes far beyond just like age, political party. I think Gallup uses eight different variables. The New York Times, the Siena College poll uses 10. and they include things like marital status and home ownership. Pew uses 12 variables. They ask things like, do you have home internet access? Do you engage in volunteerism? And all of these things have been shown to be associated. So if you're a white woman age 65 to 75 who volunteers twice a month. And lives in the suburbs.
You're a very specific person where there's a group of people out there who vote like a certain way and you represent like all those people with that. So they'll weight the results based on these additional questions that you're answering. They don't just ask you, are you going to vote for Trump or Biden? And there's also built into that question, a really important point. Are you going to vote? Yeah, that's a huge thing we haven't talked about. It's one thing to poll registered voters. But here in America, somehow presidential elections only get about 60% turnout still, which is just shameful and crazy.
But that's another podcast. But so most of the really, really good polls drill down. And to get a real, real good representation of what might actually happen, they try to drill down to whether or not you're most likely to actually vote. Right. Because who cares what your opinion is if you're not going to vote? And I mean, they generally take your word for it that you're telling the truth. you know. Yeah. But they do have like nine, I think Pew, yeah, Pew has nine questions that they basically use to establish that you are planning on voting. Like you're actually going to vote. You're not full of hot air, you know.
Yeah. I don't know what those questions are, but I imagine they have to do with, do you know where your polling place is? Do you have transportation? Stuff like that. I was thinking they were going to be like, are you really, really going to vote? Was like question three. and they just kept adding reallys. Right. Pew Pew. So you've got these people who've been called and they have answered these questions and they have participated in this survey whether they wanted to or not. And they've finally done it. Built into that margin of error, built into this poll is that understandable margin of error that just comes from the fact that it's a randomized sample, right?
But what Pew and any other legitimate group, polling group, will point out is that the margin of error is actually greater than that. That the margin of error for the average poll, according to Pew, is that it's something more like six points. Not three or four. It's actually six. And the reason why is built on top of that margin of error that's automatically part of the poll just by the virtue of it being a randomized sample are things like the person typing in the wrong key accidentally. Human error. Those kind of things add up. Or that the question isn't worded clearly enough that anybody who hears it knows the intent and knows what their answer is, that there's some sort of miscommunication involved.
There's also things that they can't control for, like people who have pseudo opinions who don't want to sound dumb. So they just answer yes or no based on something they really don't care about either way. And because they don't actually have an opinion that actually that weights things the wrong way. So when you add all these stuff, these things up, you have these additional errors. Yeah. That lead to different to like a bias overall in the poll. which can affect the outcome. But again, the companies that have the money to conduct like these genuinely big gold standard polls, they know enough to know how to kind of factor control for those as much as possible.
But still, what Pew says is, if you're listening to a poll and somebody says plus or minus three points, you should probably go ahead and double that in your mind. Double it in your mind. Double your pleasure, double your fun. Double your margin error. So let's take a break, and we're going to come back and talk about what exactly they think went wrong with those state polls right after this. you know everyone living with a rare autoimmune condition has their own story to tell and that's why in season five of untold stories life with a severe autoimmune condition a ruby Studio Production, in partnership with Argenics, you'll hear powerful, real-life perspectives.
That's right. This podcast explores stories of what life is really like with MG or CIDP. Host Martine Hackett sits down with people who faced it all, the early signs and symptoms, the search for answers, and the strength it takes to keep moving forward. Yeah, and this season, the stories go even deeper, showing us what resilience truly looks like through setbacks, breakthroughs, and the communities that make all the difference. So listen to Untold Stories, Life with a Severe Autoimmune Condition, on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Attention parents and grandparents.
If you're looking for a gift that's more than just a toy, give them something that inspires confidence and adventure all year long. Give them a Guardian bike, the easiest, safest, and number one kid's bike on the market. Yeah, with USA-made kids-specific frames and patented safety technology, kids are learning to ride in just one day with no training wheels needed. It's why Guardian is America's favorite kids' bike and the New York Times and Wirecutter's top pick three years in a row. That's right. My daughter has a Guardian bike, and she loves it, and that thing was really easy to put together.
And get this, this holiday season, Guardian is offering their biggest deal of the year, over 40% in savings on all bikes, plus $100 in free accessories. Guardian bikes have become one of the most sought-after gifts of the season, and inventory is going fast, so don't wait. Join over a half a million families who've discovered the magic of Guardian. Visit guardianbikes.com to shop now. Learning stuff with Joshua and Charles. Stuff you should know. All right. So I think it's generally acknowledged that 2016, and again, I want to say the polling was off, but apparently the polling wasn't off, but the way it was reported on was off.
But what really happened in 2016, what was off was the state polling. And what they think, they've, like you said, gone back and obsessed over these polls since then. You know, because they were already statistical wonks. But when something like this happens, they really sort of get worked into a dander and have to get to the bottom of it. I mean, people were calling for the end of polling. They just said it was a failed profession, basically. And Pew was like, I'm getting rich off this, man. Yeah. We can't end polling. Jimmy Pugh was like, stop. Stop talking like that. So what happened in 2016 is, they think, is that a lot of non-educated white people came out in big, big numbers for Donald Trump.
And that was a sort of a new, not a new factor because they had always talked about college education, but a new factor in how outsized of a factor that was. It had never been that outsized. And so all these state pollsters, they didn't wait it and they didn't adjust their polls to reflect this fact that college educated people are more likely going to respond to these surveys. So their polls were just off. Yeah, and they knew that college-educated people were more likely to respond to the surveys. That wasn't news to them. What caught them sleeping was that they had not picked up on the fact that this group of people, non-college-educated white voters, were going to go to the polls in numbers like never before and that they were going to vote for Trump.
They did not pick up on that. That was brand new. Like, that didn't exist before. Trump basically brought up a new electorate that helped get him elected, especially in battleground states like Wisconsin and Michigan and Indiana. Although I think in Indiana he was a shoe in because of Pence. But this group of voters that did not exist or this line between college educated and non-college educated white voters, that partisan gulf hadn't existed before Election Day. The pollsters didn't pick up on it, and so they didn't wait those responses because they never had to wait the responses before based on college education.
Yeah, so suburbs, exurbs, and especially the rural vote counted like it had never counted before, which is obviously why you see what's going on right now. Like a very hard push by the Trump campaign to get these same people out again in the way that they do that. That's the nicest way I can put it. It genuinely is. So, yeah. So the idea that this was already kind of a close race, a closer race than was being broadcast, that these electoral, huge electoral battleground states that got flipped, that was basically the reason that Trump was able to take the electoral college. But the idea is that these voters kind of came out of nowhere and voted for Trump and that there were some other things that happened, too, that the pollsters didn't anticipate.
One, that the undecided voters, people who'd said I'm legitimately undecided at this point a week before the election, from what I read, they broke hard in favor of Trump on election day. When they made their decision, they voted for Trump. That hadn't been predicted. That was another big one. And then one of the other things, too, is that the polls were just doing what polls do, which is sometimes they're right, sometimes they're wrong. Poll's going to poll. But polls had gotten so good in the 2000 aughts that people came to be overconfident in their ability to predict and pick winners. And the 2016 race reminded us, like, polling's not perfect.
Let's stop pretending it is. Yeah, and a lot of it has to do with, like we've been kind of harping on, the way the media presents it. And then a lot of it has to do with just our conditioned, how we're conditioned to look at things like underdogs. And it's different in politics. And I remember when these aggregators, especially at 538, they had these predictive models and they started talking about the fact that, and I think the Washington Post even wrote a good comparison to sports. and, you know, if someone is a real big underdog going into like a Super Bowl or a World Series and they end up winning, people don't get angry and go after the people who said they had a 15 or 20 percent chance of winning.
They just said, wow, what a story. The underdog won. Right. But there are so few presidential elections, you know, one every four years that it's the same thing, but people just look at it differently. like an underdog like trump was an underdog that supposedly had like a 15 to 30 chance of winning some people said one yeah well that's ridiculous but a 30 chance of winning is a real shot at winning for sure yeah the way it's framed it doesn't seem that way in politics no and so that's one thing but another thing is that we shouldn't even be talking about presidential elections with like 15 chance of winning 99 chance of winning like that is not how we should present it.
And that's not how we used to present it. We used to present it saying like this poll found that Clinton was going to lead Trump 52% to 48% or something like that, plus or minus two points. And that would have shown you like, oh, okay, well, this is a really close race, way closer than I think. And that's that. There's my information. And the problem is that you can take that same statistic, 52% chance of winning, plus or minus a four-point margin of error, if you convert that to a normal distribution, you come up with an 84% probability of a win. That's the problem, is that the statistics that are being, the data that's being produced by these polls are being converted in ways that they shouldn't be.
And then that's what the media jumps on. That's what the public laps up because that is the horse race statistic, an 84% chance of winning, a 15% chance of winning. That's what we think about. That's what we look at. And so rather than realizing that actually this is a close race, 52% plus or minus four points, we see 84% chance of winning. And that's a foregone conclusion that that person's going to win. That ultimately is where the media and the public are culpable for this. Yeah. I don't think they were meant to be extrapolated like that to begin with. No, they weren't. And that, you know, polls are valuable, but like I haven't looked at any polls.
And partially because of the way 2016 went down. And in fact, for the past week, I've taken a complete internet news and social media break. And it's been pretty great, actually. Oh, man, it's so liberating. Yeah, I mean, I literally haven't looked at a single news thing. I very sadly found out that Chadwick Boseman passed away like three days afterward. Oh, wow. Like that's how dark I've gone in not looking at the Internet unless it's something that brings me joy, which is to say, you know, old Led Zeppelin and Van Halen YouTube videos. I was looking up classic Mad Magazine covers of the 80s the other day.
That's all I've been doing is if it doesn't bring me joy on the Internet, I'm not doing it. That's good. And, you know, I got to break that soon because I do think you should be active and involved in the know. Yeah, but taking a break fairly regularly is definitely mentally healthy for sure. But that aside, I haven't – I'm not looking at any polls. I don't care what any poll says. Well, see, so I was thinking very similar stuff too. And, like, what's the point of polls? I don't know. Okay, well, I finally found it. If you look that up on Google, there's just very little on it. But I found somebody who explained it pretty well, I thought, that polls aren't meant to tell you who's going to win.
They're not forecasting models. Like I said before, they're meant to be like a snapshot of how whoever you're polling feels at the moment, right? And in doing that, because you are sampling American people, and these are independent news organizations typically who are carrying out these polls, you get to tell everybody else how America's feeling rather than the leaders saying, I'll decide how you're feeling. I can decide what you want and what you need and what you think is important. And polls prevent that from happening by telling the rest of the people, hey, this is how everybody else is feeling right now, too.
And in some ways, it is kind of sheeple-ish where, you know, the idea is like, oh, you know, is that supposed to sway my opinion that everybody's going to vote for this person and not for that person? That should have no bearing or impact on your vote. And it feels like that that's how polls are used sometimes. But if you step back and look and see that they're actually kind of an important part of sharing what other people are thinking rather than being told what we're thinking or, you know, what to think, then they actually are pretty legitimate in that sense. Yeah, well, you know, I say take your polls and sit on it.
Well, one more thing. We cannot talk about polling and not talk about Internet polling real quick. This is a completely different style of polling than it's ever been done before. Rather than a randomized sample, you actually just say, hey, you want to take the survey? And people click it. So it's called opt-in surveying. And very specific kinds of people take surveys on purpose on the internet. So they really are, because they're new, they're really now figuring out how to weight these things or not. And how to use them. because they can produce legitimate data, but it depends on who's conducting the poll, if they know what they're doing, that kind of stuff.
But just like everything else, the moving things online is democratized polling. And so anybody can conduct a poll now and basically enter the news cycle. That's how Kid Rock almost became a senator in Michigan for a second there. So on the one hand, it's good, but it's also we're in a big period of disruption as far as polling is concerned. So for you, the polling consumer, either go like Chuck and just stop listening to polls altogether or look for things like transparency. Do you recognize the company or the name that produced the poll? Are they sharing their data, like how the questions exactly were worded, what their population size was, how they weighted, all this stuff.
If all that stuff is included, you can probably trust the poll. And then beyond that, just remember what you're looking at, that this isn't a predictor of who's going to win. It was a snapshot for a very brief moment of a very specific sample of America to show how people would vote right then. And it was right then, too. This is not election day we're talking about. Yeah, and I want to be clear. I'm not poo-pooing polls. I just, they're valid and useful, but I just don't care to look at them right now. I understand. Yeah, that's my jam. Well, you got anything else about polls? Nothing else about polls.
Well, if you want to know about polls, start looking around and go check out Pew's stuff and Sciline stuff and all that stuff. And since I said stuff three times, here comes Rumpelstiltskin. Or Candyman. So this is from Keeley Price. And Keeley says this. Hi, guys. I'm writing today not only to confess my unending love for Stuff You Should Know, but also to share a link to some black-owned bookstores. It would be so cool if all of your listeners purchased your book, she should just say period, comma from a black-owned bookstore. Couldn't agree more, by the way. A couple of podcasters that I listen to while I wait for stuff you should know have books out and coming out soon, and they encourage their listeners to support black-owned businesses through the purchase of their book.
It's a win-win. I don't know why it's taken me so long to think to write this to you guys. I blame it on Corona Madness. But last but not least, I'll say I love the end of the world with Josh Clark and Movie Crush as well. Any chance to hear you guys talk is a chance worth taking. When we get a COVID vaccine and you guys can do your live shows again, please come to Nashville. Oh, yeah, for sure. I think we planned on Nashville. Yeah, Nashville got scuttled by COVID this time around. We were going to come. Now we might not ever be able to come. No. I know it's super close to Atlanta. I'd lose my mind if I got to see you guys here.
All the best, Keely Price. And so Keeley sent a link to a handy website that lists black-owned bookstores near you. I made a little URL shortener to make it easier on everyone. Oh, let's have it. So you can go to bit.ly slash S-Y-S-K-B-L-M. Nice. And find black-owned bookstores near you to purchase Stuff You Should Know, an incomplete compendium of mostly interesting things. At the very least, we like to encourage people to go to IndieBound.org and support indie bookstores. I don't know if there is an actual black-owned indie bookstore website, but I would imagine most of the black-owned bookstores are indie bookstores.
Yeah, probably. So check it out, bit.ly slash S-Y-S-K-B-L-M. Go out and buy our book, everybody. You're going to love it. It's really great. And thanks for that, Chuck. And thanks for setting us up for that too, Keeley. Much appreciated. We'll see you in Nashville. I guess Keely will be the one, like she said, losing her mind in the crowd. If you want to lose your mind on us via email, we love that kind of thing, kind of. You can send it off to stuffpodcasts at iheartradio.com. Stuff You Should Know is a production of iHeart Radio's How Stuff Works. For more podcasts from iHeart Radio, visit the iHeart Radio app.
Apple Podcasts are wherever you listen to your favorite shows. Living with an autoimmune condition isn't easy. and every journey is different. That's why Season 5 of Untold Stories, Life with a Severe Autoimmune Condition from Ruby Studio and Argenix shares powerful firsthand stories from people with conditions like MG and CIDP. Hosted by Martine Hackett, these conversations dive into what resilience really looks like through setbacks, breakthroughs, and finding strength in community. Listen on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts. Attention parents and grandparents.
If you're looking for a gift that's more than just a toy, give them something that inspires confidence and adventure all year long. Give them a Guardian bike, the easiest, safest, and number one kid's bike on the market. Yeah, with USA-made kid-specific frames and patented safety technology, kids are learning to ride in just one day with no training wheels needed. It's why Guardian is America's favorite kid's bike and the New York Times and Wirecutter's top pick three years in a row. That's right. My daughter has a Guardian bike, and she loves it, and that thing was really easy to put together.
And get this. This holiday season, Guardian is offering their biggest deal of the year, over 40% in savings on all bikes, plus $100 in free accessories. Guardian bikes have become one of the most sought-after gifts of the season, and inventory is going fast, so don't wait. Join over a half a million families who've discovered the magic of Guardian. Visit guardianbikes.com to shop now. This is an iHeart Podcast. Guaranteed Human