People often ask us how we do our research. We're not going to disclose all of our secrets, but we'll give you some tips on how to root out the bad studies from the good ones. Learn all about shady studies and reporting in this classic episode!
Learn more about your ad-choices at https://www.iheartpodcastnetwork.com
See omnystudio.com/listener for privacy information.
No summary for this episode yet.
This is an iHeart Podcast. Guaranteed human. Living with an autoimmune condition isn't easy, and every journey is different. That's why Season 5 of Untold Stories, Life with a Severe Autoimmune Condition, from Ruby Studio and Argenix, shares powerful firsthand stories from people with conditions like MG and CIDP. Hosted by Martine Hackett, these conversations dive into what resilience really looks like through setbacks, breakthroughs, and finding strength in community. Listen on the iHeart Radio app, Apple Podcasts, or wherever you get your podcasts. Everybody knows Shaq, but off camera, he's just a regular guy.
People never believe me when I say I'm just like them. I take out the trash, do dishes, and I struggle with moderate obstructive sleep apnea or OSA. And a lot of adults with obesity also struggle with moderate to severe OSA. You know those scary breathing interruptions during sleep? The loud snoring, choking, and daytime fatigue? I knew I had to talk to my doctor. Don't sleep on the symptoms. Learn more at dontsleeponosa.com. This information is provided by Lilly, a medicine company. Hey everybody, it's me, Josh. And for this week's SYS case selects, I've chosen our guide to research tips. It's a surprisingly good episode that shares the ins and outs of keeping from being duped online by bad information.
And how to read between the lines on sensational science reporting. All sorts of stuff like that. And you might notice in this episode Chuck sounds different than usual. That's because this is during the period that he was transitioning into a person with a full set of teeth. So that adds to the hilarity of the whole thing. I hope you enjoy this as much as we did making it. Welcome to Stuff You Should Know, a production of iHeartRadio's How Stuff Works. Hey, and welcome to the podcast. I'm Josh Clark with Charles W. Chuck Bryant and Jerry. This is Stuff You Should Know. Josh, we're going to do something weird today.
We're going to do a listener mail at the head of the podcast. What? I know, right? What? All right, let's do it. Okay, this is from Beyond. Wait, wait, hold on. Do we have the listener mail music going? Oh, I don't know. Jerry? Should we go the whole nine yards? Let's do it. People might freak out. I know. All right. This is from Bianca Boisich, is what I'm going to say. I think that's great. Hey, guys. Wrote you not too long ago asking about how you research your own podcast. I just got back from a class where we talked about research misrepresentation in journal articles. Apparently, journals don't publish everything that is submitted.
A lot of researchers don't even publish their studies. They don't like the results. Some laws have been put into place to prevent misrepresentation, such as researchers having to register their studies before they get results. and journals only accepting pre-registered studies, but apparently this is not happening at all, even though it is now technically law. This ends with the general public being misinformed about methods and drugs that work. For example, there are 25 studies proving a drug works and 25 that don't. It's more likely that 20 of the positive results have been published and only one or two of the negative.
And that is from Bianca. and that led us to this article on our own website, 10 Signs That That Study Is Bogus. Yeah. And here it is. Nice, Chuck. Well, we get asked a lot about research from people. Usually in college, they're like, you guys are professional researchers. How do I know I'm doing a good job and getting good info? And it's getting harder and harder these days. It really is. You know? One sign that I've learned is if you are searching about a study and all of the hits that come back are from different news organizations and they're all within like a two, three-day period from a year ago.
Copy-paste. Nothing more recent than that. Then somebody released a sensational study and no one did any actual effort into investigating it and there was no follow-up. If you dig deep enough, somebody might have done follow-up or something like that. But for the most part, it was just something that splashed across the headlines, which more often than not is the case as far as science reporting goes. So that's a bonus. That's the 11th. Boom. How about that? Yeah. Should we just start banging these out? Let's do it. Or do you have some other clever segue? Well, apart and parcel with that, I don't know if it's clever.
You do come across people who you know can be trusted and relied upon to do good science reporting. So like Ed Yong is one. Another guy named Ben Goldacre has something called Bad Science. I don't remember what outlet he's with. And then there's a guy, I think Scientific American, named John Horgan, who's awesome. Yeah, or some journalism organizations that have been around and stood the test of time that you know are really doing it right, like Nature. Yeah, Scientific American. Science. Yeah, I feel really good about using those sources. Yeah, but even they can, you know, there's something called scientism where there's a lot of like faith and dogma associated with the scientific process.
And, you know, you have to root through that as well. All right, I'm done. The first one that they have here on the list is that it's unrepeatable. And that's a big one. The Center for Open Science did a study. It was a project, really, where they took 270 researchers and they said, you know what? take these 100 studies that have been published already, psychological studies, and just pour over them. And in 2015, just last year, it took them a while, took them several years. They said, you know what? More than half of these can't even be repeated using the same methods. They're not reproducible.
Nope, not reproducible. That's a big one. And that means that when they carried out, they followed the methodology. Scientific Method Podcast, you should listen to that one. That was a good one. Yeah. that they found that their results were just not what the people published, not anywhere near them. Yeah. For example, they used one as an example where a study found that men were terrible at determining whether a woman was giving them some sort of like clues to attraction or just being friendly. Yeah. Sexy, sexy stuff or let's be friends. Or yeah, or good to meet you. Yeah. Or buzz off, jerk. Sure.
Yeah. And they did the study again as part of this Center for Open Science study or survey, and they found that that was not reproducible, or that they came up with totally different results. And that was just one of many. Yeah, and in this case specifically, they looked into that study, and they found that one was in the United Kingdom, one was in the United States. Right. That may have something to do with it. But the point is, Chuck, is if you're talking about humanity, I don't think the study was like the American male is terrible at it it's men are terrible at it so that means that whether it's in the UK which is basically the US with an accent and a penchant for T I'm just kidding UK see you soon it should be universal yeah unless you're saying no it's just this only applies to American men these 100 American men Right.
Then it's not even a study. Yeah. The next one we have is it's plausible, not necessarily provable. And this is a big one because, and I think we're talking about observational studies here more than lab experiments. Right. Because with observational studies, you know, you sit in a room and get asked 300 questions about something. And all these people get asked the same questions. And then they pour over the data and they draw out their own observations. Right. And one of the very famously, an observational study that led to false results found a correlation between having a type A personality and being prone to risk for heart attacks.
Yeah. And for a long time, you know that the news outlets were like, oh, yes, of course, that makes total sense. Right. This study proved what we've all known all along. And then it came out that, no, actually what was going on was a well-known anomaly where you have a 5% risk that chance will produce something that looks like a statistically significant correlation. When it's not at all. When really it's just total chance. And science is aware of this, especially with observational studies, because the more questions you have, the more opportunity you have for that 5% chance to create a seemingly statistically significant correlation.
Right. When really it's not there. It was just random chance where if somebody else goes back and does the same study, they're not going to come up with the same results. but if a researcher is, I would guess, willfully blind to that 5% chance, they will go ahead and produce the study and be like, no, it's true, here's the results right here. Go ahead and report on it and make my career. Yeah, well, and they also might be looking for something. In fact, chances are they are. It's not just some random study and like, let's just see what we get if we ask a bunch of weird questions. It's like, hey, we're looking to try and prove something most likely.
So that Bader-Meinhof thing might come into play where you're kind of cherry picking data. Yeah, that's a big problem. That kind of comes up. A lot of these are really kind of interrelated, too. Oh, totally. The other big thing that's interrelated is how the media reports on science these days. Yeah, you know. It's a big deal. Yeah. Like John Oliver just recently went off on this and NPR did a thing on it. That's great. Like they might even, like the researcher might say plausible, but it doesn't get portrayed that way. Oh, yeah. In the media. Sure. Remember that poor kid who thought he found the ancient Mayan city?
The media just took it and ran with it. You know, yeah, I think there was a lot of maybe or it's possible we need to go check kind of thing. The media's like, no, he discovered an ancient Mayan city never known before. Yeah, and let's put it in a headline. And that's, I mean, that's just kind of the way it is these days. Yeah. You have to be able to sort through it. I guess that's what we're doing here, aren't we, Chuck? we're telling everybody how to sort through it, or at the very least, take scientific reporting with a grain of salt. Yes. Right. Like, you don't necessarily have the time to go through and double that research and then check on that research and, you know?
Right. So take it with a grain of salt. Yeah. Unsound samples. Here was a study that basically said how you lost your virginity is going to have a very large impact and play a role on how you feel about sex and experience sex for the rest of your life. Yeah. It's possible. Sure. It seems logical. So we'll just go with it. But when you only interview college students and you don't, you only interview heterosexual people, then you can't really say you've done a robust study now, can you? Plus, you also take out of the sample size or sample population, anybody who reports having had a violent encounter, throw them out, throw that data out.
Because that's not going to inform how you feel about sex. Right, exactly. You're just narrowing it down further and further. And again, cherry picking the data by throwing people out of your population sample that will throw off the data that you want. Yeah, and I'd never heard of this acronym, WEIRD. And a lot of these studies are conducted by professors and academics. So a lot of times you've got college students as your sample. And there's something called weird, Western educated from industrialized, rich and democratic countries. Right. Those are the participants in the studies. Yes. Study subjects.
But then they will say men. Right. Well, what about the gay man in Africa? Right. Like you didn't ask him. So that's actually a really, really big deal. In 2010, these three researchers did a survey of a ton of social science and behavioral science studies. Found that 80% of them used weird study participants. So basically it was college kids for 80% of these papers. And they surveyed a bunch of papers. and they took it a little further and they said that people who fit into the weird category only make up 12% of the world population but they represent 80% of the population of these studies and a college student Chuck in North America Europe, Israel or Australia is 4,000 times more likely to be in a scientific study than anyone else on the planet and they're basing psychology and behavioral sciences are basing their findings onto everybody else based on this small tranche of humanity.
Yeah. And that's a big problem. That's extremely misleading. Yeah. And it's also a little insulting because what they're essentially saying is like, this is who matters. Well, also, yeah. What's sad is this is who I am going to go to the trouble of recruiting for my study. Yeah. It's just sheer laziness. Yeah. And I'm sure a lot of them are like, well, I don't have the funding to do that. I guess I see that. But at the same time, I guarantee there's a tremendous amount of laziness involved. Yeah. Or maybe if you don't have the money, maybe don't do that study. Yeah. Is it that simple? I'm probably oversimplifying.
I don't know. I'm sure we're going to hear from some people in academia about this one. We'll stop using weird participants. Or at the very least say, like, heterosexual Dartmouth students. Yeah, yeah. This applies to them. Right. Not everybody in the world. Exactly. I mean, 80% of these studies use those people as study participants. And they're not even emblematic of the rest of the human race. Like, college students are shown to see the world differently than other people around the world. Yeah. So it's not like you can be like, well, it still works. You can still extrapolate. No, it's like flawed in every way, shape, and form.
We should probably take a break, huh? Yeah, let's take a break because you're getting a little hot under the collar. I love it. Man. We'll be right back after this. Just like the number of stars in the sky, there is so much stuff you should know. Support for the show comes from Public, the investing platform for those who take it seriously. On Public, you can build a multi-asset portfolio of stocks, bonds, options, crypto, and now generated assets, which allow you to turn any idea into an investable index with AI. It all starts with your prompt. From renewable energy companies with high free cash flow to semiconductor suppliers growing revenue over 20% year over year, you can literally type any prompt and put the AI to work.
It screens thousands of stocks, builds a one-of-a-kind index, and lets you backtest it against the S&P 500. Then, you can invest in a few clicks. Generated assets are like EFTs with infinite possibilities, completely customizable and based on your thesis, not someone else's. Go to public.com slash podcast and earn an uncapped 1% bonus when you transfer your portfolio. That's public.com slash podcast. Paid for by Public Investing. Brokerage services by Open to the Public Investing, Inc. member finra sipc advisory services by public advisors llc sec registered advisor generated assets is an interactive analysis tool output is for informational purposes only and is not investment recommendation or advice complete disclosures available at public.
com slash disclosures and now superhuman shack i keep telling them not to say that i'm no superhuman believe it or not i struggle with moderate obstructive sleep apnea or osa in adults with obesity, moderate to severe OSA is a condition where breathing is interrupted during sleep with loud snoring, choking, gasping for air, and even daytime fatigue. Let's just say it can sound a lot like this. Sound familiar? Learn more at dontsleeponosa.com. This information is provided by Lilly, a medicine company. Learning stuff with Joshua and Charles. Stuff you should know. All right, what's next, buddy?
Very small sample sizes. Right. If you do a study with 20 mice, then you're not doing a good enough study. No. So they used this in the article. They used the idea of 10,000 smokers and 10,000 non-smokers. Yeah. And they said, okay, if you have a population sample that size, that's not bad. It's a pretty good start. And you find that 50% of the smokers develop lung cancer, but only 5% of non-smokers did, then your study has what's called a high power. Yeah. If you had something like 10 smokers and 10 non-smokers, and two of the smokers develop lung cancer and one develop lung cancer as well, you have very little power and you should have very little confidence in your findings but regardless it's still going to get reported if it's a sexy idea yeah for sure and because these are kind of overlapping in a lot of ways I want to mention this guy a scientist named Ulrich Dernagel he and his colleague Malcolm McLeod have been trying and there are a lot of scientists that are trying to clean this up because they know it's a problem.
But he co-wrote an article in Nature that's called Robust Research, colon, Institutions Must Do Their Part for Reproducibility. So this kind of ties back into the reproducing things like we said earlier. Yeah. And his whole idea is, you know what, good funding, they should tie funding to good institutional practices. Like you shouldn't get the money if you can't show that you're doing it right. Yeah. And he said that would just weed out a lot of stuff. here's one staggering stat for reproducibility and small sample size. Biomedical researchers for drug companies reported that 25% of their, only 25% of the papers that they publish are even reproducible.
And that was like an insider stat. And it doesn't matter. Drugs are still going to market. Yeah. That's a really good example of why this does matter to the average person. You know, like if you hear something like monkeys like to cuddle with one another because they are reminded of their mother's study shows. Right. You could just be like, oh, that's great. I'm going to share that on the internet. It doesn't really affect you in any way. Yeah. But when there's studies being conducted that are creating drugs that could kill you or not treat you or that kind of thing and is, is, is, it's attracting money and funding and that kind of stuff.
That's like, that's harmful. Yeah, absolutely. Uh, I found another survey. Did you like that terrible study idea that it came up with? No, I liked it. The monkeys like to cuddle. Uh, 140 trainees at the MD Anderson Cancer Center in Houston, Texas. Thank you, Houston, for being so kind to us. at a recent show, they found that nearly a third of these trainees felt pressure to support their mentors' work, like to get ahead or not get fired. So that's another issue is you've got these trainees or residents and you have these mentors. And even if you disagree or don't think it's a great study, you're pressured into just going along with it.
I could see that for sure. There seems to be a huge hierarchy in science. Yeah, for sure. Like in a lab, you know, you got the person who runs the lab, it's their lab. Yeah, you go against them. Right. But there are people like Science and Nature, two great journals, are updating their guidelines right now. They're introducing checklists. Science hired statisticians to their panel of reviewing editors, not just other, you know, peer-reviewed. Like they actually hired numbers people specifically. Oh, gotcha. Because that's a big— To help the process. That's a huge part of studies, dude. Oh, yeah.
It's like this mind-breaking statistical analysis that can be used for good or ill. And, I mean, I don't think the average scientist necessarily is a whiz at that, although it has to be part of training. Yeah, but not necessarily. I mean, that's a different kind of beast altogether, stats. We talked about it earlier. I took a stats class in college. Oh, man, I had so much trouble. I was awful at it. It really just, it's a special kind of, is it even math? Hell? Yeah, I didn't get it. I passed it, though. I passed it because my professor took pity on me. Oh, that's nice. That Ulrich Dernagel. Ulrich Dernagel.
Uh-huh. He is a big-time crusader for his jam, making sure that science is good science. Yeah. One of the things he crusades against is the idea of, you remember in that virginity study where they just threw out anybody who had a violent encounter for their first sexual experience? Apparently that's a big deal with animal studies as well. If you're studying the effects of a drug or something, like there was this one in the article, if you're studying the effects of a stroke drug and you've got a control group of mice that are taking the drug or that aren't taking the drug and then a test group that are getting the drug.
And then like three mice from the test group die, even though they're on the stroke drug, they die of a massive stroke. And you just literally and figuratively throw them out of the study and don't include them in the results. That changes the data. And he's been on a peer review on a paper before. He's like, no, this doesn't pass peer review. You can't just throw out what happened to these three rodents. You started with 10. There's only seven reported in the end, what happened to those three? And how many of them just don't report the 10? Yeah. They're like, oh, we only started with seven, you know?
Well, I was about to say I get the urge. I don't get it because it's not right. But I think what happens is you work so hard at something. Yeah. Yeah. And you're like, how can I just walk away from two years of this because it didn't get a result? Okay. The point of real science though. Yeah. You have to walk away from it. Well, you have to publish that. Yeah. And that's the other thing too. And I guarantee scientists will say, hey man, try getting a negative paper published in a good journal these days. You don't want that kind of stuff. But part of it also is, I don't think it's enough to just have to be published in like a journal.
You want to make the news cycle as well. That makes it even better, right? So I think there's a lot of factors involved, but ultimately if you take all that stuff away, if you take the culture away from it, And if you get negative results, you're supposed to publish that so that some other scientists can come along and be like, oh, somebody else already did this using these methods that I was going to use. I'm not going to waste two years of my career because somebody else already did. Thank you, buddy, for saving me this time and trouble and effort to know that this does not work. You've proven this doesn't work.
When you sought to prove it does work, you actually proved it didn't work. That's part of science. Yeah. And I wish there wasn't a negative connotation to a negative result. Because to me, the value is the same. Sure. As proving something does work is proving something doesn't work, right? Right. Again, it's just not as sexy. Yeah. But I'm not sexy either. So maybe that's why I get it. Here's one that I didn't know was a thing. Predatory publishing. I didn't know about it either. Never heard of this. So here's a scenario. You're a doctor or a scientist, and you get an email from a journal that says, hey, you got anything interesting for us?
I've heard about your work. And you say, well, I actually do. I have this study right here. They say, cool, we'll publish it. You go, great. My career is taking off. Then you get a bill that says, where's my three grand for publishing your article? And you're like, I don't owe you three grand. All right, give us two. And you're like, I can't even give you two. And if you fight them long enough, maybe they'll drop it and never work with you again. Or maybe they'll just be like, we'll talk to you next quarter. Exactly. That's called predatory publishing. And I'm not sure how new it is. It's pretty new.
Is it pretty new? But it's a thing now where you can pay, essentially, to get something published. Yes, you can. it's kind of like who's who in behavioral sciences kind of thing. Yeah. You know? And apparently it's new because it's a result of open source academic journals, which a lot of people push for, including Aaron Schwartz, very famously, who like took a bunch of academic articles and published them online and was prosecuted heavily for it. Persecuted, you could even say. Yeah. But the idea that science is behind this paywall, which is another great article from Pricenomics, by the way, really just ticks a lot of people off.
So they started to open source journals, right? And as a result, predatory publishers came about and said, oh, okay, yeah, let's make this free. But we need to make our money anyway, so we're going to charge the academic who wrote the study for publishing it. Well, yeah, and sometimes now it's just a flat-out scam operation, 100%. There's this guy named Jeffrey Beal who is a research librarian. and he is my new hero because he's truly like one of these dudes that has, he's trying to make a difference and he's not profiting from this, but he's spending a lot of time by creating a list of predatory publishers.
Yeah, a significant list too. Yeah, how many? 4,000 of them right now? Yeah. Some of these companies flat out lie, like they're literally based out of Pakistan or Nigeria, and they say, no, we're a New York publisher. So it's just a flat-out scam. Or they lie about their review practices. They might not have any review practices. Right. And they straight-up lie and say they do. There was one called Scientific Journals International out of Minnesota that he found out was just one guy. Oh, yeah. Like literally working out of his home. Yeah. Just lobbying for articles, charging to get them published.
not reviewing anything and just saying, I'm a journal. Yeah. I'm a scientific journal. Look at me go. He shut it down apparently or tried to sell it. I think he was found out. And this other one, the International Journal of Engineering Research and Applications, they created an award and then gave it to itself. And even modeled the award from an Australian TV award. Like the physical statue. Wow. That's fascinating. I didn't know you could do that. I'm going to give ourselves the best podcast in the universe award. I like that. It's going to look like the Oscar. Yeah. Okay. The Oscar crossed with the Emmy.
This other one, MedNo Publications, actually confused the meaning of STM, science, technology, and medicine. They thought it meant sports, technology, and medicine. No. Well, a lot of science journalists or scientists, too, but watchdogs like to send gibberish articles into those things to see if they'll publish them and sometimes they do. Frequently they do. They sniff them off the case? Big time. How about that callback? It's been a while. It has been. It needs to be a t-shirt. Should we take a break? Yeah. Alright, we'll be back and finish up right after this. Just like the number of stars in the sky There is so much stuff you should know Support for the show comes from Public, the investing platform for those who take it seriously.
On Public, you can build a multi-asset portfolio of stocks, bonds, options, crypto, and now generated assets, which allow you to turn any idea into an investable index with AI. It all starts with your prompt. From renewable energy companies with high free cash flow to semiconductor suppliers growing revenue over 20% year over year, you can literally type any prompt and put the AI to work. It screens thousands of stocks, builds a one-of-a-kind index, and lets you backtest it against the S&P 500. Then, you can invest in a few clicks. Generated assets are like EFTs with infinite possibilities, completely customizable and based on your thesis, not someone else's.
Go to public.com slash podcast and earn an uncapped 1% bonus when you transfer your portfolio. That's public.com slash podcast. Paid for by Public Investing. Brokerage services by Open to the Public Investing, Inc. Member FINRA SIPC. Advisory services by Public Advisors LLC. SEC registered advisor. Generated assets is an interactive analysis tool. Output is for informational purposes only and is not investment recommendation or advice. Complete disclosures available at public.com slash disclosures. And now, Superhuman Shaq. I keep telling them not to say that. I'm no superhuman. Believe it or not, I struggle with moderate obstructive sleep apnea or OSA.
In adults with obesity, moderate to severe OSA is a condition where breathing is interrupted during sleep. with loud snoring, choking, gasping for air, and even daytime fatigue. Let's just say it can sound a lot like this. Sound familiar? Learn more at dontsleeponosa.com. This information is provided by Lilly, a medicine company. Learning stuff with Joshua and Charles. Stuff you should know. So here's a big one. You ever heard the term follow the money? that's applicable to a lot of realms of society. Yeah. And most certainly in journals, if something looks hinky, just do a little investigating and see who's sponsoring their work.
Well, especially if that person is like, no, everyone else is wrong. Right. Climate change is not man-made kind of thing. Sure. You know, if you look at where their funding's coming from, you might be unsurprised to find that it's coming from people who would benefit from the idea that anthropogenic climate change isn't real. Yeah. Well, we might as well talk about him. Okay. Willie Soon. Yeah. Mr. Soon. Is he a doctor? He's a physicist of some sort. Yeah. All right. I'm just going to say Mr. or Dr. Soon because I'm not positive. He is one of a few people on the planet Earth, professionals, that is, who deny human climate change, human influenced climate change, like you said.
You said the fancier word for it, though. Anthropogenic? Yeah, it's a good word. Is that it? And he works at the Harvard-Smithsonian Center for Astrophysics. So, hey, he's with Harvard. He's got the cred, right? Right. Turns out, when you look into where he's getting his funding, he received $1.2 million over the past decade from ExxonMobil, the Southern Company. The Kochs. And the Koch brothers, their foundation, the Charles G. Koch Foundation. Exxon stopped in 2010, stopped funding him. But the bulk of his money and his funding came, I'm sorry, I forgot the American Petroleum Institute, came from people who clearly had a dog in this fight.
and it's just, how can you trust this, you know? Yeah, well, you trust it because there's a guy, and he has a PhD in aerospace engineering, by the way. All right, he's a doc. He works with this organization, the Harvard-Smithsonian Center for Astrophysics, which is a legitimate place. It doesn't get any funding from Harvard, but it gets a lot from NASA and from the Smithsonian. Well, and Harvard's very clear to point this out when people ask them about Willie Soon. Right. They're kind of like, well, here's the quote. Willie Soon is a Smithsonian staff researcher at Harvard-Smithsonian Center for Astrophysics, a collaboration of the Harvard College Observatory and the Smithsonian Astrophysical Observatory.
Like, they just want to be real clear, even though he uses a Harvard email address, he's not our employee. No, but again, he's getting lots of funding from NASA and lots of funding from the Smithsonian. And this guy, if his scientific beliefs are what they are, and he's a smart guy, then, yeah, I don't know about getting fired for saying, you know, here's a paper on the idea that climate change is not human made. Yeah, he thinks it's the sun's fault. But he doesn't reveal in any of his conflicts of interest that should go at the end of the paper. He didn't reveal where his funding was coming from.
And I get the impression that in academia, if you are totally cool with everybody thinking like you're a shill, you can get away with it. Right. Well, this stuff, a lot of this stuff is not illegal. Right. Even predatory publishing is not illegal. Yeah. Just unethical. Right. And if you're counting on people to police themselves with ethics, a lot of times they'll disappoint you. The Heartland Institute gave Willie Soon a Courage Award. For not caring about what other scientists think of him. If you've heard the Heartland Institute, you might remember them. They're a conservative think tank. You might remember them in the 90s when they worked alongside Philip Morris to deny the risks of secondhand smoke.
Yeah, that's all chronicle in that book I've talked about, Merchants of Doubt. Oh, really? Just a bunch of scientists, legitimate, bona fide scientists who are up for being bought by groups like that. It's sad. It is sad. And the whole thing is they're saying, well, you can't say beyond a shadow of a doubt with absolute certainty that that's the case. And science is like, no, science doesn't do that. Science doesn't do absolute certainty, but the average person reading a newspaper sees that. Oh, you can't say with absolute certainty. Well, then maybe it isn't man-made. Right. And then there's that doubt that the people just go and get the money for, for saying that, for writing papers about it.
Yeah. Millions of dollars. It's despicable. Yeah, it really is. Self-reviewed. You've heard of peer review. We've talked about it quite a bit. Peer review is when you have a study. and then one or more, ideally more, of your peers reviews your study and says, you know what, you had best practices, you did it right, it was reproducible, you follow the scientific method, I'm going to give it my stamp of approval and put my name on it, not literally. Or is it? I think so. It says who reviewed it? Yeah, I believe so. Okay, put my name on it. In the journal when it's published. But not my name as the author of the study, you know what I mean.
Right. As a peer reviewer. Yeah, as a peer reviewer. And that's a wonderful thing. But people have faked this and been their own peer reviewer, which is not how it works. No. Who is this guy? Well, I'm terrible at pronouncing Korean names. So all apologies. But I'm going to say Nung In Moon. Nice. Dr. Moon? I think, yeah, let's call him Dr. Moon. Okay. So Dr. Moon worked on natural medicine, I believe. and was submitting all these papers that were getting reviewed very quickly. Because apparently part of the process of peer review is to say, this paper is great. Can you recommend some people in your field that can review your paper?
And Dr. Moon said, I sure can. Yeah, he was on fire. Let me go make up some people and make up some email addresses that actually come to my inbox and just posed as all of his own peer reviewers. He was lazy, though, is the thing. Like I don't know that he would have been found out if he hadn't been careless, I guess. Because he was returning the reviews within like 24 hours sometimes. A peer review of like a real study should take, I would guess, weeks if not months. Like the study, the publication schedule for the average study or paper, I don't think is a very quick thing. There's not a lot of quick turnaround.
Right. And this guy was like 24 hours. Dr. Moon, I see your paper was reviewed and accepted by Dr. Mooney. It's like, I just added a Y to the end. Right. Seemed easy. Yeah. If you Google peer review fraud, you will be shocked at how often this happens and how many legit science publishers are having to retract studies. And it doesn't mean they're bad. They're getting duped as well. But there's one based in Berlin in 2015 had 64 retractions because of fraudulent reviews. Oh, wow. And they're just one publisher of many. Every publisher out there probably has been duped. Maybe not everyone. I'm surmising that.
But it's a big problem. We should do a study on it. I'll review it. It'll end up in the headlines now. Right. Every single publisher duped, says Chuck. And speaking of the headlines, Chuck, one of the problems with science reporting or reading science reporting is that what you usually are hearing, especially if it's making a big splash, is what's called the initial findings. Right. Somebody carried out a study and this is what they found. And it's amazing and mind blowing. And it supports everything everyone's always known. But now there's a scientific study that says, yes, that's the case. And then if you wait a year or two when people follow up and reproduce the study and find that it's actually not the case, it doesn't get reported unusually.
Yeah, and sometimes the scientist or the publisher, they're doing it right, and they say initial findings. Right. But the public, and sometimes even the reporter will say initial findings. But we as people that ingest this stuff need to understand what that means. Right. And the fine print is always like, you know, more study is needed. But no one, if it's something that you want to be true, you'll just say, hey, look at this study. Right. You know, it's brand new and they need to study it for 20 more years. But hey, look what it says. Right. But the more you start paying attention to this kind of thing, the more kind of disdain you have for that kind of just offhand sensationalist science reporting.
Yeah. But you'll still get caught up in it. Like every once in a while, I'll catch myself saying something to you and be like, oh, did you hear this? And then as I'm saying it out loud, I'm like, that's preposterous. Yeah. There's no way that's going to pan out to be true. I got click baited. I know. I mean, we have to avoid this stuff. It's tough. Yeah. Because we have our name on this podcast. But luckily, we've given ourselves the back door of saying, hey, we make mistakes a lot. It's true, though. We're not experts. No. We're not scientists. And then finally, we're going to finish up with the header on this one is it's a cool story.
Yeah. And that's a big one because it's not enough these days. and this all ties in with media and how we read things as people, but it's not enough just to have a study that might prove something. You have to wrap it up in a nice package to deliver people. Get it in the news cycle. And the cooler the better. Yep. Yep. It almost doesn't matter about the science as far as the media is concerned. They just want a good headline and a scientist who will say, yeah, that's cool. Here's what I found. Yep. This is going to change the world. Mm-hmm. Loch Ness Monster is real. This is a kind of ended up being depressing somehow.
Yeah. Not somehow. Yeah. Like, yeah, it's kind of depressing. I know. We'll figure it out, Chuck. Well, we do our best. I'll say that. Science will prevail. I hope so. If you want to know more about science and scientific studies and research fraud and all that kind of stuff, Just type some random words into the search bar at HowStuffWorks.com. See what comes up. Yeah. And since I said random, it's time for listener mail. Oh, no. Oh, yeah? You know what it's time for. What? Administrative Details. All right, Josh. Administrative Details. If you're new to the show, you don't know what it is, that's a very clunky title.
Yeah. We're saying thank you to listeners who send us neat things. It is clunky and generic and I've totally gotten used to it by now. Well, you're the one who made it up. To be clunky and generic and it's stuck. Yeah. So people send us stuff from time to time and it's just very kind of you to do so. Yes. And we like to give shout outs whether or not it's just out of the goodness of your heart or if you have a little small business that you're trying to plug. Either way. It's a sneaky way of getting it in there. Yeah, but I mean, I think we brought that on. Didn't we say like if you have a small business and you send us something, we'll be happy to say something?
Exactly. Thank you. All right. So let's get it going here. We got some coffee from 1,000 Faces right here in Athens, Georgia from Kayla. Yeah. Delicious. Yes, it was. We also got some other coffee too from Jonathan at Steamworks Coffee. He came up with a Josh and Chuck blend. Oh, yeah. It's pretty awesome. I believe it's available for sale too. Yeah. The Josh and Chuck blend is dark and bitter. Jim Simmons, he's a retired teacher who sent us some lovely handmade wooden bowls. Oh, yes. And a very nice handwritten letter, which is always great. Thanks a lot, Jim. Let's see. Chamberlain sent us homemade pasta, including a delicious, savory pumpkin fettuccine.
It was very nice. Yum. Jay Graft, two Fs, sent us a postcard from the Great Wall of China. It's kind of neat. Sometimes we get those postcards from places we've talked about. They're always like, look where I am. Thanks, Jay. You guys aren't here. Let's see. The Hammer Press team. They sent us a bunch of Mother's Day cards that are wonderful. Oh, those are really nice. Really great. You should check them out. The Hammer Press team. Yeah. Yeah. Misty, Billy, and Jessica. They sent us a care package of a lot of things. There were some cookies. Okay. Including one of my favorite, white chocolate dipped Ritz and peanut butter cracker.
Oh, yeah. Man, I love those. Homemade, right? Oh, yeah. Yeah. And then some 70s macrame for you, along with 70s macrame magazines. Yeah. Because you're obsessed with macrame. We have a macrame plant holder hanging from my microphone arm. Holding a cup. It has a coffee mug sent to us by Joe and Linda Hecht. Oh, that's right. And it has some pens in it. And they also sent us, Misty, Billy, and Jessica, a lovely little hand-drawn picture of us with their family, which was so sweet. That is very awesome. We've said it before. We'll say it again. Huge thank you to Jim Ruane. I believe that's how you say his name.
And the Crown Royal people for sending us all the Crown Royal. We are running low. Mark Silberg of the Rocky Mountain Institute sent us a book called Reinventing Fire. Oh, yeah. They're great out there. Man, they know what they're talking about. And I think it's Reinventing Fire, colon, Bold Business Solutions for the New Energy Era. Yeah, they're basically like green energy observers. But I think they're experts in all sectors of energy, but they have a focus on green energy. Which is awesome. Yeah, they're pretty cool. John, whose wife makes delightfully delicious doggy treats. Delightfully delicious is the name of the company.
There's no artificial colors or flavors. And they got Sweet Little Momo hooked on sweet potato dog treats. I thought you were going to say hooked on the junk. The sweet potato junk. She's crazy. Cuckoo for sweet potatoes. Nice. Oh, man. That's good for a dog, too. It is, very. Strat Johnson sent us his band's LP. And if you're in a band, your name is Strat. That's pretty cool. Sure. Diomea, still. Mm-hmm. I think that was great. Yeah, I'm not sure if I pronounced that right. D-I-O-M-A-E-A. Frederick, this is long overdue. Frederick at the 1521 store, 1521store.com, sent us some awesome low-profile cork iPhone cases and passport holders.
And I was telling him, Jerry walks around with her iPhone in the cork holder, and it looks pretty sweet. Oh, yeah. So he said, awesome, glad to hear it. Joe and Holly Harper sent us some really cool 3D-printed stuff-you-should-know-things, like S-Y-S-K, you know, like a little desk. Oh, like after Robert Indiana's Love sculpture? Yeah, that's what I couldn't think of what that was from. Yeah, it's awesome. It's really neat and like a bracelet made out of stuff you should know, 3D carved, like plastic. It's really neat. Yeah, they did some good stuff. So thanks, Joe and Holly Harper, for that.
And then last for this one, we got a postcard from Yosemite National Park from Laura Jackson. So thanks a lot for that. Thanks to everybody who sends us stuff. It's nice to know we're thought of, and we appreciate it. Yeah, we're going to finish up with another set on the next episode of Administrative Details. You got anything else? No, that's it. Oh, yeah. If you guys want to hang out with us on social media, you can go to SYSK Podcast on Twitter or on Instagram. You can hang out with us at Facebook.com slash StuffYouShouldKnow. You can send us an email to StuffPodcast at HowStuffWorks.com.
And as always, join us at our home on the web, StuffYouShouldKnow.com. Stuff You Should Know is a production of iHeartRadio's How Stuff Works. For more podcasts from iHeartRadio, visit the iHeartRadio app. Apple Podcasts are wherever you listen to your favorite shows. Toyotathon is on. So stop by your local Toyota dealer for incredible year-end deals on the cars, trucks, and SUVs known for their legendary reliability. Like the stylish and efficient Camry, the ready-for-anything RAV4, or check out a rough-and-rugged Tacoma or Tundra. All with great financing and lease options available to qualified customers.
Visit buyatoyota.com to find out more. Toyota, let's go places. Support for the show comes from Public, the investing platform for those who take it seriously. On public, you can build a multi-asset portfolio of stocks, bonds, options, crypto, and now generated assets, which allow you to turn any idea into an investable index with AI. It all starts with your prompt. From renewable energy companies with high free cash flow to semiconductor suppliers growing revenue over 20% year over year, you can literally type any prompt and put the AI to work. It screens thousands of stocks, builds a one-of-a-kind index, and lets you backtest it against the S&P 500.
Then, you can invest in a few clicks. Generated assets are like EFTs with infinite possibilities. Completely customizable and based on your thesis, not someone else's. Go to public.com slash podcast and earn an uncapped 1% bonus when you transfer your portfolio. That's public.com slash podcast. Paid for by Public Investing. Brokerage services by Open to the Public Investing, Inc. Member FINRA, SIPC. Advisory services by Public Advisors, LLC. SEC Registered Advisor. Generated assets is an interactive analysis tool. Output is for informational purposes only and is not investment recommendation or advice.
Complete disclosures available at public.com slash disclosures. This is an iHeart Podcast. Guaranteed human.