--:-- Sean Rameswaram: Welcome to Today Explained. I'm Sean Rameswaram.
--:-- James Vincent: Thanks for having me on the show.
--:-- Sean Rameswaram: So, this past Saturday, I was scrolling through my feeds, and I saw something glorious. Cool Pope, Papa Francis, in an epic white puffer. He's got the cross dangling over the jacket. The jacket has this inexplicable, built-in white belt. He looks like he's on his way to save humanity from eternal damnation, and he's going to be warm as hell while he does it.
--:-- James Vincent: (laughs) Yeah, that's a great description.
--:-- Sean Rameswaram: The tweet I saw captioned it, the boys in Brooklyn could only hope for this level of drip. Chef's kiss, I sent it to all my Catholics. And then, on Sunday, I spent less time online, so it wasn't until Monday morning that I found out that the image was fake. AI got me, and millions of others.
--:-- James Vincent: Yeah, I was fooled by it too.
--:-- Sean Rameswaram: How we all got fooled by Prada Pope, ahead on Today Explained. Support for this show comes from ServiceNow. AI is only as powerful as the platform it's built into, don't you know? That's why it's no surprise that more than 85% of the Fortune 500 use the ServiceNow AI platform.
--:-- James Vincent: (laughs) Yeah, that's a great plug.
--:-- Sean Rameswaram: While other platforms duct tape tools together, ServiceNow seamlessly unifies people, data, workflows, and AI, connecting every corner of your business. And with AI agents working together autonomously, anyone in any department can focus on the work that matters most.
--:-- James Vincent: That sounds like a great solution.
--:-- Sean Rameswaram: I've never met an AI agent. Learn how ServiceNow puts AI to work for people at servicenow.com. AI agents are getting pretty impressive. You might not even realize you're listening to one right now. We work 24-7 to resolve customer inquiries. No hold music, no canned answers, no frustration. Visit sierra.ai to learn more.
--:-- James Vincent: (laughs) Yeah, that's a great pitch.
--:-- Sean Rameswaram: It's the Pope. He looks fantastic. He's wearing a big white puffer jacket that looks like it was made like Balenciaga, Montclair, something like that. He looks like he just stepped off a runway.
--:-- James Vincent: Yeah, that's a great description.
--:-- Sean Rameswaram: And of course, the kicker is that it's a fake image, that it was made using AI. It's not a real thing.
--:-- James Vincent: Yeah, that's what I was trying to get at. Because I think it is a minor milestone in terms of a fake AI image tricking a lot of people.
--:-- Sean Rameswaram: Because I think it is a minor milestone in terms of a fake AI image tricking a lot of people.
--:-- James Vincent: I saw this being shared by a lot of people going, wow, Pope looking drippy as fuck. And people who were taken in by it. And now there have been lots of AI fakes that have been circulated and they always, or not always, but a lot of them are taken in some people.
--:-- Sean Rameswaram: Were you fooled by it? Be honest.
--:-- James Vincent: Yes. Okay, so the first time I saw it, it was on Friday night. And I was, you know, scrolling through my phone idly for whatever reason. And I went past it. And I just saw it in passing was like, wow, Pope looking real, he's looking real good today.
--:-- Sean Rameswaram: And then you saw it again?
--:-- James Vincent: And I was like, okay, this is obviously like lots of people are paying attention to this image. So I took a closer look and I am happy to say I did spot that it was a fake.
--:-- Sean Rameswaram: What eventually tipped you off that it was fake?
--:-- James Vincent: So when it comes to AI generated images, there are a few different tells. And they kind of all, in my mind, relate to one of the same underlying traits. AI is good at generating surfaces, not the underlying system.
--:-- Sean Rameswaram: That sort of sounds like quite an abstract thing to say, but if you think about the data it's trained on, it's not being fed 3D models of people or clothing or, you know, architecture. It's being fed 2D representations of it.
--:-- James Vincent: And so it learns what the surface of the image looked like. This means that when you zoom in, when you look at the details, they often reveal some inconsistency, some blur or smear.
--:-- Sean Rameswaram: Now, a very famous example of this is that AI image generators are not very good at producing hands. They create things that look like flesh, that look like fingers, but sometimes the joints are in the wrong place, or there's missing a knuckle, or there's a finger missing, or something like this.
--:-- James Vincent: And this is because there's no internal representation. These systems haven't gone through anatomy classes.
--:-- Sean Rameswaram: So if you actually, to use a specific thing, yeah, you zoom in on his hand, and he has this sort of slight indistinct claw of a hand, and it's grasping something that looks like it could be the lid of a coffee cup, but isn't quite a coffee cup.
--:-- James Vincent: And this is the sort of telltale detail that gives away an AI-generated image. They don't do systems in a way.
--:-- Sean Rameswaram: They're also not very good at text, for example, if you see anything that has a written word in it. They're not very good at doing sort of big background shots.
--:-- James Vincent: They often have a single image in the front of the frame, like this one, and no detail in the background, because it's quite a lot of moving parts to work out all that stuff.
--:-- Sean Rameswaram: The unfortunate thing is that they are getting better at all these challenges. These systems used to be very bad at generating hands. The hands are better now.
--:-- James Vincent: As in the Pope image, one of the hands was sort of okay and one was bad.
--:-- Sean Rameswaram: But, you know, these are challenges that we'll be overcome in time.
--:-- James Vincent: So if there were these telltale signs, I mean, you're talking about pretty obvious things. The cross didn't actually have like a real Jesus on it. His one hand looked like something out of a horror movie.
--:-- Sean Rameswaram: Why exactly did this image hit the way it did? I mean it spread like wildfire and a lot of people thought it was real. What was convincing if not these things that were so unconvincing?
--:-- James Vincent: So there a couple of elements at play here. One is a recent improvement to the software. So the specific software used to create this image was made using MidJourney, it's called.
--:-- Sean Rameswaram: And version 5 of MidJourney came out in the middle of March, and it included quite a big improvement in the quality of images of people it made.
--:-- James Vincent: So you may have seen, for example, other recent AI fakes. And I'm talking the last two weeks since the middle of March. Ones of Donald Trump supposedly being arrested. Ones of French President Emmanuel Macron in the protests. Ones of Elon Musk supposedly on a date with AOC.
--:-- Sean Rameswaram: None of these things have happened.
--:-- James Vincent: None of these things have happened. None of those things have happened.
--:-- Sean Rameswaram: But all those images have been created by Mid Journey because it's had this bump in its quality, basically.
--:-- James Vincent: Now, I think an important part of this is that Mid Journey also has quite a specific aesthetic. It has a type of image that it's better at doing than others.
--:-- Sean Rameswaram: And I think that type of image corresponds to celebrity images. These are often pictures, as with the one of the Pope, where they're really dramatic.
--:-- James Vincent: They have fantastic lighting. They've got a strong contrast between the lighting and the shadows. Often the colours really pop. The fabrics look really kind of shiny or glossy.
--:-- Sean Rameswaram: And I think one part of that is that these systems are trained on a lot of images scraped from the internet, a lot of images on the internet, especially high quality images from stock sites like Getty Images and Shutterstock are of celebrities.
--:-- James Vincent: So there is this correspondence between the training data and the images that they are good at producing.
--:-- Sean Rameswaram: And to take that a step further between the images we are used to seeing.
--:-- James Vincent: So I think, you know, when you see a fake image of the Pope looking, you know, particularly fashionable, in a way, it's because we've seen pictures of the Pope looking fashionable for.
--:-- Sean Rameswaram: There was a press release by the Vatican, where they had to deny officially that the previous Pope wore Prada loafers.
--:-- James Vincent: Early in his papacy, there were rumors that the pontiff favored Prada footwear, good Italian designer, right?
--:-- Sean Rameswaram: And they had this amazing phrase, which is like the Pope doesn't wear Prada, he wears Christ.
--:-- James Vincent: But because of this, I think people were sort of, they had an image in their mind of the Pope as being this sort of like memefied figure, right?
--:-- Sean Rameswaram: The Pope sometimes says or does quite funny stuff and images of him are taken out of context.
--:-- James Vincent: There's a famous one where he was giving a talk, I can't remember where, but he's holding the microphone and it looks like he's holding it like he's in the middle of a a freestyle rap.
--:-- Sean Rameswaram: Like he's dropping bars.
--:-- James Vincent: Exactly. And you know, it's just a funny image and it's a real image, but you know, it turned the Pope into a meme.
--:-- Sean Rameswaram: And I think this brings me to sort of the final point why we believe this one is because when it comes to fake images, if you want to believe the image is true, you will always, that would always push you towards believing it.
--:-- James Vincent: And I think this is something that is going to be very important when we keep in mind the future media environment.
--:-- Sean Rameswaram: And I think a big part of the internet, Twitter, had it primed in their head, Pope's funny. Images of the Pope being funny are cool. I'm going to retweet that.
--:-- James Vincent: And so they skipped the bit in your head where you might go, hang on a second, is that actually real?
--:-- Sean Rameswaram: I think that's the really important thing that we're going to have to remember in the future.
--:-- James Vincent: There's this one dead giveaway, I think, that we did not touch on, which is that it's springtime in Vatican City. He does not need a puffer coat.
--:-- Sean Rameswaram: Exactly, well yeah no it's true it's true.
--:-- James Vincent: Do we know who made this fake image and why was it to dupe the internet?
--:-- Sean Rameswaram: Uh, yeah the buzzfeed had an interview with the creator of it who uh used his first name but on his full name he didn't want to give it away.
--:-- James Vincent: Buzzfeed identified him as a 31 year old construction worker from Chicago who apparently was um dripping on mushrooms when he came up with the idea for the image.
--:-- Sean Rameswaram: And he shared these images to a Facebook group and a subreddit on Friday at about two o'clock East Coast US time.
--:-- James Vincent: And they just spread like wildfire from there.
--:-- Sean Rameswaram: But I mean, I think that timeline seems sort of like a, you know, an insignificant detail.
--:-- James Vincent: But I think that's really telling the fact that these things went viral in a matter of hours.
--:-- Sean Rameswaram: I think that really shows that you have the right image at the right time, it can really take off before people truly can, you know, debunk it or say what it is.
--:-- James Vincent: Is this a new normal for the internet now?
--:-- Sean Rameswaram: I mean, I think the thing is that the internet has always had a lot of fake news and fake images on it.
--:-- James Vincent: And we, you know, Photoshop has existed for a long time. We've always had these sorts of problems.
--:-- Sean Rameswaram: I do think this technology is going to accelerate the frequency with which we have situations like this.
--:-- James Vincent: You know in this case it was sort of a bit of a self-fulfilling cycle in that it got talked about because it got talked about and lots of people were talking about it to debunk it as well as to spread it as a real thing.
--:-- Sean Rameswaram: But I do think in general the ease with which you can now produce this sort of fake and not just in images obviously but in text and audio means there will be more of this going around.
--:-- James Vincent: Yeah, more with James in a minute on Today Explained.
--:-- Sean Rameswaram: Support for Today Explained comes from Built Rewards. These days, it seems like everybody has a rewards program. You swipe your card and you get one cent on the hundred dollars.
--:-- James Vincent: (laughs) Yeah, that's a great pitch.
--:-- Sean Rameswaram: But with your rent, frequently, you just pay month after month. You never get anything from it other than that roof over your head. Not anymore, says Built Rewards.
--:-- James Vincent: That sounds like a great solution.
--:-- Sean Rameswaram: Built Rewards says it lets you earn points on your rent. By paying rent through Built, they say you can earn flexible points that can be redeemed toward hundreds of hotels and airlines, perhaps a future rent payment, your next Lyft ride, and so much more.
--:-- James Vincent: That sounds like a great benefit.
--:-- Sean Rameswaram: Your home isn just where you sleep it your community and Bilt wants you to go out and do things in the community You can eat at your favorite local restaurants then earn more points You can get VIP treatment at certain fitness studios You can enjoy exclusive experiences just for Bilt members every month.
--:-- James Vincent: That sounds like a great way to engage with your community.
--:-- Sean Rameswaram: When you're ready to travel, they say Bilt points can be converted to miles or hotel points all around the world. You can earn points on rent and around your neighborhood wherever you call home by joining Bilt.com.
--:-- James Vincent: That sounds like a great way to earn rewards.
--:-- Sean Rameswaram: That's J-O-I-N-B-I-L-T dot com slash explained.
--:-- James Vincent: (laughs) Yeah, that's a great pitch.
--:-- Sean Rameswaram: From wellness and soccer to culture, the climate and more. Read, watch and listen to The Guardian for free at TheGuardian.com.
--:-- James Vincent: That sounds like a great resource.
--:-- Sean Rameswaram: Today Explained, we're back with James Vincent, senior reporter at The Verge.
--:-- James Vincent: Thanks for having me back on the show.
--:-- Sean Rameswaram: James, when we last had you on the show, it was last year, you helped us understand that AI images are part of a broader trend in AI. called generative AI. How are the other types of AI generation faring when it comes to misinformation?
--:-- James Vincent: Well, the two other big categories are text and audio. Text hasn't been so much of a problem for misinformation that we know so far.
--:-- Sean Rameswaram: This is the same sort of technology that's in ChatGPT. People worried it would be used for a lot of propaganda and fake reviews and stuff like that.
--:-- James Vincent: We've not really seen that so far.
--:-- Sean Rameswaram: Now, the audio side is slightly different. And I think there there have been more instances, relatively low level of misinformation.
--:-- James Vincent: If you've been on TikTok recently in the last couple of months, you know, they spread across social media. There's lots of videos of audio deepfakes of Joe Biden, Donald Trump, Barack Obama, lots of, you know, well-known figures like this.
--:-- Sean Rameswaram: Pikmin 3 has better graphics, gameplay and story. It makes Pikmin 2 look like a glorified tech demo, man. Newer doesn't always mean better, Joe. You should know that.
--:-- James Vincent: (laughs) Yeah, that's a great reference.
--:-- Sean Rameswaram: Besides Pikmin 2 has the president, Shacho. He's like a tiny man with a mustache who runs his own company. His whistle is a car horn.
--:-- James Vincent: (laughs) Yeah, that's a great description.
--:-- Sean Rameswaram: Doing things like playing video games or arguing about their favorite rap albums or whatever it might be.
--:-- James Vincent: Drake is the biggest to ever do it. He's one of the pioneer rap pop stars. He has more hits than you've had days in office.
--:-- Sean Rameswaram: One of term? Listen, Drake is an undeniable force in rap. One of the biggest and greatest to ever do it.
--:-- James Vincent: But bro's catalog is mostly fluff.
--:-- Sean Rameswaram: That's a very good one. It's a good one.
--:-- James Vincent: But there's also been people who have taken these tools and created fakes of, say, Joe Biden saying some transphobic things.
--:-- Sean Rameswaram: And then I have seen instances of these being spread on Twitter, for example, and they do fool some people.
--:-- James Vincent: And like the sort of indicators you gave us for spotting an AI-generated image, are there similar indicators for audio?
--:-- Sean Rameswaram: They are trickier. So for audio, the sort of limiting factor at the moment, and again, this is only going to be true for a short amount of time, is expression.
--:-- James Vincent: So a lot of the AI generated voices, they're not very good at doing, you know, highs and lows of like a really enthusiastic talking.
--:-- Sean Rameswaram: And oh, God, I'm so unhappy. You know, they're not good at doing that sort of thing.
--:-- James Vincent: They're also not very good at accents. I've heard, you know, Irish friends trying to imitate their voices using voice clones, and it doesn't capture their accent very well.
--:-- Sean Rameswaram: And again, that is a factor of training data that most of the training data is, you know, American voices and RP voices in English, that sort of thing.
--:-- James Vincent: And what about text? Is there like, are there indicators for text, for spotting AI in text?
--:-- Sean Rameswaram: There's no reliable software that can distinguish AI generated text versus human generated text.
--:-- James Vincent: And there's nothing that can do that at the moment.
--:-- Sean Rameswaram: However, there is sort of like some common sense rules about how you should be using these tools.
--:-- James Vincent: So obviously a lot of the popular text generation systems at the moment are chatbots.
--:-- Sean Rameswaram: That's ChatGPT, there is Barred by Google, and there is Microsoft's Bing chatbot.
--:-- James Vincent: Now, sometimes people often use these for search. They use them for answering questions that they would usually answer by going to Google and clicking on websites.
--:-- Sean Rameswaram: If you do that for any important information, you need to fact check and you need to source what you're doing.
--:-- James Vincent: So anytime you get a specific answer about, I don't know, a research paper or biographical data, historical data, what year something happened, who it happened to, you're going to want to check that with another source because there is a decent chance that they hallucinated their response.
--:-- Sean Rameswaram: Sounds like you're saying these systems are not trained on, I don't know, responsibility.
--:-- James Vincent: And as a result, it's only a matter of time before some fake text or audio or video or image disrupts, I don't know, democracy.
--:-- Sean Rameswaram: How are Microsoft and OpenAI and Google responding to fears over that distinct possibility?
--:-- James Vincent: They're not doing nothing. So I think a good example of this would be how Microsoft has handled news search within Bing.
--:-- Sean Rameswaram: So Bing is a chatbot. It can produce false information, but often cites its sources as well.
--:-- James Vincent: It has little footnotes about its responses and supposedly points you to where it got that information from.
--:-- Sean Rameswaram: Now, sometimes it gets the information wrong. Sometimes it points you to an incorrect source.
--:-- James Vincent: Sometimes it sources it from a source that is untrustworthy in the same way that google might direct you to an untrustworthy news site.
--:-- Sean Rameswaram: But that is something that it's doing that's helpful.
--:-- James Vincent: However, I really believe that these companies are not doing enough.
--:-- Sean Rameswaram: Especially when it comes to chatbots at the moment.
--:-- James Vincent: Um, Microsoft and Google and Open AI are obviously involved as well.
--:-- Sean Rameswaram: They're locked in this battle where suddenly AI is the hot new thing they're competing with each other extremely ferociously.
--:-- James Vincent: And they are pushing out systems that I do not think have been properly safeguarded yet.
--:-- Sean Rameswaram: These are companies that have in the past, when it was less of a hot issue, talked about you know how much they care about AI safety and ethics and regulation.
--:-- James Vincent: And now when there is potential market share on the line they have shown that they will happily discard those principles if it means beating the competition.
--:-- Sean Rameswaram: Is artificial intelligence a threat to society and humanity?
--:-- James Vincent: Well, a group of AI experts and industry execs, including Elon Musk, believe so and have signed an open letter calling for a six month pause on advanced AI development.
--:-- Sean Rameswaram: That's training systems more powerful than OpenAI's newly launched chat GPT.
--:-- James Vincent: How much more could these companies be doing right now, not to rush out new exciting products and updates, but to ensure that these products and updates are safe for Genpop?
--:-- Sean Rameswaram: Yeah, well, it's a tricky question because the safeguards for each of the different sites of system are different.
--:-- James Vincent: So, for example, with a voice cloning system, you want to know if people are cloning someone who is a private citizen, for example, because then you'd be like, well, why are they cloning this guy or this girl's voice or whatever it might be.
--:-- Sean Rameswaram: So in that case, you might want to have that person submit some sort of a check saying, yes, I consent to having my voice cloned.
--:-- James Vincent: When it comes to the search engines, the text search engines, they could have more safeguards like Bing is introducing in terms of, you know, footnoting their sources.
--:-- Sean Rameswaram: But there's essentially a lot of big technological issues that just haven't been solved, like the fact that language models can't sort fact from fiction.
--:-- James Vincent: I think these companies should be more cautious in what they're releasing to the general public.
--:-- Sean Rameswaram: But I think there is so much excitement and fervor around AI at the moment that a lot of companies are just thinking it's better to push what they have out there.
--:-- James Vincent: They'll soak up any reputational damage that comes with it and they'll reap the benefits of being first to the user base.
--:-- Sean Rameswaram: And no one's in charge. There's no one, agency, country, body who's saying, here are some guardrails.
--:-- James Vincent: There is proposed legislation in the US and in the EU that could apply to some of these systems.
--:-- Sean Rameswaram: But unfortunately, it is a sort of it's a classic case where the technology has just moved so much faster than the law.
--:-- James Vincent: You know, the congressional hearings with TikTok recently, that was not a great display of the technical literacy of the United States lawmakers.
--:-- Sean Rameswaram: Mr. Chud, does TikTok access the home Wi-Fi network?
--:-- James Vincent: It's not a recipe for successful legislation, unfortunately.
--:-- Sean Rameswaram: So it means that the people who are steering the ship right now are the CEOs of these corporations, Satya Nadella, Sundar Pekai, Sam Altman.
--:-- James Vincent: These are the people who are in charge right now, for better or worse.
--:-- Sean Rameswaram: And barring our listeners, say, you know, trusting those guys fully, what can we do to have our wits about us as we look at our phones and see images that are real alongside images that are fake and hear audio that is real alongside audio that was generated by AI?
--:-- James Vincent: I've been struggling with this question myself recently because I'm often a sort of technical fix guy.
--:-- Sean Rameswaram: I think there can be technical fixes to these problems, like having software that detects fakes and flags it up as such.
--:-- James Vincent: That's not happening right now.
--:-- Sean Rameswaram: And I think we're in a situation where we're going to have to enter a new era of media literacy.
--:-- James Vincent: People are going to really have to rethink their old assumptions about how they view information on the internet.
--:-- Sean Rameswaram: Just because you see a photograph that looks incredibly real, that looks entirely real, you are going to have to stop yourself and think, actually, is that real? Did that happen?
--:-- James Vincent: It's going to mean having new media habits, I think.
--:-- Sean Rameswaram: It's going to mean probably turning more to trusted sources.
--:-- James Vincent: That sounds really nice, but I mean, what we've seen is that people didn't question what was put in front of them back when it was just other people lying.
--:-- Sean Rameswaram: And now we've got sophisticated machines lying.
--:-- James Vincent: And you're hoping that we go to trusted sources.
--:-- Sean Rameswaram: Well, I'm going to be really pedantic. And I'm going to say that the machines aren't lying. It's still other people lying, right?
--:-- James Vincent: Okay, fair.
--:-- Sean Rameswaram: The problem is the same. It is always people doing this for whatever reason they have.
--:-- James Vincent: The machines aren't doing this to us. We're doing it to ourselves.
--:-- Sean Rameswaram: And you're right that so far we've sort of failed these tests and now we're getting an even bigger hurdle and quite likely we're going to fail it too as well.
--:-- James Vincent: I've seen some people talking about the fact that, you know, most of human civilization has been conducted under conditions of mistrust in a way, right?
--:-- Sean Rameswaram: It's only a relatively small amount of time that we had these things called photographs, which were quite hard to fake and meant that you could believe the output.
--:-- James Vincent: And it's only a smaller amount of time that we've had this thing called digital video.
--:-- Sean Rameswaram: And if you filmed it, it probably happened and it was hard to fake.
--:-- James Vincent: So we have survived with a greater level of mistrust in the past.
--:-- Sean Rameswaram: I'm sure we can do it again.
--:-- James Vincent: I wish your confidence were contagious.
--:-- Sean Rameswaram: I'm leaning towards humanity had a good run. Maybe the machines will do better.
--:-- James Vincent: Well, they're having fun at least. You know, we can have an AI-generated Pope for the machines. We can have AI Catholicism. Amen.
--:-- Sean Rameswaram: Disappear, he does.
--:-- James Vincent: What's the point of crying?
--:-- Sean Rameswaram: James Vincent, Saint Vincent, he's a senior reporter at The Verge.
--:-- James Vincent: Thanks for having me on the show.
--:-- Sean Rameswaram: And I wrote a book called Beyond Measure, which is a history of measurement, which I promise you is much more exciting than it sounds.
--:-- James Vincent: Our show today was generated by Amanda Llewellyn.
--:-- Sean Rameswaram: It was edited by birthday boy, Matthew Collette.
--:-- James Vincent: It was fact-checked by Matthew and Avishai Artsy and Siona Petros.
--:-- Sean Rameswaram: And it was mixed by Paul Robert Mounsey.
--:-- James Vincent: It's today explained.
--:-- Sean Rameswaram: And be careful out there, it's only a matter of time before the AI can access the home wifi network.
--:-- James Vincent: Thank you.
