--:-- [Upbeat music fades out]
--:-- Sean Rameswaram: Hey, happy March! You've heard of March Madness. How about a March mattress?
--:-- [Stage direction: Sean Rameswaram speaking directly to the audience]
--:-- Sean Rameswaram: The weather's turning, your bed is yearning. Check out Mattress Firm. They're open all month, except at nighttime because they're big on sleep. But their website never closes at all. Head to mattressfirm.com slash podcast to learn how you can improve your sleep.
--:-- [Music transitions to a new segment]
--:-- Sean Rameswaram: This is Today Explained. I'm Sean Rameswaram. There's this crazy thing happening online right now that could make you question whether what you're hearing, whether what you're seeing, is real.
--:-- [Music transitions to a new segment]
--:-- Sean Rameswaram: It could affect who you trust, it could affect elections, but right now, it's mostly affecting porn. It kind of goes back to this long-standing rule of the internet. If it exists, there's porn of it.
--:-- [Music transitions to a new segment]
--:-- Asia Romano: I am an internet culture reporter for Vox. Asia's been writing about algorithms lately. The ability to build your own predictive algorithm has been there for years.
--:-- Asia Romano: Predictive algorithms take some stuff, and with that stuff, they can imagine totally new stuff. Recently, people have been using these predictive algorithms to make fake videos of real people.
--:-- Asia Romano: It's basically learning what your face looks like and then predicting what your face would look like mapped onto another image that is doing something else.
--:-- Asia Romano: Like if I were to show the computer, the neural network, enough photos of your face, it would learn what you look like based on all of the photos that I give it and then predict what your face would look like doing something else that I feed it.
--:-- Asia Romano: So if I give it a video of, I don't know, someone chopping wood, it would then be able to say, this is what he would look like chopping wood.
--:-- Asia Romano: Does that make sense?
--:-- Sean Rameswaram: No, because why would anyone want to watch a fake video of a podcast host chopping wood?
--:-- Asia Romano: But because there are a lot of pervy dudes on the internet, take one guess what they've used these algorithms for.
--:-- Asia Romano: Porn? Porn. Porn? Yeah, porn.
--:-- Sean Rameswaram: So then on September 30th of last year, a Reddit user named or using the handle DeepFakes posted two celeb fakes.
--:-- Asia Romano: And he basically posted this algorithmically generated image of Maisie Williams face spliced onto the body of a porn star.
--:-- Asia Romano: Maisie Williams, for those who don't know, is an actress from Game of Thrones. A girl is Arya Stark of Winterfell.
--:-- Asia Romano: And I'm going home.
--:-- Sean Rameswaram: This was a computer neural network, essentially learning what Maisie Williams' face looks like, and then using what it learned, quote unquote, to predict what her face would look like, mapped onto the features of a real porn star.
--:-- Asia Romano: Enter deepfakes. That's the name of a Reddit user, but also the name of a new genre of porn with famous faces realistically mapped onto the bodies of porn stars.
--:-- Asia Romano: So the thing about Reddit, if you're not familiar with Reddit, is that A, a lot of dudes are on Reddit.
--:-- Asia Romano: And B, it has a history of serving as a distribution center for a lot of this stuff that's really dicey.
--:-- Asia Romano: Like in 2014, it was essentially the main distribution center for the famous Jennifer Lawrence leak of nudes.
--:-- Asia Romano: Which were real. Yes, which were real, which were basically actual leaks of actual nudes, not just of her, but of a lot of famous women.
--:-- Sean Rameswaram: Reddit eventually stepped in and shut down all these distribution centers. No more stolen photos of naked celebrities.
--:-- Asia Romano: But late last year, DeepFakes, this anonymous Reddit user, came up with something Reddit hadn't banned yet.
--:-- Asia Romano: And it was original work in a totally creepy way.
--:-- Sean Rameswaram: What he did was he went ahead and started his own subforum on Reddit called also called DeepFakes.
--:-- Asia Romano: Named after himself. Named after himself.
--:-- Sean Rameswaram: So then there was a whole subforum devoted to this.
--:-- Asia Romano: And you had tons of people trying to do this on their own.
--:-- Asia Romano: And eventually he released the code.
--:-- Asia Romano: And when he released the code, he basically gave everyone the magic tools to do this themselves without having to fumble and do their own guesswork because he'd already done the guesswork.
--:-- Sean Rameswaram: Interestingly when he made the post with releasing the code the the image that he used as an example was Nick Cage's face being mapped onto Donald Trump's face.
--:-- Asia Romano: Yes, and like showed them meeting in the middle essentially in this terrible like Trump Cage hybrid.
--:-- Asia Romano: And Nick Cage because he was the star of Face Off. You want to take his face. Yes, his face or Nick Cage. I think just because he's a perpetual internet meme.
--:-- Sean Rameswaram: Because the internet loves Nick Cage.
--:-- Asia Romano: Exactly.
--:-- Sean Rameswaram: Got it. So between October and January, when Vice noticed it, you had all of these people doing these incredibly, increasingly creepy things with, again, all women, all celebrities, all of it without consent.
--:-- Asia Romano: That should go without saying.
--:-- Sean Rameswaram: So what is the legality here? Is there an existing law that applies to this where this manipulation is obviously illegal or is this something new altogether because it's being made by computers and not even people?
--:-- Asia Romano: Well, it's new to us, but it's not necessarily new to the courts because, again, algorithms have been around for a while.
--:-- Asia Romano: Questions about the legality of using algorithms to remix existing samples and so forth have been around for a while.
--:-- Sean Rameswaram: OK. In America, under U.S. copyright law, you can pretty much remix anything as long as it's quote unquote transformative and as long as it's not essentially infringing on directly on the profit of the original source material.
--:-- Asia Romano: When I told you about the Nick Cage and Donald Trump thing. Yeah. That would be considered a remix because it's like they are both celebrities or both public figures.
--:-- Asia Romano: And this could be parody under US copyright.
--:-- Sean Rameswaram: But when you factor in the porn, what's happening is that you have a situation where A, someone's consent is being violated. Two people's consent are being violated.
--:-- Asia Romano: But also the porn market directly is being infringed upon because this is essentially not transforming the original work. It's meant to replace the original work.
--:-- Sean Rameswaram: You're saying this is like still porn and it's like a real violation of the porn in a way.
--:-- Asia Romano: Right, exactly. Like you didn't change enough substantially to make it a new thing. It's sort of incomprehensibly not legally a violation of the celebrity, but it is a violation of the porn you stole to make the deep fake.
--:-- Asia Romano: Right, exactly. Because it's really not her image that is getting violated. Because what you're doing is you're actually taking thousands or hundreds of images of her and putting them all together to teach this computer.
--:-- Asia Romano: So when the computer actually generates her face, it's generating something new. But it's generating something new onto an existing source. And it's that source that's getting replaced.
--:-- Sean Rameswaram: This sounds a lot like remixes. Like, I don't know, taking MIA's Paper Planes and chopping it up and making it sound new again.
--:-- Asia Romano: But even beyond the infringement is the ethical quandary which is that this is totally non-consensual and awful.
--:-- Sean Rameswaram: Reddit has finally stepped in and said okay we're going to ban deepfakes the subreddit.
--:-- Asia Romano: Yep, the user is still there. He can do whatever he wants.
--:-- Sean Rameswaram: They basically updated their content policy to put these videos and these photos under the category of involuntary porn.
--:-- Asia Romano: I think it's a category that's kind of unique to Reddit, but it covers a couple of facets, including revenge porn and including the leak of the nudes and so forth.
--:-- Sean Rameswaram: I mean, it's safe to assume that any celebrity would feel deeply violated by this.
--:-- Asia Romano: Right. And I think the use of the phrase involuntary is really crucial here because it covers a range of sins that all have to do with people taking your image out of your own hands, whether it was originally created by you or not.
--:-- Sean Rameswaram: I want to play you just another piece of music right now. This song is called Total Entertainment Forever, and it's by Father John Misty.
--:-- Asia Romano: All right.
--:-- Sean Rameswaram: So he said, Betting Taylor Swift every night inside the Oculus Rift.
--:-- Asia Romano: Father John Misty making maybe a provocative point about where our entertainment culture is going.
--:-- Sean Rameswaram: Right. About having sex with celebrities in virtual reality, which I guess is maybe like the next place this kind of thing is going to take us.
--:-- Asia Romano: Most people think it's dumb. Then this was shocking for a second until it's not.
--:-- Sean Rameswaram: And then the next shocking thing might be having sex with a celebrity in virtual reality until we get used to that.
--:-- Asia Romano: Is there some sort of... Is there a line? Or do we always just get used to the next technological development of perversion and violation of women's bodies and images?
--:-- Sean Rameswaram: Well, if we keep saying the line is here and then crossing the line, and this of course gets into all kinds of political ramifications and so forth, and then saying, oh, we're used to this now, let's get used to the next thing.
--:-- Asia Romano: Really, we sort of leave ourselves open to the possibility that the only thing that can really provide some sort of moral absolute is the technology itself.
--:-- Sean Rameswaram: When you have the ability to change someone face and make it look as though someone is doing and saying something that they're not, you open yourself up to all kinds of new waves of fake news and the spread of fake information.
--:-- Sean Rameswaram: Coming up, harnessing the exact same technology to make the president say whatever you want.
--:-- Sean Rameswaram: This is Today Explained.
--:-- [Music transitions to a new segment]
--:-- Sean Rameswaram: Expert mattress firm all different kinds of mattress waiting there for you that was a mattress haiku about how mattress firm wants to help you.
--:-- Sean Rameswaram: The experts at mattress firm got you covered with mattresses, obviously, but maybe less obviously. They've got you covered with headboards, adjustable bases, sheets, and bedroom decor.
--:-- Sean Rameswaram: Get to know your local Mattress Firm. Here's a conversation starter. They consider themselves America's neighborhood mattress store.
--:-- Sean Rameswaram: Mattress Firm can help you stretch your budget a little further when you're looking for ways to improve your sleep.
--:-- Sean Rameswaram: Go to mattressfirm.com slash podcast to see their latest deals.
--:-- Sean Rameswaram: Mattress Firm offers a 120-night sleep trial to ensure perfection and a 120-night low-price guarantee so you know you paid the perfect price.
--:-- Sean Rameswaram: Again, go to mattressfirm.com slash podcast to learn how your sleeping could be improved.
--:-- Sean Rameswaram: This is Today Explained. I'm Sean Ramas-Firm.
--:-- Sean Rameswaram: North Korea is a rogue nation which has become a great threat and embarrassment to China, which is trying to help, but with little success.
--:-- Sean Rameswaram: The president never said that. It sounds like him. He tweeted it, but it's not his voice. It's a recording generated by an algorithm.
--:-- Sean Rameswaram: Here's another one. North Korea has conducted a major nuclear test. Their words and actions continue to be very hostile and dangerous to the United States.
--:-- Sean Rameswaram: Now, as a radio person, I can sort of tell it's fake, but I'm not sure if my uncle who always forwards me garbage news from trashy sites could.
--:-- Sean Rameswaram: And what if the technology got a little better? Could I even still tell the difference?
--:-- Sean Rameswaram: It's sort of scary. So my name is Peter Eckersley. I'm the chief computer scientist at the Electronic Frontier Foundation.
--:-- Peter Eckersley: Yes, scary, but also maybe not. This is a technology that could be either good or bad, depending on how it's used.
--:-- Peter Eckersley: Peter and his computer science network just came out with a report this week on the use of malicious artificial intelligence, like deep fakes.
--:-- Peter Eckersley: His deep take? We need the researchers who are making these things to do a better job of thinking carefully about how to put their thumbs on the scale as they design things to ensure the beneficial applications outweigh the problems.
--:-- Sean Rameswaram: So how do they do that? This guy who made the deep fakes, he just said, hey, look what I made up.
--:-- Peter Eckersley: And then he just shared the algorithm and now people are still making them everywhere.
--:-- Sean Rameswaram: So what could he have done differently and what would you have other creators and coders and researchers do?
--:-- Peter Eckersley: Well, I think one thing he thought he was doing is warning everyone that this technology was out there.
--:-- Peter Eckersley: Whoops. Well, in a case like that, that may not be totally crazy because, of course, there are some people who are going to do this and tell everyone, which promotes a little bit of chaos, but at least we get to have this conversation.
--:-- Peter Eckersley: There are other people who might get a hold of this technology and then immediately wait to use it to intervene in a political campaign with no warning.
--:-- Peter Eckersley: And then we'd be having the conversation in retrospect, which is a much worse place to be. Like the Russians, right? The Russians could do that.
--:-- Sean Rameswaram: Exactly.
--:-- Peter Eckersley: So I think telling people isn't necessarily a bad action. As a researcher, you should always think carefully before you do that.
--:-- Peter Eckersley: And we're calling for a culture of people doing that a little more.
--:-- Peter Eckersley: I think the other thing that's important is if you're releasing something like this, it's better to release something where you can tell it's a fake.
--:-- Sean Rameswaram: And I think with these initial examples, at least, when you listen to that audio, you watch that video, you can tell it's not quite the real thing.
--:-- Peter Eckersley: Yeah, but I mean, you and I can, but not everyone can, right?
--:-- Sean Rameswaram: That's right.
--:-- Peter Eckersley: And one thing we're learning about the current US media landscape is that having experts be able to tell the difference isn't working all of the time for us to be able to tell the difference between truth and fabrication.
--:-- Peter Eckersley: And I think that a deeper political problem that people on both sides of American politics need to find ways to fix.
--:-- Sean Rameswaram: How do we have productive conversations where we might not agree on what needs to be done, but we at least have some path to agreeing on some facts and some evidence from both sides of the aisle?
--:-- Peter Eckersley: Well, a few ideas. I mean, the task that those companies are going to have is going to be complicated.
--:-- Peter Eckersley: And the first and most important thing is that they avoid censorship in response to these problems.
--:-- Peter Eckersley: Because I think a lot of people have the instinct to say, oh, we need to take all the fake stuff down, censor it.
--:-- Peter Eckersley: And the problem there is, of course, it's sometimes hard to tell when things are fake.
--:-- Peter Eckersley: And you can wind up doing more harm than good in some cases if you dive in there with censorship.
--:-- Sean Rameswaram: So instead, what I think would be really constructive is new user interfaces for people using those platforms.
--:-- Peter Eckersley: Where you have a bunch of sliders that say, look, how much are we going to weight the credibility of checking that's gone into a news story?
--:-- Peter Eckersley: When pieces are essentially opinion pieces? Can we review them to see how manipulative, essentially, they are in their style?
--:-- Peter Eckersley: And then just instead of trying to censor things based on that, just give users an option. How much of different types of content do they want to see?
--:-- Sean Rameswaram: There's a slider for those things. There can be some defaults, but we really want everyone to take ownership.
--:-- Peter Eckersley: In the same way that maybe you want to choose how many of your friend's baby photos you see or how much news you want to see at all, you'd have some opportunity to get your own perspective double-checked and the other side's perspective double-checked.
--:-- Sean Rameswaram: And I think that's the big, hard design problem that the technology companies face here.
--:-- Peter Eckersley: That sounds pretty reasonable, but I feel like so far so bad with Facebook fixing this stuff themselves.
--:-- Sean Rameswaram: Is there a way predictive algorithms could be used to help the cause? I don't know, like AI fact-checking or AI policing of fake audio and video?
--:-- Peter Eckersley: I think a lot of people have that fantasy, and it's a long-term vision that you can imagine AI being able to do this predictively.
--:-- Peter Eckersley: The reality at the moment is that AI reads at about second grade level, so it's pretty hard to get current machine learning algorithms to do really critical reading of subtle propaganda.
--:-- Peter Eckersley: So it can help a little bit if used cleverly, but there isn't going to be any shortcut around having to have huge numbers of humans with essentially journalists training to double-check sources, check facts.
--:-- Sean Rameswaram: And a lot of this is going to have to be continuous.
--:-- Peter Eckersley: It could be done using new technology in very creative ways.
--:-- Peter Eckersley: We could have APIs for fact-checking, scalable databases where people can look at things initially and give them a quick read and then keep updating as more information comes in about stories.
--:-- Sean Rameswaram: I think trying to build that technology would actually be very good for us as a society and a civilization in an era where we're struggling to tell truth from fabrication.
--:-- Peter Eckersley: We need new types of institutions to do a better job of that task for us transparently without censorship.
--:-- Sean Rameswaram: But that's what they were saying over at Skynet too and it did not work out for them, Peter.
--:-- Peter Eckersley: Fortunately I don't think we're going to exactly be living in a Skynet world the future is going to be far stranger both more beautiful and potentially more dangerous than we expect but if we plan in advance think carefully about what we're building I think we can actually make a more awesome and excellent future with AI.
--:-- Sean Rameswaram: Peter Eckersley is the chief computer scientist for the Electronic Frontier Foundation.
--:-- Sean Rameswaram: I'm Sean Ramasviram.
--:-- Sean Rameswaram: This is Today Explained.
--:-- Sean Rameswaram: Folks, let me be clear. This is Barack Obama.
--:-- Sean Rameswaram: Follow Today Explained on Twitter at today underscore explained.
--:-- Sean Rameswaram: You know what? I just went to mattressviram.com and saw that they're having a big price drop right now, which, I mean, as a consumer is exactly what you want to hear from America's neighborhood mattress store, right?
--:-- Sean Rameswaram: Don't sleep on a good deal, friends.
--:-- Sean Rameswaram: You can go to mattressfirm.com slash podcast right now to learn how you can improve your sleep.
