--:-- Sean Rameswaram: Welcome to Today Explained. I'm Sean Rameswaram. Today, San Francisco could become the first American city to ban government agencies from using facial recognition technology. Sigal Samuel writes about technology for Vox's Future Perfect.
--:-- Sigal Samuel: If you do marketing in your life, MailChimp has an all-in-one marketing platform that allows you to manage more of your marketing activities from one place. And who among us doesn't want to manage all of the things we have to do from one place? You can learn more at MailChimp.com.
--:-- Sean Rameswaram: Science fiction almost got it right. Face recognition technology has become part of daily life. The ability of computers to identify faces has gotten a hundred times better, a million times faster, and exponentially cheaper. Imagine if instead of fingerprints, we could use people's faces to find out who they are or even where they are.
--:-- Sigal Samuel: It was all fun and games in the movies, but now facial recognition technology is here, and we have to deal with it. The technology can single us out in real time as we go about our daily business, often without us ever knowing.
--:-- Sean Rameswaram: Today, San Francisco is voting on something called the Stop Secret Surveillance Ordinance. In addition to being a delightful tongue twister, it is also a really important ordinance that if it passes, will all out ban the use of facial recognition technology by government agencies.
--:-- Sigal Samuel: Seagal Samuel writes about technology for Vox's Future Perfect. It would also require other types of surveillance tech, like license plate readers, to only be adopted by cities after they receive a full public airing and they're voted on. And it would also require city governments to get super clear policies on how they're going to use any surveillance tech.
--:-- Sean Rameswaram: So, San Francisco wants to ban the government from using this kind of technology. Are they already using it? So, there are a lot of agencies that really favor this technology. The FBI, for one, has a huge database with millions of faces in it.
--:-- Sigal Samuel: If you're an American, there's like approximately a one in two chance that your face is in one of these databases. A lot of people don't realize like quite how widespread this is already.
--:-- Sean Rameswaram: One Georgetown law study found that law enforcement face recognition specifically impacts over 117 million Americans. How? You know, this facial recognition technology is already being used by police departments in many states.
--:-- Sigal Samuel: We don't know exactly how many police departments, which, where, because they're not forced to, you know, make that information public. But cops can use this to, you know, if they pull you over, they just show up at your door, wherever they find you. They can use this technology to identify who you are just by scanning your face.
--:-- Sean Rameswaram: I'm personally dying to know, are we already at the point where it's like the Jason Bourne movies where they're like searching for someone in a train station? Where the hell is he? You cannot afford to lose this guy, people.
--:-- Sigal Samuel: And all of a sudden, every camera starts turning and focusing in on people's faces and eventually, boom. Okay, there he goes. They find the guy they're looking for. Is that already happening? This is sort of a bit of the nightmare scenario, right? It's creepy.
--:-- Sean Rameswaram: Yeah, that's creepy because that puts the burden of proof on you to prove that you're not the person that the AI system is saying you are. Which, footnote, un-American. Not how it works here.
--:-- Sigal Samuel: Right. Airports are increasingly installing these systems. We've seen, like, headlines in the past few weeks about these systems being used to check you into your flight instead of you using your passport or your boarding pass.
--:-- Sean Rameswaram: It just, like, looks at your face. But like these systems are also just installed in airports so they can theoretically just surveil who's going where.
--:-- Sigal Samuel: So yes, it's happening? Yeah, I mean, I don't think it's quite at foreign identity levels yet. But in a country like China, I think yeah, they're getting pretty close to that if not already there.
--:-- Sean Rameswaram: And ordinances like San Francisco are designed to stop America from going further along in the China direction. Are there other places besides San Francisco that are, you know, scared of going down the Jason Bourne slash China road here?
--:-- Sigal Samuel: I do think San Francisco could become kind of the model city for a lot of other cities and states if San Francisco does pass this. Berkeley, Oakland, they're already considering very similar ordinances. And also Washington State, Massachusetts, they're also weighing bans.
--:-- Sean Rameswaram: The U.S. Senate is considering a bipartisan bill to regulate the commercial use of facial recognition tech. So we are seeing more and more movement around this. OK, so a handful of American cities and states are feeling ambivalent to afraid about this technology.
--:-- Sigal Samuel: What about the technology community itself? Actually, in April, a lot of leading AI researchers, including somewhere affiliated with these big companies like Amazon and Google, wrote an open letter to Amazon saying you should not be selling your facial recognition technology to law enforcement until that tech can go through an independent review process to assess its civil liberties impact and pass that test.
--:-- Sean Rameswaram: How did Amazon respond? So Amazon has actually tried really hard to quash a vote by its shareholders that would seek to bar Amazon from selling that tech to law enforcement.
--:-- Sigal Samuel: In the end, Amazon was essentially forced to let that vote take place, and it's going to take place later this May.
--:-- Sean Rameswaram: Do you think there could be like a large-scale review of this technology and what constitutes its ethical use?
--:-- Sigal Samuel: I think realistically what's going to happen is we probably will see increased regulation of this technology. I doubt that we're going to see it abandoned wholesale because it is very attractive to police departments and ICE and folks like that who see it as a method to increase their efficiency.
--:-- Sean Rameswaram: After the break, facial recognition software isn't just creepy. It's also racist because, of course.
--:-- Sigal Samuel: At the top of the show, I told you that MailChimp has this all-in-one marketing platform that allows you to manage more of your marketing activities from one place. Guess what that will help you do? It'll help you grow your business if you're into that kind of thing.
--:-- Sean Rameswaram: MailChimp wants to help you get to know who to talk to, what to say, and when to say it, and the best channel to deliver the message. And that's what its marketing tools are going to help you do.
--:-- Sigal Samuel: The complete marketing platform has everything you need to start marketing your business today. And the folks at MailChimp will be there to help you as your business grows and needs new capabilities.
--:-- Sean Rameswaram: Let MailChimp help make you smarter. Learn more at MailChimp.com.
--:-- Sigal Samuel: Even if you're a perfectly law-abiding citizen with a halo over your head, you shouldn't take too much comfort in this facial recognition tech. It's not always accurate.
--:-- Sean Rameswaram: It is pretty accurate if you're a white male. How accurate is it if you're a white male? It only will misidentify you maybe 1% of the time.
--:-- Sigal Samuel: Okay, that sounds pretty good. But if you're a woman or a person of color or both, it's a lot worse. How much worse? More like a 35% chance of being misidentified.
--:-- Sean Rameswaram: Wow. Yeah. Last year, the ACLU wanted to test how much of a problem this is. And they actually ran a facial recognition test and they found that Amazon's facial recognition system, which is called Rekognition, actually matched, wrongly, 28 members of Congress to criminal mugshots.
--:-- Sigal Samuel: This is happening largely because these AI systems are trained on data that is biased against people of color. Systems that have been fed more images of, say, white men, so they're better at recognizing white men, properly identifying them. But the AI systems haven't been trained on lots of images of women, of people of color, so it's worse at identifying them correctly.
--:-- Sean Rameswaram: Didn't Google, like, tag black people as gorillas or something awful like that some time back? It was a glitch. Like, it was totally a mistake.
--:-- Sigal Samuel: That was probably the most sort of glaring example of this. Again, you know, their system had not been trained on enough images of African-American faces. And so when the system saw this, it just didn't know what to do with it.
--:-- Sean Rameswaram: There's also this very strange lawsuit going around right now. Basically, a teenager is suing Apple for a billion dollars. 18-year-old Yuz Mainba says the tech giant's facial recognition software wrongly linked him to a series of thefts from Apple stores.
--:-- Sigal Samuel: Basically, what happened is last autumn, police showed up at his door at four in the morning to arrest him for these thefts. Beyond being creepy, that's just rude. Four in the morning?
--:-- Sean Rameswaram: I know. Pretty harsh timing. I'm guessing this guy wasn't white? Nope.
--:-- Sigal Samuel: He then subsequently got summons from multiple states, alleging that he committed thefts in Apple stores like all over the map. And he was saying, I didn't do any of this. What the hell? And he says it caused him so much emotional distress. And that's why he's suing for one billion.
--:-- Sean Rameswaram: And what did Apple say? Apple denies that it uses facial recognition tech in its stores. But the teenager alleges that they did use this tech, and the teenager is actually going off the police detective's hypothesis.
--:-- Sigal Samuel: So whatever the truth turns out to be in this particular case, this high-profile, very expensive lawsuit is just emblematic of this, like, much larger wave of backlash that's happening across the country.
--:-- Sean Rameswaram: So why haven't the people behind these technologies or the people using them, be it Amazon or government agencies, fed more data into these systems so that they're more accurate? Why haven't they fed them more pictures of white women or black women or black men?
--:-- Sigal Samuel: They have, to some degree, been working on feeding more faces of different types of people into these systems. But that's only part of the problem. Another part of the problem is that these systems just kind of amplify pre-existing biases in our society.
--:-- Sean Rameswaram: Basically, human bias can creep into these AI systems. If communities of color have been over-policed in the past, which they have been, like black communities, then any AI system that is trained on that kind of data is going to just like replicate that bias.
--:-- Sigal Samuel: Are communities of color freaking out about this? Are they even aware that it's happening? Yeah. Other communities that are kind of vulnerable in the U.S. right now, let's say Muslim communities, undocumented immigrants, are really, really concerned about the use of facial recognition tech.
--:-- Sean Rameswaram: I know, especially, for example, the Muslim community in California has been super adamant about fighting this technology. And one woman who works for the Council on American-Islamic Relations recently was chatting with me and she told me, you know, let's say you go to mosque and you know these facial recognition technologies are operating around you.
--:-- Sigal Samuel: And you know that the U.S. has a history, especially since 9/11, of kind of surveilling Muslims at mosques. And you start to be afraid to even like show your face in public in certain places.
--:-- Sean Rameswaram: So it can have an impact on, you know, freedom of assembly and things like that. Do we know if anything like this has happened already, targeting specific racial groups or ethnic communities with facial recognition technology?
--:-- Sigal Samuel: To my knowledge, we don't know of it being used in nefarious ways in circumstances like those. But it is coming up now as a possibility because mosques, churches, synagogues, places like that are increasingly worried about security in the wake of lots of shootings and tragedies.
--:-- Sean Rameswaram: And so some people are saying, hey, let's bring facial recognition tech into our spaces. And others are saying, whoa, whoa, whoa, like that's actually a terrible idea. That's going to make us more prone to being surveilled.
--:-- Sigal Samuel: It kind of feels like even if this gets 100% accurate, it might just amplify a lot of deeper problems.
--:-- Sean Rameswaram: Just a few weeks ago, the AI Now Institute released a report on this question. And they said, like, let's just consider that sometimes de-biasing your AI system is not necessarily a good thing.
--:-- Sigal Samuel: Let's say you're an African-American. And right now, facial recognition tech is not great at identifying your face. Maybe you kind of want to keep it not great at identifying your face because your community has already been over-policed.
--:-- Sean Rameswaram: And if the police gets better at identifying you and the rest of your community, that might make it actually worse for you. So the AI Now report basically argued that we need to not just focus on technical de-biasing, but look at how these technologies are actually used in the real world, where we have histories of racial injustice, et cetera, and then decide who benefits from using these technologies.
--:-- Sigal Samuel: You know, just because you make a technology work just as well on everyone, it doesn't mean it's working just as well for everyone.
--:-- Sean Rameswaram: On a wider scale, like this is the basic bargain that we've all been wittingly or unwittingly striking with big tech for years now is like we're trading our data and our privacy for convenience, right? So like Google and Facebook offer us these cool free services and they're not asking for our money.
--:-- Sigal Samuel: All we have to give up in return is our data or our privacy. But our data and our privacy are actually a pretty huge deal.
--:-- Sean Rameswaram: And I think the big problem is people see AI through this sort of techno-solutionism lens and they just think, oh, this cool, shiny new gadget, it'll solve everything. It'll make our lives more efficient.
--:-- Sigal Samuel: People also think AI will make our judgments more impartial and objective. So there's a real sort of seductive logic around it that we're just starting to see people push back more against that recently.
--:-- Sean Rameswaram: We didn't even talk about how game people were to have Facebook suggest tags for our friends and our photos or to pay with our face or whatever. If so many people are just going to hit agree and sign up, is the spread of this technology inevitable?
--:-- Sigal Samuel: It does seem like there's been this huge wave toward adopting this technology on a really large scale. But the past month, we've seen so much activism against this and lawsuits and, you know, just public outcry.
--:-- Sean Rameswaram: We could be starting to see real results of that activism. We could be a society of people who potentially fight back against this if that's not the reality we want to live in.
--:-- Sigal Samuel: Seagal Samuel writes about technology and artificial intelligence for Vox's Future Perfect. I'm Sean Rameswaram from Today Explained. Thank you to MailChimp for supporting the show today.
--:-- Sean Rameswaram: One last reminder that MailChimp's all-in-one marketing platform allows you to manage more of your marketing activities from one place. So you don't have to move while you make your marketing smarter and faster and better. Learn more at MailChimp.com.
