Main Topic: The episode discusses the growing concerns and backlash against facial recognition technology, particularly in the context of racial justice and police brutality.
Key Points:
Participants:
Takeaways:
Theme music fades out
Host (Sean Rameswaram)
Welcome to Today Explained. I'm Sean Rameswaram.
Support for this show comes from ServiceNow. AI is only as powerful as the platform it's built into, don't you know? That's why it's no surprise that more than 85% of the Fortune 500 use the ServiceNow AI platform. Didn't know that. While other platforms duct tape tools together, ServiceNow seamlessly unifies people, data, workflows, and AI, connecting every corner of your business. And with AI agents working together autonomously, anyone in any department can focus on the work that matters most. I've never met an AI agent. Learn how ServiceNow puts AI to work for people at servicenow.com.
Not all journalism is the same. Take The Guardian. Our coverage has something unique. Fierce independence. Nobody owns us or tells us what we can and can't say. So we're free to report the whole picture. We connect what's happening in Washington to the rest of the globe. Expose corruption wherever we find it. And give fresh perspective on everything. From wellness and soccer to culture, the climate and more. Read, watch, and listen to The Guardian for free at theguardian.com.
A while back on the show, we talked about the dangers of facial recognition technology. And we talked about the first city to ban it, San Francisco. But over the last few weeks, the anti-facial ID movement has gone from a trickle to a flash flood.
Until now, a lot of the conversation has been around privacy. These days, it's racial justice. Amazon says it is temporarily banning police and law enforcement from using its controversial facial recognition software. The decision follows weeks of protests against police brutality in the wake of George Floyd's death. IBM said that it is getting rid of their facial recognition programs. Microsoft is urging Congress to put more regulation on facial recognition technology. IBM CEO sent a letter to Congress saying that he wants to work with Washington to promote justice and create a dialogue on whether facial recognition software should be used by local police.
But it's not just big tech. Boston just became the largest city after San Francisco to ban government use of facial recognition technology. Boston should not be using racially discriminatory technology and technology that threatens our basic rights. And now the coders themselves are speaking out. This week, the Association for Computing Machinery, the world's largest computing group, wrote this in a statement. The technology too often produces results demonstrating clear bias based on ethnic, racial, gender, and other human characteristics. Such bias and its effects are scientifically and socially unacceptable.
Guest introduction
I asked Diana Howard why all of this is happening now. She's a roboticist who also teaches about ethics, robots, and AI at Georgia Tech.
Diana Howard
It's because with George Floyd, the word I keep hearing is an awakening, that people started to realize that black folks, as well as black and brown folks, but black folks specifically have been kind of on the wrong side of some of these practices and policies with police and with law enforcement. And facial recognition has, of all the AI applications that have been out there, facial recognition has had kind of the most movement into some of these fields with law enforcement.
We know that AI is used by Border Patrol. The FBI and U.S. Immigration and Customs Enforcement are reportedly using driver's license photos for facial recognition searches without their owner's knowledge or permission. We know that the court system for criminal recidivism determining who should be paroled and they use AI quite a bit. What's scary is when we see AI is being used in areas where our civil liberties can be violated.
How does that work exactly? If you think about facial recognition, it means you need a lot of faces to put into the system. Now, where do you think those images are taken from? They're actually taken from police records. They're taken from the web. They're taken from all the places you can think of that you have images.
Well, we already know if you look online and you look at media, they historically have represented black and brown people in a negative light. And then if you're adding in the fact that you're using police records, we already know that there's a bias, and everyone knows this, with respect to specifically black men. That's the data that's being fed.
And so if most of the data you have that's being fed into AI basically says 80% of these individuals that look like this are perpetrators, it means that if you come in and you have someone innocent, well, you know, you're going to match to a larger percentage of the database of negative images because that's what's been learned.
Say you're innocent, but you're brought in. Well, first off, a mugshot is taken. You have fingerprints. Even if you are exonerated you now in the system. You have been targeted which then means you searchable. So it becomes a systematic cycle where it very difficult to break out especially because it AI.
Have we seen any recent examples of AI wrongly accusing a person of color? There was a robbery in Michigan. Someone stole nearly $4,000 in watches from this Detroit Shinola Boutique. Robert Williams, who has no criminal history, was wrongly arrested for the crime on his front lawn in front of his wife and young daughters. Facial recognition software falsely matched his driver's license photo to security footage of a shoplifter.
Julia comes out while they're putting me in handcuffs, my oldest daughter, and I tell her, hey, Juju, go back in the house. They took the images of the camera. They then put it within a system and created a mugshot. A detective turns over a picture of a guy inside Shinola and he's like, so that's not you? I look, I said, no, that's not me. He turns another paper over and he said, I guess that's not you either. I picked that paper up and hold it next to my face. I said, this is not me. I'm like, I hope y'all don't think all Black people look alike.
They then found out, oh, sorry, wrong person, which means that also there's human biases that was introduced along with the AI bias. And so you had a double whammy.
Guest introduction
Host (Sean Rameswaram)
There's been a lot of discussion in the last month about defunding the police or radically rethinking police instead of making, you know, smaller fixes. Do we need to radically rethink AI too? Defunding the police basically means you don't have as many police officers. But you still have to do something in terms of, quote-unquote, law and order. What's going to happen? You're going to bring in AI by default, right? That's like what people do. You bring in automation when you reduce your workload, which means we need to be able to fix AI.
If you abolish AI and you also abolish the workforce, I don't know what you would replace it with. But should AI have a place in policing at all? I mean, is there an example of law enforcement using it responsibly?
Guest introduction
So Englewood, a neighborhood in Chicago, actually had a nice way of using AI. They had an AI tool, basically predictive policing, that identified hotspots. Traditionally, what police do is they would send police officers to places that have hotspots, you know, to make sure that they find the crime before it happens or find individuals.
Instead, what they did is that they used those hotspots to then engage with the community leaders within those hotspots. So they worked with the community to actually alleviate the crime. And crime went down. Robberies went down. They used the AI not to deploy police officers. They used AI as a tool to have much more human-human collaboration.
So that's rethinking what you're doing. And so if you think about the justice system, instead of using AI for criminal recidivism, how do you use AI to do something else, right? Like, what is it that enables human interaction to then fix the problem? It just seems a lot like playing with fire to me, even if people or an institution have the best intentions.
I mean, if we're going to use AI, how do we keep it under control? You know, one of the things we don't have, which I think might actually work, is this aspect of accountability, right? And it's because if you think about what AI was used for, it was, you know, to make our lives better. So we have, you know, nice chatbots online. It wasn't really used for things that can impact us in our day-to-day lives.
But when we have something that impacts us, and I would say like drugs and medicine, right? Like there's an organization, example, the FDA, that monitors. Like I, as a company, if I decide to make a drug in my house, I can't just release it. Like I have to go through a process. And sometimes it takes long and sometimes it gets people frustrated. But there's a process where I have to show, like, this drug has this positive benefit. And guess what? It does have these harms as well. And sometimes FDA is like, yeah, you can't do it. You got to go back to the drawing board. Sometimes FDA is like, OK, this is an acceptable harm, but you have to put it on the label.
If we think about AI for those things that impact our civil liberties, like facial recognition, You know, when a company is developing, it should go through this whole aspect of, these are the positive things, these are the harms, and we've done the studies. And you have a third body basically saying, yeah, no, not acceptable.
Guest introduction
Up next, how the federal government could manage artificial intelligence or even ban law enforcement from using it.
Theme music transitions to commercial break
Theme music fades back in
Support for Today Explained comes from Built Rewards. These days, it seems like everybody has a rewards program. You swipe your card and you get one cent on the hundred dollars. But with your rent, frequently, you just pay month after month. You never get anything from it other than that roof over your head. Not anymore, says Built Rewards.
Built Rewards says it lets you earn points on your rent. By paying rent through Built, they say you can earn flexible points that can be redeemed toward hundreds of hotels and airlines, perhaps a future rent payment, your next Lyft ride, and so much more. Your home isn't just where you sleep. It's your community. And Built wants you to go out and do things in the community. You can eat at your favorite local restaurants, then earn more points. You can get VIP treatment at certain fitness studios. You can enjoy exclusive experiences just for Built members every month.
When you're ready to travel, they say Built points can be converted to miles or hotel points all around the world. You can earn points on rent and around your neighborhood wherever you call home by joining Built.com slash Explained. That's J-O-I-N-B-I-L-T dot com slash Explained.
Theme music transitions to co-host introduction
Seagal Samuel, co-host of the Future Perfect podcast here at Vox.
Seagal Samuel
Thanks for having me on the show.
Host (Sean Rameswaram)
In the first half of the show, Ayanna Howard mentioned that she wanted to see something like an FDA for artificial intelligence. Is there any indication that the federal government is going to do something like that?
Seagal Samuel
So far, I haven't seen any indications that the federal government is actually willing to do that. But it's not only Ayanna who's calling for that. Definitely, I'm seeing an increase in calls for that from groups like the Algorithmic Justice League, which is headed up by Joy Bolamwini, researcher at MIT.
Last year, I did a piece for Vox. It was presenting a crowdsourced algorithmic bill of rights. And a lot of the experts I spoke to for that piece also said this same idea. For example, Ben Schneiderman, who's a computer science professor at University of Maryland, he said that we need to create what he called a National Algorithm Safety Board.
If you're a major company and you're about to put out a major algorithm or you're a bank and you're going to change the way credit is assigned, I think it's appropriate that you come before the National Algorithm Safety Board and that there's a review. Just like we have oversight boards for airplanes. You know, they investigate the crashes. We need the same thing for facial recognition.
So we've got some proposals for these oversight committees, but is anyone working on legislation that could make these a reality somewhat quicker? Something pretty exciting just happened on June 25th, where lawmakers in the House and the Senate jointly introduced this new legislation that would effectively ban law enforcement from using facial recognition in the U.S.
It's a pretty big deal. It's called the Facial Recognition and Biometric Technology Moratorium Act of 2020. That was a mouthful. It's sponsored by Senators Markey and Merkley and Representatives Jayapal and Presley. The criminal justice system is already rigged against Black and brown Americans. We have to act with urgency to ensure that this technology doesn't become a new tool in the 21st century to subjugate and fill the system with people of color.
Basically, it would right away put a stop to U.S. federal agencies like the FBI from using facial recognition. And it would also require state police agencies to put in place similar policies banning the use of the tech if they want to be able to receive certain federal grants.
So if that passes, that could be pretty significant, especially since we're not talking about a moratorium here. We're talking about a permanent ban. It's the kind of legislation that would stay in effect until new legislation is passed to unban it.
How likely is that to pass? You know, a few weeks ago, I would have been more cynical and I would have said, I don't think it's that likely to pass. But now I actually think it's more likely, you know, this legislation came just one day after Robert Williams told his story to the press. That's the Black man in Detroit who was arrested falsely due to this racially biased facial recognition algorithm.
And this is all coming on the heels of the upswell in Black Lives Matter protests and the major national conversation that has sparked about facial recognition. So given the cultural climate we seeing now around all this stuff, I think that might've actually teed up quite nicely this moment where that kind of legislation might be more likely to pass.
So that's the political side of things, but we've seen companies taking steps on this too. I guess, how cynical should we be about companies sort of trying to police themselves?
Personally, I think that we have every reason to be skeptical and not too credulous of giant tech companies when they say they're going to regulate or put moratoria on the technologies that they're creating and selling, right? Their interest is in their own bottom line at the end of the day.
So, you know, IBM said, okay, we don't need to do facial recognition anymore, but it wasn't making that much money off of its facial recognition tech to begin with, so it's not a really big deal for it to pull out of that business. Notice that Amazon said, you know, we're doing this one-year moratorium on the technology to give Congress enough time to come up with regulations.
Let me actually just read you what Amazon said in its statement. It said, we hope this one-year moratorium might give Congress enough time to implement appropriate rules, and we stand ready to help if requested. So what does that mean, right? Translation, we want to help write the regulations so that these new rules won't totally destroy our ability to profit from this tech.
So I guess we have government regulation that seems like it's a bit away from being a reality. We have companies that are maybe saying that they want the government to regulate them, but could be actually trying to get in on the regulation themselves.
I mean, how hopeful are you that we can put the cat back in the bag here? It's funny. I tend to be quite cynical about this stuff. But I would say if ever there was a moment when it looked like we actually might be able to push back strongly in law against facial recognition, it's now.
A lot of that has to do with the Black Lives Matter protests that we've just been seeing over the past few weeks. During these protests, you saw the FBI plus police in various cities like Seattle, Austin, Dallas publicly explicitly asking citizens to send them videos of the protests so that they could capture visual images of the protesters and then use facial recognition to identify them by name so that they could punish them if they were damaging property or looting.
You know, and I think the news of that really alarmed people and it helped kind of fuel this national uproar over facial recognition. I think the other thing, which is a bit bittersweet, is that the protesters were this racially mixed population. You had, of course, a lot of black people. You also had white people. You had people of all backgrounds protesting together in the streets.
And so I think for some of the white people, some of the non-Black people at the protests, that was maybe the first time they came to realize this technology could actually be used on me. It could actually have harmful consequences for my life, too. And so more and more people outside of that Black community and outside of the kind of privacy nerd community are starting to see this as a real problem.
Theme music transitions to outro
Host (Sean Rameswaram)
Seagal Samuel is the co-host of the Future Perfect podcast at Vox. Future Perfect actually just released a new limited series called The Way Through. It's all about how to use philosophy and faith to help us out in times of COVID.
I'm Noam Hassenfeld, filling in for Sean Ramos for him. who'll be back on Monday.
The rest of the team is Muj Zaydi, Afim Shapiro, Jillian Weinberger, Bridget McCarthy, Amina Alsadi, and Halima Shah. Cecilia Lay checks our facts, and the Mysterious Breakmaster Cylinder makes our music.
We had help from Hannes Brown this week. And Liz Kelly Nelson is Vox's editorial director of podcasts.
One more thing. We're working on some kids' episodes for later this summer, and we want to know all the creative ways your kids have passed the time. Have they made up games, written a play, gotten into mysterious kinds of trouble? Or are they just really bored?
We'll listen to that too. Have them record a voice memo and email it to us. Todayexplained at vox.com. Or they can call and leave a message at 202-688-5944. That's 202-688-5944.
We're off tomorrow for the holiday weekend. We'll be back in your feed on Monday.
Theme music fades out