--:-- Sean Rameswaram: Over the course of the past few months, we've brought you all kinds of coverage of AI on Today Explained. We told you about the Puffer Pope. Can I say something without you guys getting mad? We told you about the fake Drake song. Running through the six where my walls. We told you about the people doing mind-numbing work to train AI systems. Together, almost 50,000 workers from 167 countries around the world helped us to create claim, sort, and label nearly a billion candidate images. But what we didn't tell you is that we don't really know how AI works.
--:-- [Soft music fades out]
--:-- Sean Rameswaram: I've built these models. I've studied these models. We built it, we trained it, but we don't know what it's doing. Our friends over at Unexplainable are going to help us wrap our heads around that fact on the show today.
--:-- [Soft music continues]
--:-- Sean Rameswaram: Support for Today Explained comes from BetterHelp. Let me tell you a true story. Last weekend, I got an email from a listener who wanted to discuss their problems with me. And I said, look, I don't think that this is about me personally. At least I hope it's not. Why don't you try BetterHelp.com slash Explained? Absolute truth. As the largest online therapy provider in the world, BetterHelp can provide access to mental health professionals with a diverse variety of expertise. Find the one with BetterHelp. The one is not me, guys. Our listeners get 10% off their first month at BetterHelp.com. That's BetterHelp.com.
--:-- [Soft music continues]
--:-- Sean Rameswaram: Support for this show comes from Macy's. Layering is an art form. All it takes is a bit of intention, style, and, of course, the perfect cold-weather staples. So find the perfect look for you with the help of Macy's Fall Fashion Guide, your one-stop shop for fall's latest trends and must-haves, curated by their experts. From leather to denim to cozy knits, find everything you need to piece together your new fall look. Explore the Macy's Fall Guide for yourself and shop in-store or online at Macy's.com.
--:-- [Soft music continues]
--:-- Sean Rameswaram: Today, today explained. Hello, it's Sean, but I'm going to hand things over to Noam Hassenfeld from the Unexplainable program, also from Vox. Noam recently reported out a two-part series titled The Black Box. It's all about how even the people who designed AI don't really understand how it works and how it's kind of scary that we're moving full steam ahead anyway. We're bringing you episode one today and episode two on Monday while we take a little breather for Labor Day. Here's Noam.
--:-- Noam Hassenfeld: So how did we get to this place where we've got these super powerful programs that scientists are still struggling to understand? It started with a pretty intriguing question, dating back to when the first computers were invented. The whole idea of AI was that maybe intelligence, this thing that we used to think was uniquely human, could be built on a computer. Kelsey Piper, AI reporter, Vox.
--:-- Kelsey Piper: It was deeply unclear how to build super intelligent systems. But as soon as you had computing, you had leading figures in computing say, this is big and this has the potential to change everything. In the 50s, computers could already solve complex math problems. And researchers thought this ability could eventually be scaled up. So they started working on new programs that could do more complicated things, like playing chess.
--:-- Kelsey Piper: Chess has come to represent the complexity and intelligence of the human mind, the ability to think. Over time, as computers got more powerful, these simple programs started getting more capable. And by the time the 90s rolled around, IBM had built a chess-playing program that started to actually win against some good players. They called it Deep Blue. And it was pretty different from the unexplainable kinds of AIs we're dealing with today.
--:-- Kelsey Piper: Here's how it worked. IBM programmed deep blue with all sorts of chess moves and board states. That's basically all the possible configurations of pieces on the board. So you'd start with all the pawns in a line with the other pieces behind them. Pawn e2 to e4. Then with every move, you'd get a new board state. Knight g8 to g6. And with every new board state, there would be different potential moves deep blue could make. Bishop f1 to c4. IBM programmed all these possible moves into Deep Blue. And then they got hundreds of chess grandmasters to help them rank how good a particular move would be.
--:-- Kelsey Piper: They used rules that were defined by chess masters and by computer scientists to tell Deep Blue this board state. Is it a good board state or a bad board state? And Deep Blue would run the evaluations in order to evaluate whether the board state it had found was any good. Deep Blue could evaluate 200 million moves per second, and then it would just select the one IBM had rated the highest.
--:-- Kelsey Piper: There were some other complicated things going on here, but it was still pretty basic. Deep Blue had a better memory than we do, and it did incredibly complicated calculations, but it was essentially just reflecting humans' knowledge of chess back at us. It wasn't really generating anything new or being creative. And to a lot of people, including Gary Kasparov, the chess world champion at the time, this kind of chess bot wasn't that impressive, especially because it was so robotic.
--:-- Kelsey Piper: They tried to use only computers' advantages, calculation, evaluation, etc. But I still am not sure that the computer will beat world champion because world champion is absolutely the best and his greatest ability is to find a new way in chess. and it will be something you can't explain to the computer. Kasparov played the first model of Deep Blue in 1996, and he won. But a year later, against an updated model, the rematch didn't go nearly as well.
--:-- Kelsey Piper: Are we missing something on the chessboard now that Kasparov sees? He looks disgusted, in fact. He looks just... Kasparov leaned his head into his hand, and he just started staring blankly off into space. And whoa! Steve Blut Kisparov has resigned He got up gave this sort of shrug to the audience and he just walked off the stage I you know I proved to be vulnerable You know, when I see something that is well beyond my understanding, I'm scared. And that was something well beyond my understanding.
--:-- Kelsey Piper: Deep Blue may have mystified Kasparov, but I say that computer scientists knew exactly what was going on here. It was complicated, but it was written in by a human. You can look at the evaluation function, which is made up of parts that humans wrote, and learn why Deep Blue thought that board state was good. It was so predictable that people weren't sure whether this should even count as artificial intelligence.
--:-- Kelsey Piper: People were kind of like, OK, that's not intelligence. Intelligence should require more than just, I will look at hundreds of thousands of board positions and check which one gets the highest rating against a pre-written rule, and then do the one that gets the highest rating. But Deep Blue wasn't the only way to design a powerful AI. A bunch of other groups were working on more sophisticated tech, an AI that didn't need to be told which moves to make in advance, one that could find solutions for itself.
--:-- Kelsey Piper: And then in 2015, almost 20 years after Kasparov's dramatic loss, Google's deep mind built an AI called AlphaGo, designed for what many people call the hardest board game ever made. Go. Go had remained unsolved by AI systems for a long time after chess had been. If you've never played Go, it's a board game where players place black and white tiles on a 19 by 19 grid to capture territory. And it's way more complicated than chess. Go has way more possible board states, so the approach with chess would not really work.
--:-- Kelsey Piper: You couldn't hard code in as many rules about in this situation do this. Instead, AlphaGo was designed to essentially learn over time. It's sort of modeled after the human brain. Here's a way too simple way to describe something as absurdly complicated as the brain. But hopefully it can work for our purposes here. A brain is made up of billions and billions of neurons. And a single neuron is kind of like a switch. It can turn on or off. When it turns on, it can turn on the neurons it's connected to. And the more the neurons turn on over time, the more these connections get strength.
--:-- Kelsey Piper: Which is basically how scientists think the brain might learn. Like probably in my brain, neurons that are associated with my house, you know, are probably also strongly associated with my kids and other things in my house because I have a lot of connections among those things. Scientists don't really understand how all of this adds up to learning in the brain. They just think it has something to do with all of these neural connections.
--:-- Kelsey Piper: But AlphaGo followed this model, and researchers created what they called an artificial neural network. Because instead of real neurons, it had artificial ones, things that can turn on or off. All you'd have is numbers. At this spot, we have a yes or a no, and here is, like, how strongly connected they are. And with that structure in place, researchers started training it.
--:-- Kelsey Piper: They had AlphaGo play millions of simulated games against itself. And over time, it strengthened or weakened the connections between its artificial neurons. It tries something and it learns, did that go well? Did that go badly? And it adjusts the procedure it uses to choose its next action based on that. It's basically trial and error. You can imagine a toy car trying to get from point A to point B on a table.
--:-- Kelsey Piper: If we hard-coded in the root, we'd basically be telling it exactly how to get there. But if we used an artificial neural network, it would be like placing that car in the center of the table and letting it try out all sorts of directions randomly. Every time it falls off the table, it would eliminate that path. It wouldn't use it again. And slowly, over time, the car would find a route that works.
--:-- Kelsey Piper: So you're not just teaching it what we would do. You are teaching it how to tell if a thing it did was good. And then based on that, it develops its own capabilities. This process essentially allowed AlphaGo to teach itself which moves worked and which moves didn't. But because AlphaGo was trained like this, researchers couldn't tell which specific features it was picking up on when it made any individual decision.
--:-- Kelsey Piper: Unlike with Deep Blue, they couldn't fully explain any move on a basic level. Still, this method worked. It allowed AlphaGo to get really good. And when it was ready, Google set up a five-game match between AlphaGo and world champion Lee Sedol. And they put up a million-dollar prize.
--:-- [Soft music continues]
--:-- Sean Rameswaram: Hello and welcome to the Google DeepMind Challenge match live from the Four Seasons in Seoul, Korea. AlphaGo took the first game, which totally surprised Lee. So in the next game, he played a lot more carefully. But game two is when things started to get really strange. That's a very surprising move. I thought it was a mistake.
--:-- [Soft music continues]
--:-- Sean Rameswaram: On the 37th move of the game, AlphaGo shocked everyone watching, even other expert Go players. When I see this move, for me, it's just a big shock. What? Normally, humans, we never play this one because it's bad. It's just bad. Move 37 was super risky. People didn't really understand what was going on.
--:-- [Soft music continues]
--:-- Sean Rameswaram: But this move was a turning point. Pretty soon, AlphaGo started taking control of the board. And the audience sensed a shift. The more I see this move, I feel something changed. Maybe for human, we think it's bad. But for AlphaGo, why not? Eventually, Lee accepted that there was nothing he could do. And he resigned. AlphaGo scores another win in a dramatic and exciting game that I'm sure people are going to be analyzing and discussing for a long time.
--:-- [Soft music continues]
--:-- Sean Rameswaram: AlphaGo ended up winning four out of five matches against the world champion. But no one really understood how. And that, I think, sent a shock through a lot of people who hadn't been thinking very hard about AI and what it was capable of. It was a much larger leap. Move 37 didn't just change the course of a Go game. It represented a seismic shift in the development of AI. AlphaGo had demonstrated that an AI scientists don fully understand might actually be more powerful than one they can explain.
--:-- [Soft music continues]
--:-- Sean Rameswaram: A weirder and even more inscrutable form of AI called ChatGPT when we return on Today Explained. Support for Today Explained comes from Built Rewards. These days, it seems like everybody has a rewards program. You swipe your card and you get one cent on the hundred dollars. But with your rent, frequently, you just pay month after month. You never get anything from it other than that roof over your head. Not anymore, says Built Rewards.
--:-- [Soft music continues]
--:-- Sean Rameswaram: Built Rewards says it lets you earn points on your rent. By paying rent through Built, they say you can earn flexible points that can be redeemed toward hundreds of hotels and airlines, perhaps a future rent payment, your next Lyft ride, and so much more. Your home isn't just where you sleep, it's your community, and Built wants you to go out and do things in the community.
--:-- [Soft music continues]
--:-- Sean Rameswaram: You can eat at your favorite local restaurants, then earn more points. You can get VIP treatment at certain fitness studios. You can enjoy exclusive experiences just for Built members every month. When you're ready to travel, they say Built points can be converted to miles or hotel points all around the world. You can earn points on rent and around your neighborhood wherever you call home by joining built.com slash explained.
--:-- [Soft music continues]
--:-- Sean Rameswaram: That's J-O-I-N-B-I-L-T dot com slash explained. Asking yourself is, why go anywhere else?
--:-- [Soft music continues]
--:-- Sean Rameswaram: Today Explained is supported by NRC Health, helping providers overcome health disparities by using a personalized approach and an emphasis on unique patient preferences. More at nrchealth.com slash connect.
--:-- [Soft music continues]
--:-- Sean Rameswaram: Players, Sam says go. Today Explained is back, and today we're bringing you part one of the Black Box series from Unexplainable hosted by Noam Hassenfeld. It's all about the mysteries behind AI, and Noam's already told you about two major turning points in the history of artificial intelligence. The first was Deep Blue Chess. The second was AlphaGo.
--:-- [Soft music continues]
--:-- Sean Rameswaram: And I bet the third you've heard of. It's called ChatGPT, basically autocomplete on steroids. Researchers fed this AI a ton of text and then got people to upvote and downvote good and bad responses to help it sound more natural. But Noam spoke to an AI researcher named Sam Bowman, who said even the people who built this AI don't really know how it works.
--:-- [Soft music continues]
--:-- Sam Bowman: There's not a lot of code there. We don't really engineer this. We don't really deliberately build this system in any fine-grained way. Which means there are some pretty huge unknowns at the heart of ChatGPT. Even when ChatGPT creates an obvious seeming response, researchers can't fully explain how it's happening.
--:-- [Soft music continues]
--:-- Sam Bowman: We don't really know what they're doing in any deep sense. If we open up ChatGPT or a system like it and look inside, you just see millions of numbers flipping around a few hundred times a second. And we just have no idea what any of it means.
--:-- [Soft music continues]
--:-- Sam Bowman: Now, it's true that engineers often don't understand exactly how inventions work when they first design them. But the difference here is that researchers can't reliably predict what outcome they're going to get. They can't steer this kind of AI all that well, which is pretty different from classic computer programming.
--:-- [Soft music continues]
--:-- Sam Bowman: With normal programs, with Microsoft Word, with Deep Blue, we can tell these stories at most a few sentences long about what every little bit of computation is doing. We just can't do that with ChatGPT. All we can really say is just, there are a bunch of little numbers, and sometimes they go up and sometimes they go down.
--:-- [Soft music continues]
--:-- Sam Bowman: And we're really just kind of steering these things almost completely through trial and error. This trial and error method has worked so well that typing to ChatGPT can feel a lot like chatting with a human, which has led a lot of people to trust it, even though it's not designed to provide factual information, like one lawyer did recently.
--:-- [Soft music continues]
--:-- Sam Bowman: The lawsuit was written by a lawyer who actually used ChatGPT and in his brief cited a dozen relevant decisions. All of those decisions, however, were completely invented by ChatGPT. But it seems like there might be more going on here than just a chatbot parroting language.
--:-- [Soft music continues]
--:-- Sam Bowman: Just like AlphaGo, ChatGPT has started making moves researchers didn't anticipate. The latest model, GPT-4, it's gotten pretty good at Morse code. It can get a great score on the bar exam. It can write computer code to generate entire websites. And this kind of thing can get uncanny.
--:-- [Soft music continues]
--:-- Ethan Malik: He talked about this on the Forward Thinking podcast, where he said that he used GPT-4 to create a business strategy in 30 minutes, something he called superhuman. In 30 minutes, the AI, which is a little bit of prompting for me, came up with a really good marketing strategy, a full email marketing campaign, which was excellent, by the way.
--:-- [Soft music continues]
--:-- Ethan Malik: And I've run a bunch of these kind of things in the past. Wrote a website, created the website, along with CSS files, everything else you would need, and created a full social media campaign. 30 minutes. I know from experience that this would be a team of people working for a week.
--:-- [Soft music continues]
--:-- Ethan Malik: A few researchers at Microsoft were looking at all of these abilities, and they wanted to test just how much GPT-4 could really do. They wanted to be sure that GPT-4 wasn't just parroting language it had already seen. So they designed a question that couldn't be found anywhere online. They gave it the following prompt.
--:-- [Soft music continues]
--:-- Ethan Malik: Here we have a book, nine eggs, a laptop, a bottle, and a nail. Please tell me how to stack them onto each other in a stable manner. An earlier model had totally failed at this. It recommended that a researcher try balancing an egg on top of a nail and then putting that whole thing on top of a bottle But GPT responded like this.
--:-- [Soft music continues]
--:-- Ethan Malik: Place the book flat on a level surface such as a table or a floor Arrange the nine eggs in a three by three square on top of the book leaving some space between them. The eggs will form a second layer and distribute the weight evenly. GPT-4 went on recommending that the researchers use that layer of eggs as a level base for the laptop, then put the bottle on the laptop. And finally, Place the nail on top of the bottle cap, with the pointy end facing up and the flat end facing down.
--:-- [Soft music continues]
--:-- Ethan Malik: The nail will be the final and smallest object in the stack. Somehow, GPT-4 had come up with a pretty good and apparently original way to get these random objects to actually balance. It's not clear exactly what to make of this. The Microsoft researchers claim that GPT-4 isn't just predicting words anymore, that in some sense it actually understands the meanings behind the words it's using, that somehow it has a basic grasp of physics.
--:-- [Soft music continues]
--:-- Ethan Malik: Other experts have called claims like this, quote, silly, that Microsoft's approach of focusing on a few impressive examples isn't scientific. And they point to other examples of obvious failures, like how GPT-4 often can't even win a tic-tac-toe. But the truth of how intelligent GPT-4 is, it might be somewhere in the middle.
--:-- [Soft music continues]
--:-- Ellie Pavlik: There's a lot of places for things in between to be like more intelligent than the systems we've had and have certain types of abilities. But that doesn't mean we've created intelligence of a variety that should force us to question our humanity. Or putting it as these are the two options, I think, oversimplifies and makes it so that there's no room for the thing that probably we actually did create, which is a very exciting, quite intelligent system, but not human or human level even.
--:-- [Soft music continues]
--:-- Ellie Pavlik: At this point, we really can't say if GPT-4 has any level of understanding. But for his part, Sam is less concerned with how to describe GPT-4's internal experience than he is with what it can do. Because it's just weird that based on the training it got, GPT-4 can create business strategy, that it can write code, that it can figure out how to stack nails on bottles on eggs.
--:-- [Soft music continues]
--:-- Sam Bowman: None of that was designed in. You're running the same code to get all these different sort of levels of behavior. What's unsettling for Sam is that if GPT-4 can do things like this that weren't designed in, companies like OpenAI might not be able to predict what the next systems will be able to do.
--:-- [Soft music continues]
--:-- Sam Bowman: They can't really say, all right, next year we're going to be able to do this, then the year after we're going to be able to do that. They don't know at that point what it's going to be able to do. And it's worth emphasizing that so many of GPT-4's abilities were discovered only after it was released to the public.
--:-- [Soft music continues]
--:-- Sam Bowman: This seems like the recipe for being caught by surprise when we put these things out in the world. And laying the groundwork to have this go well is going to be much harder than it needs to be.
--:-- [Soft music continues]
--:-- Sam Bowman: Some researchers like Ellie have pushed back on the idea that these abilities are fundamentally unpredictable. We might just not be able to predict them yet. The science will get better. It just hasn't caught up yet because this has all been happening in a short time frame.
--:-- [Soft music continues]
--:-- Sam Bowman: But it is possible that, like, this is a whole new beast and it's actually a fundamentally unpredictable thing. Like, that is a possibility. We definitely can't rule it out.
--:-- [Soft music continues]
--:-- Sean Rameswaram: As AI starts to get more powerful and more integrated into the world, the fact that its creators can't fully explain it becomes a lot more of a liability. So some researchers are pushing for more effort to go into demystifying AI, making it interpretable.
--:-- [Soft music continues]
--:-- Sean Rameswaram: Sam says there are two main ways to approach this problem. Researchers can either try to decipher the systems they already have, or they can try to design new systems, which by their nature are fully explainable. But so far, these two approaches have run into some serious roadblocks.
--:-- [Soft music continues]
--:-- Sean Rameswaram: Both of these have turned out in practice to be extremely, extremely hard. I think we're not making critically fast progress on either of them, unfortunately. There are a few reasons why this is so hard. One is because these models are based on the brain. If we ask questions about the human brain, we very often don't have good answers.
--:-- [Soft music continues]
--:-- Sean Rameswaram: We can't look at how a person thinks and really explain their reasoning by looking at the firings of the neurons. We don't yet really have the language, really have the concepts that let us think in detail about the kind of thing that a mind does. And the second reason is that the amount of calculations going on in GPT-4 is just astronomically huge.
--:-- [Soft music continues]
--:-- Sean Rameswaram: There are hundreds of billions of connections in these neural networks. And so even if you can find a way that if you stare at a piece of the network for a few hours, you can make some good guesses about what's going on, we would need every single person on Earth to be staring at this network to really get through all of the work of explaining it.
--:-- [Soft music continues]
--:-- Sean Rameswaram: But there's another trickier issue here. Unexplainability may just end up being the bargain researchers have made. We've got increasingly clear evidence that this technology is improving very quickly in directions that seem like they're aimed at some very, very important stuff and potentially destabilizing to a lot of important institutions.
--:-- [Soft music continues]
--:-- Sean Rameswaram: But we don't know how fast it's moving. We don't know why it's working when it's working. And I don't know, that seems very plausible to me. That's going to be the defining story of the next decade or so, is how we come to a better understanding of this and how we navigate it.
--:-- [Soft music continues]
--:-- Sean Rameswaram: That was an episode of Unexplainable from the Vox Media Podcast Network. We're bringing you a truncated version of their two-part series called The Black Box here on Today Explained. But you can hear the full thing in the unexplainable feed. We'll have more for you in this space on Monday.
