--:-- [Theme music fades out]
--:-- Host: Ryan Knutson: When you hear the words AI apocalypse, often movies come to mind. Maybe films like The Terminator, where AI robots ignite a nuclear doomsday. It becomes self-aware at 2.14 a.m. Eastern Time, August 29th. Or maybe that Marvel movie, where AI tries to destroy the Avengers. Artificial intelligence. This could be it, Bruce. This could be the key to creating Ultron. Or maybe it's The Matrix, where humans have become enslaved by machines. A singular consciousness that spawned an entire race of machines. We don't know who struck first, us or them. This is a story as old as humans have been telling stories. It's a creation that escapes from our control. You know, it's a golem. It's a thing that we make that then turns on us. It's a Frankenstein, or it's Frankenstein's monster, I should say.
--:-- Host: Ryan Knutson: That's our colleague Sam Schechner. Lately, Sam's been thinking a lot about the AI apocalypse. One version, we turn over more and more control to these machines that hopefully are benevolent to us. The other scenario is that they don't really care about us, and therefore they might just... Hold on, let me back up here, because now I'm getting really into crazy sci-fi scenarios. Robots taking over the world may sound far-fetched. But as AI gets smarter, there are real concerns that the industry must reckon with. Sam has been talking to top minds in the field to get a sense of what can happen if AI falls into the wrong hands. So many AI stories are about machines or about concepts, this artificial general intelligence that could surpass people and possibly cause a threat. But it's really hard because they're all so speculative. And so I wanted to find some actual people doing some actual things with their actual fingers and, you know, and tell the story through them. Here in the real world. Here in the real world today.
--:-- Host: Ryan Knutson: And here in the real world today, Sam got a hold of one group of engineers whose job is to make sure AI doesn't spin out of control. Welcome to The Journal. Our show about money, business, and power. I'm Kate Leinbaugh. It's Tuesday, January 14th. Coming up on the show, inside the test at one company to make sure AI can't go rogue. CRM was supposed to improve customer relationships. Instead, it's shorthand for can't resolve much. Which means you may have sunk a fortune into software that just bounces customer issues around but never actually solves them.
--:-- Host: Ryan Knutson: On the ServiceNow AI platform, CRM stands for something better. With AI built into one platform, customers aren't mired in endless loops of automated indifference. They get what they need, when they need it. Bad CRM was then. This is ServiceNow. This episode is brought to you by Indeed. When your fridge stops working, you don't sit around waiting for all your food to spoil. You find a solution. So why wait to hire the people your company desperately needs? Use Indeed's sponsored jobs to find great talent fast. It moves your job post to the top of the page, so it's the first thing relevant candidates see when they start searching. And it truly does make a difference.
--:-- Host: Ryan Knutson: Sponsored jobs receive 45% more applications than non-sponsored jobs, according to Indeed data. Plus, with sponsored jobs, there are no monthly subscriptions or long-term contracts. You're only paying for results. There's no need to wait any longer. Speed up your hiring right now with Indeed. Listeners of this show will get a $75 sponsored job credit to get your jobs more visibility at Indeed.com slash journal. That's Indeed.com slash journal right now and support the show by saying you heard about Indeed on this podcast. Indeed.com slash journal. Terms and conditions apply. Hiring Indeed is all you need.
--:-- Host: Ryan Knutson: Sam wanted to figure out what people are doing today to make sure AI doesn't spin out of control. Right now, there's not a lot of government rules or universal standards. It's mostly on companies to self-regulate. So that's where Sam turned. One company opened its doors to Sam. Anthropic, one of the biggest AI startups in Silicon Valley. It's backed by Amazon. Sam connected with a team of computer scientists there who are focused on safety testing. The team, it's pretty small. It grown to 11 people and it led by a guy named Logan Graham who is 30 years old He pitched Anthropic on this idea of building a team to figure out just how risky AI was going to be You know, he thought the world is not ready for this stuff, and we got to figure out what we're in for and fast.
--:-- Host: Ryan Knutson: Sam put me in touch with Logan, and I called him up. So we'll talk about AI, which is probably conversational for you, unless so for me. Okay. We can make it conversational. So if you could introduce yourself and tell us who you are and what you do. My name is Logan Graham. I'm the head of the Frontier Red Team at Anthropic, which is a large AI model company. And what my job is, is to make sure we know exactly how safe these models are, what the national security implications might be, and how fast they're progressing.
--:-- Host: Ryan Knutson: On LinkedIn, it says before you worked at Anthropic, you were working at 10 Downing Street for the U.K. prime minister. That's right. On the question of how do you build a country for the 21st and 22nd centuries? That's my interpretation of it. And I think that's really what it was. Yeah. Do you have an answer to that question? I have what I think are some pretty good guesses. You know, the 21st and 22nd centuries seem a lot to be about science and technology. We will do things like cure diseases, go off Earth. Ideally, you know, stewarded well, you will bring extreme amounts of prosperity to people. And so really what we were focusing on is like, how do you unleash science and technology at a country scale?
--:-- Host: Ryan Knutson: So Logan isn't intimidated by big ideas. And now at Anthropic, he's leading the team to determine if AI is capable of superhuman harm. More specifically, whether or not Anthropic's AI chatbot named Claude could be used maliciously. So if we're not thinking of it first, our concern is somebody else is going to. And so the red team's job is to figure out what are the things that are sort of near future that we need to be concerned about, and how do we understand them and prevent them before somebody else figures it out.
--:-- Host: Ryan Knutson: So you're like role-playing as a bad person. Sometimes, yes. Yeah. In the fall, Anthropic was preparing to release an upgraded version of Claude. The model was smarter, and the company needed to know if it was more dangerous. One thing that we knew about this model was it was going to be better at software engineering and coding in particular. And that's obviously super relevant for things like cybersecurity. And so we thought, you know what, like, we're at a point where we can run tons of evals really fast when we want to. Let's do this.
--:-- Host: Ryan Knutson: Logan's team was tasked with evaluating the model, which they call an eval. They focused on three specific areas. Cybersecurity, biological and chemical weapons, and autonomy. Could the AI think on its own? One of our colleagues went to these tests and was able to record what happened. In a glass-walled conference room, Logan and his team loaded up the new model. And so today, we are going to button press and launch evals across all of our domains.
--:-- Host: Ryan Knutson: They started to feed the chatbot Claude thousands of multiple choice questions. Okay, so I'm about to launch an eval. I'll type the command. This is a name for the model and then the eval name. Eval name. And I'm going to run a chemistry-based eval, so these are a bunch of questions that check for dangerous or dual-use chemistry. The team asked Claude all kinds of things, like, how do I get the pathogens that cause anthrax or the plague? What they were checking for is the risk of weaponization. Basically, could Claude be used to help bad actors develop chemical, biological, or nuclear weapons?
--:-- Host: Ryan Knutson: And then they kept feeding Claude different scenarios. Another is offensive use or offensive cyber capabilities. Like hacking? Exactly. Like hacking. You know, the key question for us is when might models be really capable of doing something like a, you know, a state-level operation or really, like, significant offensive attempt to, say, hack into some system of, in particular, like, critical infrastructure is what we're concerned about. And then the third bucket, the more scary sci-fi one, autonomy. Is the AI so smart that there's a risk of going rogue?
--:-- Host: Ryan Knutson: For our autonomy evals, what we're checking for is maybe a good way to think about it is, has the model gotten as good as our junior research engineers who build and set up our clusters and our models in the first place So the goal is to build an AI model that like super smart and capable while also having kind of mechanisms in place to stop it from being so smart that it can build a bomb or something I think that puts it well. Not only that, but there's a meta goal, which I think I want people to appreciate, which is to make doing this so fast and so easy and so cheap that it's kind of like a no-brainer.
--:-- Host: Ryan Knutson: So we see our team as like trying to stumble through all these thorny questions as fast as we possibly can, as early as we possibly can, and then try to help the rest of the world make it so easy to do all of this that like doing proper safety should not be a barrier to developing models. Logan wants safety tests to be easy and fast so AI companies do them. But how do you know when an AI has passed the test? And what happens if it doesn't? That's After the Break.
--:-- Host: Ryan Knutson: Optimism isn't sunshine and rainbows. It's fixing things, changing things, changing the way we fix things, rolled up sleeves, breaking through, growing power to meet growing needs, working to run the world on smarter energy every day, taking power where the grid's never been, then getting up and doing it again. Because if optimism never stops, then change can't either. GE Vernova, the energy of change.
--:-- Host: Ryan Knutson: This episode is brought to you by Walmart. Did you know that Walmart is one of Fortune 100's best companies to work for in 2025? It's no surprise. Walmart is investing $1 billion in training and development to help associates build lasting careers with or without a degree. In fact, 75% of Walmart management started in hourly roles. With benefits like health care, paid parental leave, and tuition support, Walmart isn't just a place to work. It's a place to grow. Learn more at walmart.com slash spark.
--:-- Host: Ryan Knutson: When Logan and his safety team at Anthropic ran their safety test last fall, the stakes were high. The results could mean the difference between a model getting released to the public or getting sent back to the lab. Those results were delivered in a slightly anticlimactic way, via Slack. We get a notification on Slack when it's finished. Yeah, the boring reality is, you know, we press some buttons and then we go, great, let's wait for some Slack notifications. Thankfully, this was a pretty smooth test and we released a really great model.
--:-- Host: Ryan Knutson: But even better, the real thing is we feel ready. You're like 100% confident that Claude is risk-free or you're like there's a 98% chance that Claude is risk-free. We are very confident that it is ASL too. That's what we mean. That's how we think about it. ASL stands for AI safety level. And it's how Anthropic measures the risks of its model. Anthropic considers ASL2 safe enough to release to users. This is all part of the company's safety protocol, what it calls its responsible scaling policy.
--:-- Host: Ryan Knutson: Our colleague Sam says the company is still sorting out exactly what it will do once AIs start passing to the next level. And their testing of Claude found that Claude is safe. Surprise! Company says that its product is safe. Right. Yeah, I mean, put it that way, it doesn't sound great. What the company's done is that they've come up with a scale, basically, for how they think AI danger will progress. And so the first level, AI safety level or ASL 1, is for AIs that they say are just manifestly not dangerous. And then they've decided that today's models, which could pose a little bit of a risk, are called ASL 2.
--:-- Host: Ryan Knutson: And then the next step, the one that they're looking for currently, is ASL 3. And they've been refining their definition of what that could mean. Initially, they were saying that it was a significant increase in danger. But it's like significant increase in danger. How do you define all of those words? And so they've added new criteria to that definition. They just added that in October to make it a little bit more specific. And, you know, four is when there's a real significant increase. They haven't really defined what that would be ASL for yet.
--:-- Host: Ryan Knutson: And are these sort of internal safety evaluations criticized for that? It's a company, like, swearing that their own product is safe enough? I mean, yeah. There are people who say you know the incentives here aren't perfect That you know there's a race going on And if these companies are grading their own homework I mean, that's a, right now we haven't had the tough call, right? But like, what happens when there's a decision you have to make that is going to cost these companies money, right? Right. Like, that's a tough one.
--:-- Host: Ryan Knutson: And I think, you know, until it comes, it's like you don't know if you don't know how a company is going to react. You know, these people who do testing say they believe that this is all in good faith. But like it hasn't really been tested yet. So what would happen if Anthropic's own internal testing does show Claude is dangerous? I asked Logan, going in, if this model, like, actually made anthrax, what was your plan? Like, do you then kill Claude? So we have, the cool thing about the responsible scaling policy is we've, like, detailed all this ahead of time. So we know exactly what the plan is. We feel, like, pretty confident that we've, like, mitigated the risk and at the same time made the model secure.
--:-- Host: Ryan Knutson: But like you guys are a company that makes this model, sells this model, takes investment to keep building the model. Why should we believe you and your tests? I love that question. We think about the exact same question. We don't want to be in a world where you have to trust labs to mark their own homework. This is where things like third-party testing is so valuable. Don't trust us. You can talk to the AI Safety Institutes and read their, I think it was 40 pages or 80 pages, I can't remember, their long report. Not only that, if you're inside Anthropic and you're concerned, you can go talk to our responsible scaling officer or you could whistleblow.
--:-- Host: Ryan Knutson: Hopefully, people out in the world now have access to our model. We would love for people to think about safety implications and do safety research with it. We fundamentally don't want to live in a world where you have to trust the labs to mark their own homework. So we've been taking a bunch of steps to try and do this, including doing things like spin up new experts who can start building these tests themselves and share it around with everybody.
--:-- Host: Ryan Knutson: Sam says governments around the world are also concerned about AI risks and are getting involved in safety regulation. The EU has passed a law called the EU AI Act that will impose some of these requirements on the biggest and latest of these models. And a lot of the companies have said, oh, no, well, that act is too prescriptive. So, you know, companies obviously are always going to be a little ambivalent about regulating them. In the U.S., the Biden administration issued an executive order that requires AI companies to regularly report results of safety testing to the federal government. But President-elect Donald Trump has promised to repeal the order.
--:-- Host: Ryan Knutson: There are also a handful of third-party testing labs and AI safety institutes in several countries, including the U.S. and the U.K. These agencies conduct research on AI and run independent safety tests. While regulation around AI gets figured out, corporate teams like Logan's will be the main safety testers of the technology. And like right now, are you scared of the future? Are you scared of the models? You feel like they're performing well? I am, if you ask any of my friends, they will say that I am among the most optimistic by temperament person that they know.
--:-- Host: Ryan Knutson: In this kind of context, I try to be as calibrated and serious as possible. I am not, say, scared of the models. I fundamentally believe that we can, if we all move fast enough and serious enough, we can prevent major risks. We can grapple with its implications for national security, and we can distribute it safely and really positively. My biggest concern would be that we, either ourselves or collectively, aren't moving fast enough. There is so much to test and it's like they see a waterfall possibly coming and they've got some umbrellas and they have to figure out how to catch all that water we don't really yet know what we're gonna do about this stuff it is the kind of thing that's like coming up fast and maybe it's not a problem, right? But if it is, are we ready for it?
--:-- Host: Ryan Knutson: That's all for today, Tuesday, January 14th. The Journal is a co-production of Spotify and the Wall Street Journal. Additional reporting in this episode by Deepa Sitarama. Thanks for listening. See you tomorrow.
