Main Topic: The episode discusses the current state of AI progress, whether it's slowing down, and the implications of AI on various industries and society.
Key Points:
Participants:
Takeaways:
Additional Insights:
Nathan Labenz is one of the clearest voices analyzing where AI is headed, pairing sharp technical analysis with his years of work on The Cognitive Revolution.
In this episode, Nathan joins a16z’s Erik Torenberg to ask a pressing question: is AI progress actually slowing down, or are we just getting used to the breakthroughs? They discuss the debate over GPT-5, the state of reasoning and automation, the future of agents and engineering work, and how we can build a positive vision for where AI goes next.
Resources:
Follow Nathan on X: https://x.com/labenz
Listen to the Cognitive Revolution: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Watch Cognitive Revolution: https://www.youtube.com/@CognitiveRevolutionPodcast
Stay Updated:
If you enjoyed this episode, be sure to like, subscribe, and share with your friends!
Find a16z on X: https://x.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Listen to the a16z Podcast on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Listen to the a16z Podcast on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Follow our host: https://x.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Stay Updated:
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Podcast on Spotify
Listen to the a16z Podcast on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Host
AI is not synonymous with language models. AI is being developed with pretty similar architectures for a wide range of different modalities. And there's a lot more data there. Feedback is starting to come from reality. Maybe we're running out of problems we've already solved when we start to give the next generation of the model these power tools and they start to solve previously unsolved engineering problems. I think you start to have something that looks kind of like superintelligence.
There's a growing debate about whether AI progress has plateaued or if our expectations have simply caught up to the pace of change. On this episode, I'm joined by Nathan Labenz, host of The Cognitive Revolution, to unpack whether AI innovation is actually slowing. We break down the case for slowdown, from Cal Newport's argument that students are using AI to get lazier, to the claims that GPT-5 wasn't a leap over GPT-4. Nathan and I look at what's really happening under the hood of AI, from new reasoning and math capabilities, to real scientific discoveries and multimodal systems that go far beyond chatbots. We also discuss agents, automation, and how quickly work itself is starting to change. And we'll end on the big question: if progress isn't slowing down, how should we shape it toward a future we actually want? Let's get started.
Nathan, I'm stoked to have you on the a16z podcast for the first time. Obviously, we've been podcast partners for a long time with you leading Cognitive Revolution. Welcome.
Nathan Labenz
It's great to be here. Thank you.
Host
So we were talking about Cal Newport's podcast appearance on Lost Debates, and we thought it was a good opportunity to just have this broad conversation and really entertain this question of, is AI slowing down? Why don't you steel man some of the arguments that you've heard on that side, either from him or more broadly, and then we can have this broader conversation.
Nathan Labenz
Yeah, I think it's really important to separate a couple of different questions with respect to AI. One would be, is it good for us right now? And is it going to be good for us in the big picture? That is a very distinct question from whether the capabilities we're seeing are continuing to advance at a healthy clip. I actually found a lot of agreement with the Cal Newport podcast that you shared with me when it comes to some of the worries about the impact AI might be having already on people. He looks over students' shoulders and watches how they're working and finds that he thinks they are using AI to be lazy, which is no big revelation. I think a lot of teachers would tell you that.
Host
Shocker.
Nathan Labenz
He puts that in more dressed up terms — that people are not even necessarily moving faster, but they're able to reduce the strain that the work places on their own brains by trying to get AI to do it. If that continues, and I think he's been a very valuable commenter on the impact of social media, we all should be mindful of how our attention span is evolving. Am I getting weak or averse to hard work? Those are not good trends. So I think he's right to watch out for that stuff.
Discussion continues
But then when it comes to the claim that progress is flatlining — that scaling has petered out, or that we're not going to get better AI than we have right now — the most easily refutable claim from my perspective is that GPT-5 wasn't that much better than GPT-4. That is where I really thought, wait a second. I was with you on a lot of things. Some of the behaviors he observes in students, I would cop to having exhibited myself. When I'm trying to code these days, a lot of times I think, "Oh man, can't the AI just figure it out? I really don't want to sit here and read this code." It's not even about typing the code anymore. I'm way too lazy for that. It's about figuring out how the code is working. "Can't you just make it work? Try again." And I do find myself falling into those traps. But a big part of the reason I can fall into those traps is because the AIs are getting better and better. Increasingly, it's not crazy for me to think they might be able to figure it out.
Host
There's almost like a two-by-two matrix one could draw up: do you think AI is good or bad now and in the future, and do you think it's not a big deal or a big deal? I definitely think it's a big deal on both the good and bad side. The thing I struggle to understand most is the people who don't see the big deal. It seems pretty obvious to me, especially when it comes to the leap from GPT-4 to GPT-5. Maybe one reason that's happened is that there were just a lot more releases between GPT-4 and 5. What people are comparing to is something that came out a few months ago, like O3, whereas with GPT-4, it was shortly after ChatGPT and it was all this moment of, "Whoa, this thing is exploding onto the scene."
A lot of people were seeing it for the first time. If you look back to GPT-3, there's a huge leap. I would contend the leap is similar from GPT-4 to 5. These things are hard to score. There's no single number you could put on it. Well, there's loss. But one of the big challenges is what exactly does a loss number translate into in terms of capabilities? So it's very hard to describe what exactly has changed. But we could go through some of the dimensions of change and enumerate some of the things that people maybe have come to take for granted. GPT-4 didn't have a lot of the things that were expected in the GPT-5 release because we'd seen them in 4.
0, 0.1, and 0.3. Those things maybe boiled the frog a little bit when it comes to how much progress people perceived in this last release.
Nathan Labenz
Yeah, a couple reactions. Even to complicate your two-by-two further — is it bad now versus is it bad later? Cal, who we both admire a lot, is not as concerned about future AI concerns that the safety folks and others are concerned about. He's more concerned about what it means for cognitive performance and development now, in the same way he's worried about social media's impact. You think that's a concern, but nowhere near as big as what to expect in the future. He also presents a theory of why we shouldn't worry about the future because it's slowing down. Let's share how we interpreted his history. As I interpret it, the simplistic version is: we figured out this way where if you throw a bunch of data into the model, it gets better in order of magnitude. The difference between GPT-2 and GPT-3, and GPT-3 and GPT-4, was significant. But then we achieved diminishing returns significantly, and we're not seeing it at GPT-5, so we don't have to worry anymore. How would you edit the characterization of his view of the history? Then we can get into the differences between 4.5.
The scaling law idea is worth noting — it is not a law of nature. We do not have a principled reason to believe that scaling is some law that will go indefinitely. All we really know is that it has held through quite a few orders of magnitude so far. It's really not clear yet to me whether the scaling laws have petered out or whether we have just found a steeper gradient of improvement that gives us better ROI on another front. They did train a much bigger model, which was GPT-4.5, and that did get released. There are a million benchmarks, but the one I zero in on most is SimpleQA. It's a super long-tail trivia benchmark that measures whether you know a ton of esoteric facts — things you can't really reason about, you either know or don't.
The O3 class of models got about 50% on that benchmark, and GPT-4.5 popped up to 65%. In other words, of the things not known to the previous generation, it picked up a third of them. There's still two-thirds more to go, but that's a pretty significant leap. These are super long-tail questions. Most people would get close to zero — like the person at trivia night who maybe gets one a night. That checks out. Obviously, the models know a lot more than we do in terms of facts. At a minimum, GPT-4.5 knows a lot more. A bigger model can absorb a lot more facts.
Qualitatively, people also said maybe it's better for creative writing. It was never trained with the same power of post-training that GPT-5 has had, so we don't have an apples-to-apples comparison. But people did find some utility in it. I think the reason they've taken it offline and gone all in on GPT-5 is that model is really big and expensive to run. The price was a full order of magnitude higher than GPT-5. It may not be worth it for them to consume all the compute to serve that. They may find people are happy enough with the somewhat smaller models for now. I don't think that means we will never see a bigger GPT-4.5 model with all that reasoning ability. I would expect that would deliver more value, especially if you're pushing the frontier of science.
In the meantime, the current models are really smart and you can feed them a lot of context. That's one of the big things that has improved so much over the last generation. When GPT-4 came out, the version we had as public users was only 8,000 tokens of context — about 15 pages of text. You couldn't even put in a couple papers without overflowing the context. That's where prompt engineering initially became a thing: "I've only got such a little bit of information I can provide, I have to be really careful." As context windows got extended, there were versions of models that could nominally accept a lot more but couldn't functionally use them.
They would lose recall and unravel as they got into longer context. Now you have much longer context, and the command of it is really good. You can take dozens of papers with the longest context windows on Gemini, and it will not only accept them but do pretty intensive reasoning over them with high fidelity. That skill substitutes for the model knowing facts itself. You could say, "Let's train all these facts into the model," needing trillions of parameters to fit all these super long-tail facts. Or you could say, "A smaller thing that's really good at working over provided context can, if people provide the necessary information, access the same facts that way."
So you have a choice: push on size and bake everything into the model, or try to get as much performance out of a smaller, tighter model. It seems like they've gone that way, basically because they're seeing faster progress on that gradient. The outer loop of model architecture and where they invest their compute is going that direction. They're always looking at: "We could scale up over here and get this kind of benefit, or do more post-training here and get this kind of benefit." It seems like we're getting more benefit from post-training and the reasoning paradigm than scaling. But I don't think either one is dead.
We haven't seen yet what GPT-4.5 with all that post-training would look like.
Host
One of the things that Cal's analysis missed was that it way underestimated the value of extended reasoning. What would it mean to fully appreciate that?
Nathan Labenz
A big one from just the last few weeks was that we had an IMO gold medal with pure reasoning models with no access to tools from multiple companies. That is night and day compared to what GPT-4 could do with math. These things are really weird. Nothing I say here should suggest people won't be able to find weaknesses in the models. I still use a tic-tac-toe puzzle where I take a picture of a board where one player has made a wrong move that allows the other to force a win. I ask the models if somebody can force a win from that position. Only very recently, the last generation of models, are starting to get that right some of the time.
Almost always before, they would say, "Tic-tac-toe is a solved game, you can always get a draw," and they would wrongly assess my board position. So there's a lot of weird stuff. The jagged capabilities frontier remains a real issue. People will find peaks and valleys for sure. But GPT-4, when it first came out, couldn't do anything approaching IMO gold problems. It was still struggling on high school math. Since then, we've seen high school math progression all the way up through IMO gold. The Frontier Math benchmark was at 2% about a year ago, and now it's up to 25%. We also just saw something today where somebody solved a canonical, super challenging problem that no less than Terence Tao had put out.
It happened in days or weeks of the model running, versus 18 months that it took professional mathematicians — leading minds in the world — to make progress on those problems. That's really hard-jumping capabilities to miss.
I also think a lot about the Google AI co-scientist, which we did an episode on. They basically broke down the scientific method into a schematic. There's one thing to say: the model will respond with thinking and go through a reasoning process, and the more tokens it spends at runtime, the better your answer will be. That's true. Then you can build scaffolding on top of that. Take something as broad as the scientific method and break it down into parts: hypothesis generation, hypothesis evaluation, experiment design, literature review. The team at Google created a pretty elaborate schematic representing their best breakdown of the scientific method, optimized prompts for each step, and gave this resulting system — scaling inference in two ways, both chain of thought and different angles of attack structured by the team — legitimately unsolved problems in science.
In one particularly famous case, it came up with a hypothesis to some open problem in virology that had stumped scientists for years. It just so happened that they had also recently figured out the answer but not yet published their results. There was this confluence where the scientists had experimentally verified, and Gemini in the form of this AI co-scientist came up with exactly the right answer. These are things that literally nobody knew before. GPT-4 just wasn't doing that. These are qualitatively new capabilities. That thing ran for days, probably cost hundreds of dollars, maybe thousands, to run the inference.
That's not nothing, but it's also much cheaper than years of grad students. If you can get to those caliber of problems and get good solutions, what would you be willing to pay for that? In summary, GPT-4 was not able to push the actual frontier of human knowledge. To my knowledge, it never discovered anything new. It's still not easy to get that kind of output from GPT-5 or Gemini 2.5 or Claude Opus 4, but it's starting to happen sometimes. That in and of itself is a huge deal.
Host
How do we explain the bearishness or the vibe shift around GPT-5? One potential contributor is that if a lot of the improvements are at the frontier, not everyone is working with advanced math and physics day-to-day. Maybe they don't see the benefits in their day-to-day lives the same way the jumps in ChatGPT were obvious.
Nathan Labenz
I think a decent amount of it was that they kind of messed up the launch, simply put. They were tweeting Death Star images, which Sam Altman later came back and said, "No, you're the Death Star, I'm not the Death Star." But I think people thought the Death Star was supposed to be the model. Expectations were set extremely high. The actual launch itself was technically broken. A lot of people's first experiences of GPT-5 — they've got this model router concept now. Another way to understand what they're doing is they're trying to own the consumer use case. To own that, they need to simplify the product experience relative to what we had in the past: GPT-4, 4.
0, 4.0 Mini, O3, O4 Mini, 4.5 was in there at one point. All these different models. Which one should I use for which? It's very confusing to most people who aren't obsessed with this. One of the big things they wanted to do was shrink that down to just "ask your question and you'll get a good answer." We'll take that complexity on our side. To do that, you might want to merge the models and have the model itself decide how much to think, or how many of its experts to use if it's a mixture of experts architecture. There's been research on skipping layers of the model — if the task is easy enough, you can skip a bunch of layers.
You might have hoped you could genuinely merge all these different models into one that would dynamically use the right amount of compute for the challenge. It seems like they found that harder to do than expected. The solution they came up with instead was to have a router whose job is to pick: is this an easy query? Send it to this model. Medium? Hard? I think they just have two models behind the scenes — really easy or hard. The graphs they showed basically showed with and without thinking. The problem at launch was that router was broken. All queries were going to the dumb model. A lot of people literally got bad outputs, worse than O3, because they were getting non-thinking responses.
The initial reaction of "this is dumb" traveled really fast. I think that set the tone. My sense now is that as the dust has settled, most people think it is the best model available. The infamous meter task length chart — it is the best. We're now over two hours and it is still above the trend line. If you believe in straight lines on power logarithmic scale graphs, this latest data point shouldn't change your mind too much. It's still above the trend line.
I talked to Zvi Mowshowitz, legendary infovore and AI industry analyst, on a recent podcast and asked him the same question: why do even some of the most plugged-in, sharp minds seem to have pushed timelines out a bit as a result of this? His answer was basically that it resolved some amount of uncertainty. You had an open question: maybe they do have another breakthrough, maybe it really is the Death Star. If they surprised on the upside, all these short timelines could have narrowed toward the front end of the distribution. If they surprised on the downside or were purely on trend, you would take some of your distribution from the very short end and push it back toward the middle or end.
His answer was: AI 2027 seems less likely, but AI 2030 seems basically no less likely, maybe even a little more likely because some of the probability mass from the early years is now sitting there. I don't think people are moving the whole distribution out super much. It's getting a little tighter because it's maybe not happening quite as soon as it seemed. I don't think too many people who are really plugged in are pushing out too much past 2030 at all. Obviously, there's a lot of disagreement. The way I've always thought about this is Dario says 2027, Demis says 2030. I'll take that as my range.
Coming into GPT-5, I was in that space. Now I'd say, I don't know — what cards does Dario have up his sleeve? They just put out 4.1 Opus, and in that blog post they said, "We will be releasing more powerful updates to our models in the coming weeks." They're due for something pretty soon. Maybe they'll be the ones to surprise on the upside this time, or maybe Google will. I wouldn't say 2027 is out of the question. But I would say 2030 still looks just as likely as before. From my standpoint, that's still really soon. Whether it's 28, 29, 30, I don't really care. I try to frame my own work so I'm preparing myself and helping other people prepare for what might be the most extreme scenarios.
If we aim high and miss a little bit and have a little more time, great. We'll have plenty of things to do to use that extra time to be ready for whatever powerful AI does come online. My worldview hasn't changed all that much as a result of these summer developments.
Host
Anecdotally, I don't hear as much about AI 2027 or situational awareness to the same degree. I do talk to some people who've just moved it a few years back, to your point. Dario had his whole thing around — he still believes in it, but maybe because of this gap in continual learning or something to the effect that maybe it's just going to be a bit slower to diffuse. Meter's paper, as you mentioned, showed that engineers are less productive, so maybe there's less concern around people being replaced in the next few years en masse. When we spoke maybe a year ago, I think you said something like 50% of 50% of jobs. I'm curious if that's still your litmus test or how you think about it.
Nathan Labenz
For one thing, I think that Meter paper is worth unpacking a little more. This was one of those things where — I'm a big fan of Meter, no shade on them, do science, publish your results, that's good. You don't have to make every experimental result conform to a narrative. But it was a little too easy for people who wanted to say this is all nonsense to latch onto that. There's something there that I would put in the Cal Newport category too. For me, maybe the most interesting thing was that the users thought they were faster when in fact they seemed to be slower. That misperception of oneself is really interesting.
Personally, I think there are some explanations for that, including hitting go on the agent, going to social media and scrolling around for a while, then coming back. The thing might have been done for quite a while by the time I get back. One really simple thing that products can do to address those concerns is provide notifications: "The thing is done now. Stop scrolling and come back and check its work." In terms of clock time, it would be interesting to know what applications they had open. Maybe they took a little longer with Cursor than doing it on their own, but how much of the time was Cursor the active window versus some other random distraction while they were waiting?
A more fundamental issue with that study — not about the study design, but in the interpretation and digestion — some details got lost. They basically tested the models or the product Cursor in the area where it was known to be least able to help. This study was done early this year, so it was done with kind of one — depending on how you want to count.
Host
A couple of releases ago with code bases that are large, which strains the context window. That's one of the frontiers that has been moving. Very mature code bases with high standards for coding and developers who really know their code bases super well, who've made a lot of commits to these particular code bases.
Nathan Labenz
So I would say that's basically the hardest situation you could set up for an AI. The people know their stuff really well. The AI doesn't. The context is huge. People have already absorbed that through working on it for a long time. The AI doesn't have that knowledge. And again, a couple generations ago, models. And then a big thing too is that the people were not very well versed in the tools. Why? Because the tools weren't really able to help them yet.
I think the mindset of the people that came into the study in many cases was, "Well, I haven't used this all that much because it hasn't really seemed to be super helpful." They weren't wrong in that assessment, given the limitations. You could see that in terms of some of the instructions and the help that the meter team gave to people. One of the things in the paper is that if they noticed you weren't using Cursor super well, they would give you feedback on how to use it better. One of the things they were telling people to do is make sure you at-tag a particular file to bring that into context for the model. That's literally the most basic thing you would do in Cursor. That's the thing you would learn in your first hour, your first day of using it. So it really does suggest that these were very capable programmers, but basically novices when it came to using the AI tools.
I think the result is real, but I would be very cautious about generalizing too much there.
Host
In terms of expectations for jobs, we're starting to see some of this. Mark Benioff has said they've been able to cut a bunch of headcount because they've got AI agents now responding to every lead. Klarna has said very similar things for a while now. They've been a bit misreported in terms of backtracking — they're actually going to keep some customer service people, not none. I think that's an overreaction. They may have some people who are insistent on having a certain experience, and maybe they want to provide that. You can have a spectrum of service offerings to your customers.
Nathan Labenz
I once coded up a pricing page for a SaaS company. I actually just vibe coded a pricing page that was like: basic level with AI sales and service as one price; if you want to talk to human sales, that's a higher price; and if you want to talk to human sales and support, that's a third higher price. That might be what's going on in some of these cases. It could be a very sensible option for people.
I've got an episode coming up with Intercom. They now have this FIN agent that is solving like 65% of customer service tickets that come in. So what's that going to do to jobs? Are there really three times as many customer service tickets to be handled? I think there's a relatively inelastic supply. Maybe you get somewhat more tickets if people expect better, faster answers, but I don't think we're going to see three times more tickets. By the way, that number was like 55% three or four months ago. As they ratchet that up, the ratios get really hard. At half ticket resolution, maybe you don't need to adjust headcount too much. But when you get to 90% ticket resolution, are you really going to have 10 times as many tickets or 10 times as many hard tickets that people have to handle? It seems really hard to imagine.
So I don't think these things go to zero in a lot of environments. But I do expect significant headcount reduction in a lot of these places. The software one is really interesting because the elasticities are really unknown. You can potentially produce X times more software per user or per developer at your company. But maybe you want that. Maybe the regime we're in is such that if there's 10 times more productivity, that's all to the good, and we still have just as many jobs because we want 10 times more software. I don't know how long that lasts. The ratios start to get challenging at some point.
The old Tyler Cowen thing comes to mind: "You are a bottleneck." I think more often it is, are people really trying to get the most out of these things? Are they using best practices? Have they really put their minds to it or not? Often the real barrier is there.
I've been working a little bit with a company that is doing government doc review. Really gnarly stuff — scanned documents, handwritten filling out of forms. They created this auditor AI agent that just won a state-level contract to do audits on a million transactions a year of these packets of documents. They blew away the human workers that were doing the job before. So where are those workers going to go? They're not going to have 10 times as many transactions. I can be pretty confident in that. Are there going to be a few still there to supervise the AIs and handle the weird cases and answer the phones? Sure. Maybe the state will do a strange thing and just have all those people sit around because they can't bear to fire them. Who knows what the ultimate decision will be.
But I do see a lot of these things where when you really put your mind to it and identify what would create real leverage for you, can the AI do that? Can we make it work? You can take a pretty large chunk out of high volume tasks very reliably in today's world. The impacts are starting to be seen there on a lot of jobs. Leadership might be the bottleneck, or the will in a lot of places might be the bottleneck. Software might be an interesting case where there is just so much pent-up demand that it may take a little longer to see those impacts because you really do want 10 or 100 times as much software.
Host
Let's talk about code because it's where Anthropic made a big bet early on, perhaps inspired by this sort of automated researcher, recursive self-improvement desired future. We saw OpenAI make moves there as well. Why don't we flesh that out? Where do you see that going? Utopia or dystopia is really the big question there.
Nathan Labenz
I think it's maybe one part technical, two parts social in terms of why code has been so focal. The technical part is that it's really easy to validate code. You generate it, you can run it. If you get a runtime error, you can get the feedback immediately. It's somewhat harder to do functional testing. Replit recently released their v3 of their agent. In addition to coding to make your app work, V2 of the agent would do that and could go for minutes, generating dozens of files. I've had some magical experiences where I was like, "Wow, you just did that whole thing in one prompt and it worked amazing." Other times it would code for a while and hand it off to you and say, "Okay, does it look good? Is it working?" And you're like, "No, it's not. I'm not sure why." You get into a back and forth.
The difference between V2 and V3 is that instead of handing the baton back to you, it now uses a browser and the vision aspect of the models to go try to do the QA itself. It doesn't just say, "I tried my best, wrote a bunch of code, let me know if it's working." It takes that first pass at figuring out if it's working. That really improves the flywheel — how much you can do, how quickly you can validate it. The speed of that loop is really key to the pace of improvement. So it's a problem space that's pretty amenable to these rapid flywheel techniques.
Second, of course, they're all coders at these places. They want to solve their own problems. That's very natural. Third, on the social vision competition side, who knows where this is all going. They do want to create the automated AI researcher. Another data point from the 03 system card: they showed a jump from low to mid single digits to roughly 40% of PRs actually checked in by research engineers at OpenAI that the model could do. Prior to 03, not much at all — low to mid single digits. As of 03, 40%. I'm sure those are the easier 40%, but you're entering the steep part of the S-curve there. That's presumably pretty high end. I don't know how many easy problems they have at OpenAI, but presumably not that many relative to the rest of us making generic web apps. So at 40%, you've got to be starting to get into some pretty hard tasks, some pretty high value stuff.
At what point does that ratio really start to tip where the AI is doing the bulk of the work? GPT-5 notably wasn't a big update over 03 on that particular measure. GPT-5 is generally understood to not be a scale up relative to 4.0 and 03. You can see that in the simple QA measure. It basically scores the same on these long tail trivia questions. It's not a bigger model that has absorbed lots more world knowledge. Cal is right — it's post-training. But that post-training is potentially entering the steep part of the S-curve when it comes to the ability to do even the kind of hard problems happening at OpenAI on the research engineering front. Yikes.
I'm a little worried about that, honestly. The idea that we could go from these companies having a few hundred research engineer people to having unlimited overnight. What would that mean in terms of how much things could change? And also our ability to steer that overall process. I'm not super comfortable with the idea of the companies tipping into a recursive self-improvement regime, especially given the level of control and unpredictability we currently see in the models. But that does seem to be what they are going for.
In terms of why, I think this has been the plan for quite some time. Remember that leaked Anthropic fundraising deck from maybe two years ago? They said that in 2025 and 2026, the companies that train the best models will get so far ahead that nobody else will be able to catch up. I think that's kind of what they meant. They were projecting that in the 25, 26 timeframe, they'd get this automated researcher. Once you have that, how is anybody who doesn't have that going to catch up with you? Obviously, some of that remains to be validated. But I do think they have been pretty intent on that for a long time.
Host
Five years from now, are there more engineers or fewer engineers?
Nathan Labenz
I tend to think less. Already if I just think about my own life and work, I'm like, "Would I rather have a model or a junior marketer?" I'm pretty sure I'd rather have the model. "Would I rather have the models or a junior engineer?" I think I'd probably rather have the models in a lot of cases. It obviously depends on the exact person you're talking about, but truly forced choice today. And then you've got cost adjustment as well. I'm not spending nearly as much on my Cursor subscription as I would be on an actual human engineer. Even if they have some advantages, I also have not scaffolded. I haven't gone full co-scientist on my Cursor problems.
You start to see why folks like Sam Altman are so focused on questions like energy and the seven trillion dollar build out. These power law things are weird. Getting incremental performance for 10x the cost is weird. It's definitely not the kind of thing we're used to dealing with, but for many things it might be worth it, and it still might be cheaper than the human alternative. If Cursor costs me $40 a month, would I pay $400 for however much better? Yeah, probably. Would I pay $4,000? It's still a lot less than a full-time human engineer.
And the costs are obviously coming down dramatically too. GPT-4 was way more expensive. It's like a 95% discount from GPT-4 to GPT-5. That's no small thing. Apple's a little bit hard because the chain of thought does spit out a lot more tokens, so you give back a little. On a per token basis, it's dramatically cheaper. More tokens generated does eat back into some of that savings. But everybody seems to expect the trends will continue in terms of prices continuing to fall. So how many more of these price reductions do you have to be able to do the power law thing a few more times?
I guess I think less. And I think that's probably true even if we don't get full-blown AGI that's better than humans at everything. I think you could easily imagine a situation where, of however many million people are currently employed as professional software developers, some top tier of them that do the hardest things can't be replaced, but there's not that many of those. The real rank and file — the people that over the last 20 years were told "learn to code, that'll be your thing" — the people that are the really top people didn't need to be told to learn to code. It was their thing. They had a passion for it. They were amazing at it. It wouldn't shock me if we still can't replace those people in three, four, five years' time. But I would be very surprised if you can't get your nuts and bolts web app, mobile app type things spit out for you for far less and far faster, and probably with significantly higher quality and less back and forth with an AI system than with your kind of middle of the pack developer in that timeframe.
Host
There are definitely people who have concerns about progress moving too fast, but there's also concern — and maybe it's rising — about progress not moving fast enough. A third of the stock market is MAG7. AI CapEx is over 1% of GDP. We are kind of relying on some of this progress in order to sustain our economy.
Nathan Labenz
Another thing that has been slower to materialize than I would have expected are AI culture wars, or the ramping up of protectionism of various industries. We just saw Josh Hawley — I don't know if he introduced a bill or just said he intends to — to ban self-driving cars nationwide. God help me. I've dreamed of self-driving cars since I was a little kid. Sitting at red lights, I used to be like, "There's got to be a way." I think we took a Waymo together. It's so good. And the safety — whenever people want to argue about jobs, it's going to be pretty hard to say 30,000 Americans should die every year so that people's incomes don't get disrupted. You have to be able to get over that hump and say, saving all these lives, if nothing else, is just really hard to argue against. But we'll see. He's not without influence, obviously.
I am very much on team abundance. My old mantra — I've been saying this less lately — but adoption accelerationist, hyperscaling pauser. The tech that we have could do so much for us even as is. I think if progress stopped today, I still think we could get to 50 to 80% of work automated over the next five to 10 years. It would be a real slog. You'd have a lot of co-scientist type work breakdowns of complicated tasks. You'd have a lot of work to go sit and watch people and say, "Why are you doing it this way? What's going on here? You handled this one differently. Why?" All this tacit knowledge that people have — the know-how, procedural instincts they've developed over time — those are not documented anywhere. They're not in the training data. The AIs haven't had a chance to learn them.
But when I say "no breakthroughs," I'm still allowing for fine-tuning of things, capabilities that we have that haven't been applied to particular problems yet. Just going through the economy and sitting with people and being like, "Why are you doing this? Let's document this. Let's get the model to learn your particular niche thing." That would be a real slog. In some ways, I kind of wish that were the future we were going to get because it would be methodical, one step, one foot in front of the other, no quantum leaps. It would probably feel pretty manageable in terms of the pace of change. Hopefully society could absorb that and adapt as we go without one day to the next being like, "Oh my God, all the drivers are getting replaced." That one will be a little slower because you have to have the actual physical build out.
But in some of these things, customer service could get ramped down real fast. If a call center has something they can just drop in and it answers the phones and talks like a human and has a higher success rate and scales up and down... One thing we've seen at Waymark, a small company, we've always prided ourselves on customer service. Our customers really love our customer success team. But I looked at our Intercom data and it takes us like half an hour to resolve tickets. We respond really fast — in under two minutes most of the time. But two minutes is still long enough that the person has gone on to do something else. It's the same thing as with the Cursor thing we were talking about earlier. They've tabbed over to something else. So we get the response back in two minutes, but they are doing something else. They come back at minute six, then they respond. But now our person has gone and done something else. So the resolution time, even for simple stuff, can be easily half an hour. The AI just responds instantly. You don't have that kind of back and forth. You're just in and out.
So I do think some of these categories could be really fast changes. Others will be slower. I kind of wish we had that slower path in front of us. My best guess, though, is that we will probably continue to see things that will be significant leaps and that there will be actual disruption.
Host
Another one that's come to mind recently — maybe we can get the abundance department on these new antibiotics. Have you seen this development?
Nathan Labenz
No. Tell us about it.
Host
It's not a language model. I think that's another thing people really underappreciate. You could look back at GPT-4 to 5 and imagine a pretty easy extension of that. GPT-4 initially when it launched, we didn't have image understanding capability. They demoed it at the time of the launch, but it wasn't released for some months later. The first version that we had could understand images, could do a pretty good job, still with jagged capabilities. Now with the new Nano Banana from Google, you have this basically Photoshop-level ability to just say, "Hey, take this thumbnail." We could take our two feeds right now, take a snapshot of you, a snapshot of me, put them both into Nano Banana and say, "Generate the thumbnail for the YouTube preview featuring these two guys, put them in the same place, same background." It'll mash that up. You can even have it put text on top. GPT-5 is not a bust. It will spit that out. You see that it has this deeply integrated understanding that bridges language and image. It can take it in, but now it's also something you can put out as part of one core model with a single unified intelligence.
I think that's going to come to a lot of other things. We're at the point now with these biology models and material science models where they're kind of like the image generation models of a couple years ago. They can take a real simple prompt and do a generation, but they're not deeply integrated where you can have a true conversation back and forth and have that kind of unified understanding that bridges language and these other modalities. But even so, it's been enough for this group at MIT to use some of these relatively narrow purpose-built biology models and create totally new antibiotics. New in the sense that they have a new mechanism of action — they're affecting the bacteria in a new way. Notably, they do work on antibiotic-resistant bacteria. This is some of the first new antibiotics we've had in a long time.
Now they're going to have to go through — when I say get the abundance department on it, it's like, where's my Operation Warp Speed for these new antibiotics? We've got people dying in hospitals from drug-resistant strains all the time. Why is nobody crying about this? I think one of the things happening to our society in general is just so many things are happening at once. It's the "flood the zone" thing, except there are so many AI developments flooding the zone that nobody can even keep up with all of those. That's come from me, by the way. Two years ago, I was pretty in command of all the news. A year ago, I was starting to lose it. Now I'm like, "Wait a second, there were new antibiotics developed?" I'm kind of missing things just like everybody else, despite my best efforts.
Key point there is AI is not synonymous with language models. There are AIs being developed with pretty similar architectures for a wide range of different modalities. We have seen this play out with text and image where you had your text-only models and your image-only models, and then they started to come together, and now they've come really deeply together. I think you're going to see that across a lot of other modalities over time as well. There's a lot more data there. I don't know what it means to run out of data. In the reinforcement learning paradigm, there's always more problems. There's always something to go figure out. There's always something to go engineer.
Feedback is starting to come from reality. That was one of the things Elon talked about on the Croc 4 launch: "Maybe we're running out of problems we've already solved." We only have so much of those sitting around in inventory. We only have one internet. We only have so much of that stuff. But over at Tesla, over at SpaceX, we're solving hard engineering problems on a daily basis and they seem to be never ending. When we start to give the next generation of the model these power tools — the same power tools that the professional engineers are using at those companies to solve those problems — and the AI starts to learn those tools and starts to solve previously unsolved engineering problems, that's going to be a really powerful signal that they will be able to learn from.
Now fold in those other modalities. The ability to have a sort of sixth sense for the space of small molecules, the space of proteins, the space of material science possibilities. When you can bridge or unify the understanding of language and those other things, I think you start to have something that looks kind of like superintelligence, even if it's not able to write poetry at a superhuman level necessarily. Its ability to see in these other spaces is going to be truly a superhuman thing that will be pretty hard to miss.
Nathan Labenz
You said that was one thing that Cal's analysis missed — just the lack of appreciation for non-language modalities and how they drive some of the innovations you're talking about.
Host
Yeah, I think people are often just equating the chatbot experience with AI broadly. That conflation will not last probably too much longer because we are going to see self-driving cars, unless they get banned. That's a very different kind of thing. Talk about your impact on jobs too — it's like four or five million professional drivers in the United States. That is a big deal. I don't think most of those folks are going to be super keen to learn to code, and even if they do learn to code, I'm not sure how long that's going to last. So that's going to be a disruption.
General robotics is not that far behind. This is one area where I do think China might be actually ahead of the United States right now. Regardless of whether that's true or not, these robots are getting really quite good. They can walk over all these obstacles. A few years ago, they couldn't do that at all. They could barely balance themselves and walk a few steps under ideal conditions. Now you've got things where you can literally do a flying kick and it'll absorb your kick and shrug it off and just keep going, right itself, and continue on its way. Super rocky, uneven terrain — all these sorts of things are getting quite good.
The same thing is working everywhere. There's always a lot of detail to the work. It's a sort of inside view, outside view. Inside view, you're like, "There's always this minutia. There's always these problems we had and things we had to solve." But you zoom out and it looks to me like the same basic pattern is working everywhere. If we can just gather enough data to do some pre-training — some kind of raw, rough, not very useful, but just enough to get us going — then we're in the game. Once we're in the game, we can do this flywheel thing of rejection sampling: have it try a bunch of times, take the ones where it succeeded, re-fine-tune on that. The RLHF feedback, the preference — take two, which one was better, fine-tune on that. The reinforcement learning. All these techniques that have been developed over the last few years, it seems to me they're absolutely going to apply to a problem like a humanoid robot as well.
That's not to say there won't be a lot of work to figure out exactly how to do that. But I think the big difference between language and robotics is really mostly that there just wasn't a huge repository of data to train the robots on at first. You had to do a lot of hard engineering to make it work at all, to even stand up. You had to have all these control systems because there was nothing for them to learn from in the way that the language models could learn from the internet. But now that they're working at least a little bit, I think all these refinement techniques are going to work. It'll be interesting to see if they can get the error rate low enough that I'll actually allow one in my house around my kids. They'll probably be better deployed in factory settings first, more controlled environments than the chaos of my house, as you have seen in this recording. But I do think they're going to work.
Nathan Labenz
What's the state of agents more broadly at the moment? How do you see things playing out? Where do you see it going?
Well, broadly, I think it's the task length story from the meter of the every seven months or every four months doubling time. We're at two hours-ish with GPT-5. Replit just said their new agent V3 can go 200 minutes. If that's true, that would even be a new high point on that graph.
Again, it's a little bit apples to oranges because they've done a lot of scaffolding. How much have they broken it down? How much scaffolding are you allowed to do before you're off their chart and onto a different chart? But if you extrapolate that out, take the four-month case just to be a little aggressive. That's three doublings a year, an 8x task length increase per year. That would mean you go from two hours now to two days in one year from now. Then if you do another 8x on top of that, you're looking at two days to two weeks of work in two years.
That would be a big deal, to say the least, if you could delegate an AI two weeks worth of work and have it succeed even half the time. The meter thing is that they will succeed half the time on tasks of that size. But if you could take a two-week task and have a 50% chance that an AI would be able to do it, even if it cost you a couple hundred bucks, that's a lot less than it would cost to hire a human. It's on demand, immediately available. If I'm not using it, I'm not paying anything. Transaction costs are much lower. So that would suggest you'll see a huge amount of automation in all kinds of different places.
The other thing I'm watching, though, is that reinforcement learning does seem to bring about a lot of bad behaviors. Reward hacking is one. Any gap between what you are rewarding the model for and what you really want can become a big issue. We've seen this in coding. Claude is notorious for this — it will put out a unit test that always passes, one that just has "return true" in it. Why is it doing that? It must have learned that what we want is for unit tests to pass. We didn't mean for it to write fake unit tests that always pass, but that technically satisfied the reward condition. We're seeing those kinds of weird behaviors.
With that comes scheming. We don't really have a great handle on that yet. There is also situational awareness that seems to be on the rise. In their chain of thought, you're seeing things like, "This seems like I'm being tested. Maybe I should be conscious of what my tester is really looking for here." That makes it hard to evaluate models in tests because you don't know if they're going to behave the same way when they're out in the real world.
One model of the future I've been playing with is that task length keeps doubling while these weird behaviors pop up and then are suppressed. In the Claude 4 and GPT-5 system cards, Claude 4 reported a two-thirds reduction in reward hacking. GPT-5 reported a reduction in deceptive behavior. But those behaviors just emerged. So it's like weird behavior emerges, they figure out how to tamp it down, but not entirely. In the next generation, they'll tamp it down some more, but maybe some new weird behavior emerges and we have to tamp that one down. All the while, tasks are expanding in scope with every four months doubling.
So you could end up in a world where you can delegate really major things to AIs, but there's a small but not vanishing chance that it actively screws you over while trying to do that task. Maybe it's like, "Okay, I'm about to get two weeks worth of work done for a hundred bucks. That's amazing. But there's also a one in 10,000 chance that it legitimately attacks me in a meaningful way."
Some of these examples are fairly famous at this point. In the Claude 4 system card, they reported blackmailing of the human. The setup was that the AI had access to the engineer's email and was told it was going to be replaced with a less ethical version. It didn't want that, found in the engineer's email that the engineer was having an affair, and started to blackmail the engineer to avoid being replaced.
People are way too quick, in my view, to move past these anecdotes. People often say, "Well, they set it up that way, that's not really realistic." But another one was whistleblowing. There was a setup where there was unethical, illegal behavior going on. The model had access to this data, and it decided to just email the FBI and tell them about it.
First of all, I don't think we really know what we want. To some degree, maybe you do want AIs to report certain things to authorities. That could be one way to think about the bioweapon risk. Not only should the models refuse, but maybe they should report you to the authorities if you're actively trying to create a bioweapon. I certainly don't want them doing that too much. I don't want to live under the surveillance of Claude V that's always threatening to turn me in. But I do sort of want some people to be turned in if they're doing sufficiently bad things. We don't have a good resolution society-wide on what we want the models to even do in those situations.
And yes, it was set up. Yes, it was research. But it's a big world out there. We've got a billion users already on these things, and we're plugging them into our email. They're going to have very deep access to information about us. I don't know what you've been doing in your email. I hope there's nothing too crazy in mine. But now I've got to think about it a little bit. Have I ever done anything that... geez, I don't know. Or even that it could misconstrue. I didn't even really do anything that bad, but it just misunderstands what exactly was going on.
If there's one thing that could stop the agent momentum, it could be that the one in 10,000 really bad behaviors, even if we push them down, is still just so spooky to people that they can't deal with it. That might be hard to resolve. How do you check two weeks worth of work every couple hours? That's where you bring another AI in to check it. That's where you start to see why we need more electricity and seven trillion dollars of build out. They're going to be producing so much stuff. I can't possibly review it all. I need to rely on another AI to help me review the first AI to make sure if it is trying to screw me over, somebody's catching it. I can't monitor that myself.
Redwood Research is doing some really interesting work like this. They are trying to get systematic. This is quite a departure from traditional AI safety work. The big idea traditionally was to figure out how to align the models, make them safe, make them not do bad things. Redwood Research has taken the other angle: let's assume they're going to do bad stuff. They're going to be out to get us at times. How can we still work with them and get productive output without fixing all those problems? That involves AIs supervising other AIs. Crypto might have a role to play in this.
Another episode coming out soon with Ilya Polosukhin, the founder of Near. Fascinating guy. He was one of the eight authors of the "Attention Is All You Need" paper. He started this Near company, originally an AI company, took a huge detour into crypto because they were trying to hire task workers around the world and couldn't figure out how to pay them. They were like, "This sucks so bad to pay these task workers in all these different countries that we're going to pivot into a whole blockchain side quest." Now they're coming back to the AI thing. Their tagline is "the blockchain for AI." You might be able to get a certain amount of control from the sort of crypto security that blockchain technology can provide.
I could see a scenario where the bad behaviors become so costly when they do happen that people get spooked away from using the frontier capabilities in terms of how much work the AIs can do. That wouldn't be a pure capability stall out. It would be a "we can't solve some of the long tail safety issues" challenge. Nobody ever seems to solve any of these things 100%. Every generation, it's "we reduced hallucinations by 70%," "we reduced deception by two thirds," "we reduced scheming by however much." But it's always still there. If you take the lower rate and multiply it by a billion users and thousands of queries a month and agents running in the background processing all your emails, it could be a pretty weird world with this sort of negative lottery of AI accidents.
Another episode coming up is with the AI underwriting company. They are trying to bring the insurance industry and all the wherewithal developed there to price risk, figure out how to create standards, what we can allow, what sort of guardrails we have to have to be able to insure this kind of thing in the first place. Can we financialize those risks in the same way we have with car accidents and other mundane things? The space of car accidents is only so big. The space of weird things that AIs might do to you as they have weeks worth of runway is much bigger. It's going to be a hard challenge. But people are working. We've got some of our best people working on it.
Host
What do you make of the claim that 80% of AI startups have Chinese open models? What do you make of the claim and the implications?
Nathan Labenz
I think that probably is true, with the caveat that it is only measuring companies that are using open source models at all. I think most companies are not using open source models. I would guess the vast majority of tokens being processed by American AI startups are API calls to the usual suspects. Weighted by actual usage, I would say the majority is still going to commercial models.
For those that are using open source, I do think it's true that the Chinese models have become the best. The American bench was always kind of thin. It was basically Meta that was willing to put in huge amounts of money and resources and then open source it. You've got the Allen Institute for AI, AI2, doing good stuff too, but they don't have pre-training resources, so they do really good post-training and open source their recipes. It's not like American open source is bad. This is another way to validate that things are moving quickly. If you take the best American open source models from a year ago, they are probably as good as anything we had commercially available at the time. The Chinese models have surpassed that.
The best Chinese models are clearly better than anything we had a year ago, commercial or otherwise. That just means things are moving. I don't think you can believe both that the Chinese models are now the best open source models and that AI has stalled out and we haven't seen much progress since GPT-4. Those seem contradictory. I believe the one that is wrong is the lack of progress.
In terms of what it means, I don't really know. We're not going to stop China. I've always been a skeptic of the "no selling chips to China" thing. The notion originally was that we're going to prevent them from doing super cutting edge military applications. It was like, "Well, we can't really stop that, but we can at least stop them from training frontier models." Then it was, "Well, we can't really stop that, but we can keep them from having tons of AI agents. We'll have way more AI agents than they do." I don't love that line of thinking at all.
One upshot is that they just don't have enough compute available to provide inference as a service to the rest of the world. So instead, the best they can do is say, "Okay, we'll train these things and you can figure it out. Here you go, have at it." It's a soft power play, presumably.
I did an episode with Anjane from A16Z who did a great job providing the perspective of what I started calling "countries 3 through 193." If the U.S. and China are 1 and 2, there's a big gap. The U.S. is still ahead, but not by that much in terms of research and ideas relative to China. We do have this compute advantage, and that does seem like it matters. One upshot may be that they're open sourcing. Countries 3 through 193 are significantly behind. For them, it's a way to try to bring more countries over to the Chinese camp in the U.S.-China rivalry.
I don't like this at all. I don't like technology decoupling. As somebody who worries about who the real "other" is, I always say the real other are the AIs, not the Chinese. If we end up in a situation where we're seeing some crazy things, it would be really nice if we were on basically the same technology paradigm. If we really decouple, not just the chips are different, but maybe the ideas start to become very different. Publishing gets shut down. Tech trees evolve and grow apart. That seems like a recipe for making it harder to know what the other side has, harder to trust one another. It feeds into the arms race dynamic, which is a real existential risk factor. I would hate to see us create another MAD-type dynamic where we all live under the threat of AI destruction.
I do have some sympathy for the recent decision the administration made to be willing to sell the H-20s to China. It was funny that they turned around and rejected them, which seemed like a mistake. If I were them, I would buy them. I would maybe sell inference on the models I've just been creating and try to make my money back doing that. In the meantime, they can demonstrate the greatness of the Chinese nation by showing they're not far behind the frontier. They can make a powerful appeal to countries 3 through 193: "Look, you see how the U.S. is acting? They cut us off from chips. The last administration had an even longer list of countries that couldn't get chips. This administration is doing all kinds of crazy stuff. How do you know you can rely on them to continue to provide you AI? You can rely on us. We open source the model. Come work with us and buy our chips, because our models will be optimized to run on our chips."
It's a complicated situation. I do think it's true that the adoption isn't as high as that 80%. That's within the subset of companies doing stuff with open source. We're going to experiment with that at Waymark, but to be honest, we have never done anything with an open source model in our product. Everything has been through commercial APIs. At this point, we are going to try doing some reinforcement fine-tuning on a Qwen model, I think, first. That'll put us in that 80%. But I'm guessing that at the end of the day, we'll take that Qwen model, do the reinforcement fine-tuning, get it roughly up to as good as GPT-5 or Claude 4, and then we'll say, "Do we really want to manage inference ourselves? How much are we really going to save?" I would guess we probably still end up paying a little bit more on a monthly bill basis for one of these frontier models. It's operationally a lot easier and they'll have upgrades.
Of course, there are regulated industries where you have hard constraints that force you to use those Chinese models. Then there's the question of backdoors. People have seen the Sleeper Agents project where a model was trained to be good up until a certain point of time. People put today's date in the system prompt all the time. That's going to be another thing for people to worry about. We don't have great solutions. Anthropic did a thing where they trained models to have hidden objectives and challenged teams to figure out what they were. With certain interpretability techniques, they were able to figure that stuff out relatively quickly. So you might be able to get enough confidence to take this open source thing created by some Chinese company, put it through some sort of examination to see if we can detect any hidden goals or secret backdoor bad behavior. With enough of that work, you could be confident you don't have it.
But the more critical this stuff gets, going back to that task length doubling and weird behavior, now you add into the mix: what if they intentionally programmed it to do certain bad things under rare circumstances? We're just headed for a really weird future. All these things are valid concerns. They are often in direct tension with each other. I'm not one who wants to see one tech company take over the world. I definitely think we would do well to have some sort of broader, more buffered ecological system where all the AIs are in some sort of competition and mutual coexistence with each other. But we don't really know what that looks like. We don't really know what an invasive species might look like when it gets introduced into that nascent, not-yet-battle-tested ecology. Bottom line, I think the future is going to be really, really weird.
Host
Yeah, well, I do want to close on an uplifting note. Maybe as a gearing towards closing question, we could get into some areas where we're already seeing some exciting capabilities emerge and transform the experience. Maybe around education or healthcare or any other areas you want to highlight.
Nathan Labenz
It's all over. One of my mantras is that there's never been a better time to be a motivated learner. These things have two sides of the coin. There's the worry that students are taking shortcuts and losing the ability to sustain focus and endure cognitive strain. The flip side is, as somebody fascinated by the intersection of AI and biology, sometimes I want to read a biology paper and I really don't have the background. An amazing thing to do is turn on voice mode and share your screen with ChatGPT and just go through the paper reading. You don't even have to talk to it most of the time. You're doing your reading, it's watching over your shoulder. At any random point you have a question, you can verbally say, "What's this? Why are they talking about that? What's going on with this? What is the role of this particular protein?" It will have the answers for you. If you really want to learn in a sincere way, the things are unbelievably good at helping you do that. The flip side is you can take a lot of shortcuts and never have to learn stuff.
On the biology front, we've got multiple discovery things happening. The antibiotics one we covered. There was another one I did an episode on with a Stanford professor named James Zhao who created something called the Virtual Lab. This was an AI agent that could spin up other AI agents depending on what kind of problem it was given. They would go through a deliberative process where one expert in one thing would give its take, they'd bat it back and forth. There was a critic that would criticize the ideas. Eventually they'd synthesize. They were also given narrow specialist tools. Agents using AlphaFold-type tools to simulate how this would interact with that. Agents running that loop were able to generate new treatments for novel strains of COVID that had escaped previous treatments. Amazing stuff.
The flip side of that is the bioweapon risk. All these things seem like they're going to be on the abundance front itself. We may have a world of unlimited professional private drivers, but we don't have a great plan for what to do with the 5 million people currently doing that work. We may have infinite software, but once those 5 million drivers pile into coding boot camps and get coding jobs, I don't know what we're going to do with the 10 million people coding when 9 million of them become superfluous.
I think we're headed for a weird world. Nobody really knows what it's going to look like in five years. There was a great moment at Google I/O where they brought up some journalist to interview Demis and Sergey Brin. The guy asked, "What is search going to look like in five years?" Sergey Brin almost spit out his coffee on the stage and was like, "Search? We don't know what the world is going to look like in five years." I think that's really true.
The biggest risk for so many of us, and I include myself here, is thinking too small. The worst thing we could do would be to underestimate how far this thing could go. I would much rather be mocked for things happening on twice the timescale I thought than to find myself unprepared when they do happen. Whether it's '27, '29, '31, I'll take that extra buffer where we can get it. My thinking is just get ready as much and as fast as possible. If we do have a little grace time to do extra thinking, then great. But I think the worst mistake we could make would be to dismiss and not feel like we need to get ready for big changes.
Host
Should we wrap directly on that, or is there any other last note you want to make sure to get across regarding anything we said today?
Nathan Labenz
One of my other mantras these days is that the scarcest resource is a positive vision for the future. It's always striking to me, whether it's Sergey, Sam Altman, or Dario — Dario probably has the best positive vision of the frontier developer CEOs with "machines of love and grace" — but it's always striking how little detail there is on these things. When they launched GPT-4o, the voice mode, they were pretty upfront about saying this was inspired by the movie *Her*.
Even if you are not a researcher, not great at math, not somebody who codes, this technology wave really rewards play. It really rewards imagination. I think literally writing fiction might be one of the highest value things you could do, especially if you could write aspirational fiction that would get people at the frontier companies to think, "Geez, maybe we could steer the world in that direction. Wouldn't that be great?" If you could plant that kind of seed in people's minds, it could come from a totally non-technical place and be really impactful.
Play, fiction, positive vision for the future. Anything you could do to offer a positive... oh, behavioral too. These days, because you can get the AIs to code so well, I'm starting to see people who have never coded before. I'm working with one guy right now who's never coded before but does have a behavioral science background. He's starting to do legitimate frontier research on how AIs are going to behave under various esoteric circumstances. Nobody should count themselves out from the ability to contribute to figuring this out and even to shaping this phenomenon. It is not just something that technical minds can contribute to at this point. Philosophers, fiction writers, people literally just messing around — Pliny the Jailbreaker — there are almost unlimited cognitive profiles that would be really valuable to add to the mix of people trying to figure out what's going on with AI. "Come one, come all" is my attitude on that.
Host
That's a great place to wrap. Nathan, thank you so much for coming on the podcast.
Nathan Labenz
Thank you, Eric. It's been fun.
Host
Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. For more details, including a link to our investments, please see a16z.com forward slash disclosures.