Is intelligence just compute plus data? And if so... what happens next?
In this episode, Dwarkesh Patel, host of one of the most respected AI podcasts and author of “The Scaling Era”, joins us to break down the exponential rise of artificial intelligence. We explore how scaling laws became the engine behind modern models, why AI still can’t learn on the job, and whether we’re truly on the brink of AGI.
From coding to consciousness, from geopolitics to governance, Dwarkesh shares a grounded but urgent perspective on what it means to live through the transition and how we might prepare for what’s coming.
------
💫 LIMITLESS | SUBSCRIBE & FOLLOW
https://limitless.bankless.com/
https://x.com/LimitlessFT
------
BANKLESS SPONSOR TOOLS:
🪙FRAX | SELF SUFFICIENT DeFi
https://bankless.cc/Frax
🦄UNISWAP | SWAP ON UNICHAIN
https://bankless.cc/unichain
🛞MANTLE | MODULAR LAYER 2 NETWORK
https://bankless.cc/Mantle
🟠BINANCE | THE WORLDS #1 CRYPTO EXCHANGE
https://bankless.cc/binance
🦎COINGECKO API | REAL-TIME CRYPTO PRICE & MARKET DATA
https://bankless.cc/coingecko
------
TIMESTAMPS
0:00 Intro
6:59 What Is the Scaling Era?
9:31 Compute + Data = Intelligence?
15:08 Are We Building God?
17:57 The Catalyst: Discovering the Scaling Laws
20:54 Model Capabilities
24:35 Why Coding Is the AI Frontier
31:43 What AIs Still Can’t Do
36:26 AI Isn’t Around the Corner
46:01 Is AI Really Smart?
50:54 Inside the Black Box
54:46 AI Safety and Alignment
1:03:49 Who Controls the Models?
1:08:16 Swiss Cheese Safety & Liberal AI
1:19:30 Scaling Inputs: Energy, Compute, Data
1:26:40 Jobs, Growth & the Intelligence Curse
1:32:07 Why 60% AGI by 2040?
1:36:14 Dwarkesh’s Personal View
------
RESOURCES
Dwarkesh Patel
https://x.com/dwarkesh_sp
Dwarkesh’s Book
https://www.amazon.com/Scaling-Era-Oral-History-2019-2025/dp/1953953557
Dwarkesh’s Podcast
https://www.dwarkesh.com/
------
Not financial or tax advice. See our investment disclosures here:
https://www.bankless.com/disclosures
No summary for this episode yet.
Hey guys, we have a special episode today. We have Dora Kesh Patel on the podcast. Now Dora Kesh is probably one of my favorite podcasters. He is most in my podcast rotation list. And specifically, he's great on AI topics, which is the subject of today's episode. This is about the scaling era of AI. Scaling era is something new in AI, new for humanity, it's something we've never seen before, and Dora Kesh is very much on the frontier of this movement. So we get through the full history and where we've come today, and I loved every minute of this conversation. Now you can usually catch episodes like this on our limitless podcast feed.
That's where my co-host Josh and David and Ejaz all go deep into the AI rabbit hole. It's like bank less, but for AI. So if limitless is not your podcast feed rotation yet, you got to go subscribe to it, catch it on Spotify, on YouTube, or wherever you access these podcasts. Now on crypto, I did ask Dora Kesh when he's going to do a crypto podcast because I would very much love to hear Vitalik Buterin on Dora Kesh, but it still might be a while out. Apparently, the only podcast Dora Kesh has ever done on crypto was with Sam Beckman Freed. And we all know what happened there. It's more hangover, I guess, from the crypto criminals of 2022.
But we've certainly come a long way since then, and so has AI. Please enjoy this episode with Dora Kesh Patel. Imagine a world where traditional finance meets the power of blockchain seamlessly. That's what mantle is pioneering with blockchain for banking, a revolutionary new category at the intersection of trade five and web three. At the heart is you are the world's first money out built fully on chain. It gives you a Swiss iBan account blending fiat currencies like the euro, the Swiss fronk, the United States dollar, or the Ramimbi with crypto all in one place. Enjoy real world usability and blockchains trust and programmability.
Transactions post directly to the blockchain compatible with tradify rails and packed with integrated DeFi features. UR transforms mantle network into the ultimate platform for on chain financial services, unifying payments, trading, and assets like the MI4, the MEs protocol, and functions FBTC. Backed by developer grants, ecosystem incentives, and top distribution through the UR app, reward stations, and buy bit launch pool. For M&T holders, every economic activity in UR drives value back to you, embodying the entire stack and future growth of this super app ecosystem. Follow mantle on x at mantle underscore official for the latest updates on blockchain for banking.
That's x.com slash mantle underscore official finance is the world's number one crypto exchange over 275 million users already trust their world to class security. Finance makes starting crypto as simple as it should be whether it's learning about crypto on finance academy or browsing hundreds of assets and viewing your newly created portfolio in a clear easy to track dashboard. Finance helps you go at your own pace. For hard core traders, Binance Pro opens up industry leading services for trading professionals with fully bespoke trading products along with a suite of white glove services for VIP and institutional clients.
Need support? 24-7 customer service is on hand whenever you need it and with some of the lowest fees and deep list liquidity in the market is no surprise why over 275 million users trust finance for everything crypto. Download Binance today and get started in minutes. Binance is not available in certain countries including the United States, check its terms for more information. Ethereum's Layer 2 universe is exploding with choices, but if you're looking for the best place to park and move your tokens, make your next stop Unichain. First, liquidity. Unichain hosts the most liquid Uniswap V4 deployment on any Layer 2, giving you deeper pools for flagship pairs like each USDC.
More liquidity means better prices, less slippage and smoother swaps exactly what traders rate. The numbers back it up. Unichain leads all Layer 2's in total value locked for Uniswap V4, and it's not just a deep, it's fast and fully transparent, purpose-built to be the home base for DeFi and cross-chain liquidity. When it comes to costs, Unichain is a no-brainer. Transaction fees come in about 95% cheaper than Ethereum mainnet, slashing the price of creating or accessing liquidity. Want to stay in the loop on Unichain? Visit Unichain.org or follow at Unichain on X for all the updates. To our cash patelle, we are big fans, it's an honor to have you.
Thank you so much for having me on. OK, so you have a book out. It's called The Scaling Era, an oral history of AI from 2019 to 2025. These are some key dates here. This is really a story of how AI emerged, and it seemed to have exploded on people's radar over the past five years, and everyone in the world it feels like is trying to figure out what just happened and what is about to happen. And I feel like for this story, we should start at the beginning as your book does. What is the Scaling Era of AI and what about to start? What were the key milestones? So I think the under-told story about everybody's, of course, been hearing more and more about AI, the under-told story is that the big contributor to these AI models getting better over time has been the fact that we are throwing exponentially more compute into trading frontier systems every year.
So by some estimates, we spend four X every single year over the last decade trading the frontier system than the one before it. And that just means that we're spending hundreds of thousands of times more compute than the systems of the early 2010s. Of course, we've also had algorithmic breakthroughs in the meantime 2018, we had the transformer. Since then, obviously, many companies have made small improvements here and there. And the overwhelming fact that we're spending already hundreds of billions of dollars in building up the infrastructure, the data centers, the chips for these models. And this picture is only going to intensify if this exponential keeps going.
For X a year, over the next two years, is something that is on the minds of the CFOs of the big hyperscalers and the people planning the expenditures and training going forward. But is not as common in the conversation around where AI is headed. So what do you feel like people should know about this? What is the scaling era? There have been other eras, maybe, of AI or a compute, but what's special about the scaling era? People started noticing, well, first of all, in 2012, there's this, I guess that's over on others, started using neural networks in order to categorize images. And just noticing that instead of doing something hand-coded, you can get a lot of juice out of just neural networks, black boxes, you just trained them to identify what thing is like what.
And then people started playing around these neural networks more using this for different kinds of applications. And then the question became, we're noticing that these models get better if you throw more data at them and you throw more compute at them. How can we shove as much compute into these models as possible? And the solution ended up being obviously internet text. So you need an architecture, which is amenable to the trillions of tokens that have been written over the last few decades and put up on the internet. And we had this happy coincidence of the kinds of architectures that are amenable to this kind of training with the GPUs that were originally made for gaming.
We've had decades of internet text being compiled. And Elyasets are actually called it the fossil fuel of AI, this reservoir that we can call upon to train these minds, which are like, they're fitting the mold of human thought because they're trading on trillions of tokens of human thought. And so then there's just been a question of making these models bigger, of using this data that we're getting from internet text to further keep training them. And over the last year, as you know, the last six months, the new paradigm has been not only are we going to pre-train on all this internet text, we're going to see if we can have them solve math puzzles, coding puzzles, and through this give them reasoning capabilities.
The kind of thing, by the way, I mean, I have some skepticism around AGI just around the corner, which we'll get into. But just the fact that we now have machines, which can like, reason, you can like ask a question to a machine and it'll go away for a long time and it'll like think about it. And then like, we'll come back to you with a smart answer. And we just sort of take it for granted, but obviously, we also know that they're extremely good at coding, especially. I don't know if you actually got a chance to play around with cloud code or cursor or something. But it's a wild experience to design, explain it at a high level by one, an application that does X.
15 minutes later, there's like 10, 10 files of code and the application is built. After we stand, I have takes on how much this can continue. The other important dynamic, I'll add my monologue here, but the other important dynamic is that if we're going to be living in the scaling era, you can't continue exponentials forever, and certainly not exponentials that are 4x a year forever. And so right now, we're approaching a point where within by 2028, at most by 2030, we will literally run out of the energy we need to keep training these frontier systems, the capacity at the leading edge nodes, which manufacture the chips that go into the dyes, which go into these GPUs, even the raw fraction of GDP that we'll have to use to train frontier systems.
So we have a couple more years left of the scaling era, and the big question is, will we get to AGI before them? I mean, that's kind of a key insight of your book that, like, we're in the middle of the scaling era. I guess we're like, you know, six years in or so, and we're not quite sure as like, like the protagonist in the middle of the story, like, we don't know exactly which way things are going to go. But I want you to maybe to our cash help folks get an intuition for why scaling in this way even works, because I'll tell you, like, for me and for most people, I mean, our experience with these revolutionary AI models probably started in 2022 with ChatGPD 3, and then ChatGPD 4, and seeing all the progress, all these AI models.
And it just seems really unintuitive that if you take a certain amount of compute, and you take a certain amount of data, outpops AI, outpops intelligence, could you help us like get an intuition for this magic? Like, how does the scaling law even work? Compute plus data equals intelligence? Is that really all it is? To be honest, I've asked so many AI researchers this exact question on my podcast, and I could tell you some potential theories of why it might work. I don't think we understand. I'll just say that. I don't think we understand. We don't understand how this works. We don't understand how it works.
We have evidence from actually, of all things, primatology of what could be going on here, or at least like why it was similar patterns in other parts of the world. So what I found really interesting, there's this research by this researcher, Susanna Herculina Huzel, which shows that if you look at how the number of neurons in the brain of different kinds of rat species increases, as the weight of their brain's increase from species to species, there's this very sublime your pattern. So if their brain size doubles, the neuron count will not double between different rat species. And there's other animals, there's other kinds of families of species, which is as true.
The two interesting exceptions to this rule, where there is actually a linear increase in neuron count and brain size, is one, certain kinds of birds. So birds are actually very smart, given the size of their brains, and primates. So the theory of what happened with humans is that we unlocked an architecture that was very scalable. So the way people talk about transformers being more scalable than LSTMs, the thing that preceded them in 2018, we unlocked this architecture as it's very scalable. And then we were in an evolutionary niche millions of years ago, which rewarded marginal increases in intelligence.
If you get slightly smarter, yes, the brain costs more energy, but you can save energy in terms of not having to, you can cook food so you don't have to spend that much on digestion, you can find a game, you can find different ways of foraging. Birds were not able to find this evolutionary niche, which rewarded the incremental increases in intelligence, because if your brain gets too heavy as a bird, you're not going to fly. So it was this happy coincidence of these two things. Now why is it the case that the fact that our brains could get bigger resulted in us becoming as smart as we are? We still don't know.
And there's many different dissimilarities between AI's and humans, while our brains are quite big, we don't need to be trained, you know, a human from the age there's zero to 18 is not seeing within an order of magnitude of the amount of information these LLMs are trained on. They're extremely data inefficient, they need a lot more data, but the pattern of scaling I think we see in many different places. So is that a fair kind of analog? The same analog has always made sense to me, it's just like in transformers or like neurons, you know, AI models are sort of like the human brain, evolutionary pressures are like gradient descent, reward algorithms, and out pops human intelligence.
We don't really understand that, we also don't understand AI intelligence, but it's basically the same principle at work. I think it's a super fascinating but also very thorny question because is gradient intelligence like evolution? Well yes in one sense, but also when we do gradient descent on these models, we start off with the weights and then we're, you know, it's like learning how does chemistry work, how does coding work, how does math work, and that's actually more similar to lifetime learning which is say that like by the time you're already born to the time you turn 18 or 25, the things you learn, and that's not evolution, evolution designed the system or the brain by which you can do that learning, but the lifetime learning itself is not evolution.
And so there's also this interesting question of yeah, is training more like evolution? In which case actually we might be very far from age AI because the amount of compute that's been spent over the course of evolution to discover the human brain, you know, could be like 10 to the 40 flops, there's been estimates, you know, whatever. I'm sure it will bore you to discover, talk about how these estimates are derived, but just like how much versus is it like like a single lifetime, like going from the age of zero to the age of 18, which is closer to I think 10 to the 24 flops, which is actually less than compute than we use to train frontier systems.
All right, anyways, we'll get back to more relevant questions. Well, here's kind of a big picture question as well as like I'm constantly fasting with the metaphysical types of discussions that some AI researchers kind of take, like a lot of AI researchers will talk in terms of when they describe what they're making, we're making God, like why do they say things like that? What is this talk of like making God? What does that mean? Is it just the idea that scaling laws don't cease? And if we can scale intelligence to AGI, then there's no reason we can't scale far beyond that and create some sort of a God-like entity, and essentially that's what the quest is.
We're making artificial superintelligence, we're making a God, we're making God. I didn't even focus too much on when the, I think this God discussion focuses too much on the hypothetical intelligence of a single copy of an AI. I do believe in the notion of a superintelligence, which is not just functionally, which is not just like, oh, it knows a lot of things, but it's actually qualitatively different than human society. But the reason is not because I think it's so powerful that anyone individual copy of AI will be smart, but because of the collective advantages that AI's will have, which have nothing to do with their raw intelligence, but rather the fact that these models will be digital or they already are digital, but eventually they'll be smart as humans at least.
But unlike humans, because of our biological constraints, these models can be copied. If there's a model that has learned a lot about a specific domain, you can make infinite copies of it, and now you have an infinite copies of Jeff Dean or Ilya Satskiver or Elon Musk or any skilled person you can think of, they can be merged. So the knowledge that each copy is learning can be amalgamated back into the model and then back to all the copies. They can be distilled. They can run at superhuman speeds. These collective advantages, also they can communicate in lane space. They're immortal. I mean, you know, as another example.
Yeah, exactly. So I mean, that's actually, tell me if I'm rabbit holding too much, but one really interesting question we'll come about is, how do we prosecute AI's? Because the way we prosecute humans is that we will throw you in jail if you commit a crime. But if there's trillions of copies or thousands of copies of an AI model, if a copy of an AI model, if an instance of an AI model does something bad, what do you do? Does the whole model have to get, and how do you even punish a model, right? Does it care about its weights being squandered? Yeah, there's all kinds of questions that arise because of the nature of what AI's are.
And also, who is liable for that, right? Like is it the tool maker? Is it the person using the tool? Who is responsible for these things? There's one topic that I do want to come to here about scaling laws, and what time did we realize that scaling laws were going to work? Because there were a lot of DCs early in the days, early 2000s, about AI, how we were going to build better models, eventually we got to the transformer. But at what point did researchers and engineers start to realize that, hey, this is the correct idea. We should start throwing lots of money and resources towards this versus other ideas that were just kind of theoretical research ideas, but never really took off.
We kind of saw this with GPT 2 to 3 where there's this huge improvement, a lot of resources went into it. Was there a specific moment in time or a specific breakthrough that led to the start of the scaling laws? I think it's been a slow process of more and more people appreciating the this like nature of the overwhelming role of compute in driving forward progress. In 2018, I believe Dario Amadei wrote a memo that was secret while he was at OpenAI, now he's the CEO of Anthropic, but while he's OpenAI, he subsequently revealed on my podcast that he wrote this memo where he, the title of the memo was called big blob of compute.
And it says basically what you expected to say, which is that like, yes, there's ways you can mess up the process of training. You have the wrong kinds of data or initializations, but fundamentally, AGI is just a big blob of compute. And then we've gotten over the subsequent years, there was more empirical evidence, so a big update. So it was 2021, but correct me, somebody who definitely will correct me in the comments now wrong. There were these, there's been multiple papers of the scaling laws where you can show that the loss of the model on the objective of predicting the next token goes down very predictably, almost to like multiple decimal places of correctness based on how much more compute you throw in these models and the compute itself as a function of the amount of data you use and how big the model is, how many parameters it has.
And so that was an incredibly strong evidence back in the day, a couple of years ago, because then you could say, well, okay, if it really has this incredibly low loss of predicting the next token in all human output, including scientific papers, including GitHub repositories, and doesn't it mean it has actually had to learn coding and science and all these skills in order to make those predictions, which actually ended up being true. And it was something people, you know, we take it for granted now, but it actually, even as of a year or two ago, people were really even denying that premise. But some people, a couple of years ago, just like thought about it and like, yeah, actually that would mean that it's learned the skills.
And that's crazy that we just have this strong empirical pattern that tells us exactly what we need to do in order to learn these skills. And it creates this weird perception right where like very early on, and so it's this day, it really is just a token predictor, right? Like we're just predicting the next word in the sentence, but somewhere along the lines, it actually creates this perception of intelligence. So I guess we covered the early historical context. I kind of want to bring the listeners up to today where we are currently where the scaling loss I brought us in the year 2025. So can you kind of outline where we've gotten to from early days of GPTs to, we now we have GPT-4, we have gym and ultra, we have club, which you mentioned earlier, we had the breakthrough of reasoning.
So what can leading frontier models do today? So there's what they can do and then there's the question of what, what methods seem to be working? I guess we can start at what they seem to be able to do, they've shown to be remarkably useful at coding and not just at answering direct questions about how does this line of code work or something. But genuinely just autonomously working for 30 minutes or an hour, during the task it would take a front end developer a whole day to do and you can just ask them at high level do this kind of thing and they can go ahead and do it. Obviously, we played around with it, you know that they're extremely useful assistance in terms of research, in terms of even therapists, whatever other use case is.
On the question of what's training methods that seem to be working, we do seem to be getting evidence that pre-trading is plateauing, which is to say that we had GPD 4.5, which was just following this old mold of make the model bigger, but it's fundamentally doing the same thing of next token prediction and apparently didn't pass muster the opening I had to deprecate it because there's this dynamic where the bigger the model is, the more it costs not only to train, but also to serve, right, because every time you serve a user, you're having to run the whole model, which is going, so what does seem to be working is RL, which is this process of not just training them on existing tokens on the internet, but having the model itself try to answer math and coding problems.
And finally, we got to the point where the model is smart enough to get it right some of the time. And so you can give it some reward and then it can saturate these tough reasoning problems. And then what was the breakthrough with reasoning for the people who aren't familiar? What made reasoning so special that we hadn't discovered before and what did that kind of like unlock for models that we use today? I'm honestly not sure. I mean, we had GPD 4 came out a little over two years ago. And then it was after two years after GPD 4 came out that O1 came out, which was the original reasoning breakthrough, I think last November.
And subsequently a couple of months later, deep sea showed in their R1 paper. So deep sea open source to their research and they explained exactly how their algorithm worked. And it wasn't that complicated. It was just like what you would expect, which is get some math problems, give for some initial problems, tell the model exactly what the reasoning trace looks like, how you solve it, just like write it out, and then have the model like try to do it raw on the remaining problems. Now, I know it sounds incredibly arrogant to say, well, I wasn't that complicated, why did I take your years? I think there's an interesting insight there of even things which you think will be simple.
In terms of high level description, how to solve the problem, end up taking longer in terms of haggling out the remaining engineering hurdles, then you might naively assume. And that should update us on how long it will take us to go through the remaining bottlenecks on the path to AGI. Maybe that will be tougher than people imagine, especially the people who think we're only two to three years away. All of this to say, yeah, I'm not sure why it took so long after GBD4 to get a model trained on a similar level capabilities that could then do reasoning. And in terms of those abilities, the first answer you had to what can it do was coding.
And I hear that a lot of the time when I talk to a lot of people is that coding seems to be a really strong suit and a really huge unlock to using these models. And I'm curious why coding over general intelligence? Is it because it's placed in a more confined box of parameters? I know in the early days, we had the AlphaGo and we had the AI's playing chess. And they perform so well because they were kind of contained within this box of parameters that was a little less open-ended to general intelligence. Is that the reason why coding is kind of at the frontier right now of the ability of these models?
There's two different hypotheses. One is based around this idea called Moravax Paradox. And this was an idea. By the way, one super interesting figure, actually, I should have mentioned him earlier. One super interesting figure in the history of scaling is Hans Moravax, who I think in the 90s predicts that 2028 will be the year that we will get to AGI in the way he predicts this, which is like, we'll see what happens, but not that far off the money as far as I'm concerned. The way he predicts this is he just looks at the growth in computing power year over a year, and then looks at how much compute he estimated the human brain to require.
And he's just like, okay, we'll have computers as powerful as the human brain by 2028. Which is like at once a deceptively simple argument, but also ended up being incredibly accurate and worked, right? I might not have fact that I was 2028, but within something you would consider a reasonable guess given what we know now. Sorry, anyway, so the Moravax Paradox is this idea that computers seemed in AGI get better first at the skills which humans are the worst at, or at least there's a huge variation in the human repertoire. So we think of coding as incredibly hard, right? We think this is like the top 1% of people will be excellent coders.
We also think of reasoning as very hard, right? Most of you read Aristotle, he says, the thing which makes human special, which distinguishes us from animals is reasoning. And these models aren't that useful yet at almost anything. The one thing they can do is reasoning. So how do we explain this pattern? And Moravax's answer is that evolution has spent billions of years optimizing us to do things we take for granted. Move around this room, right? Pick up this can of coke, move it around, drink from it. And that we can't even get robots to do at all yet. And in fact, it's so ingrained in us by evolution that there's no human or at least humans who don't have disabilities will all be able to do this.
And so we just take it for granted that this is an easy thing to do. But in fact, it's an evidence of how long evolution has spent getting humans up to this point. And it has reasoning, logic, all of these skills have only been optimized by evolution over the course of the last few million years. So there's been a thousand full less evolutionary pressure towards coding than towards just basic locomotion. And this has actually been very accurate in predicting what kinds of progress we see even before we got deep learning, right? In the 40s, when we got our first computers, the first thing that we could use them to do is long calculations for ballistic trajectories at the time for World War II.
Humans suck at long calculations by hand. And anyways, so that's the explanation for coding, which seems hard for humans is the first thing that went to AI's. Now there's another theory, which is that this is actually totally wrong. It has nothing to do with this, this seeming paradox of how long evolution is optimized as for and everything to do with the availability of data. So we have GitHub, this repository of all of human code, this all open source code written in all these different languages, trillions and trillions of tokens. We don't have an analogous thing for robotics. We don't have this pre-training corpus.
And that explains why code has made so much more progress than robotics. That's fascinating because if there's one thing that I could list that we'd want AI to be good at, probably coding software is number one on that list. Because if you have a term complete intelligence that can create, term complete software, is there anything you can't create once you have that? Also, like the idea of Morvax, paradox. I guess that sort of implies a certain complementarianism with humanity. So if robots can do things that robots can do really well and can't do the things humans can do well, well, perhaps there's a place for us in this world.
And that's fantastic news. It also may be implies that humans have kind of scratched the surface on reasoning potential. I mean, if we've only had a couple of million years of evolution and we haven't had the data set to actually get really good at reasoning, it seems like there'd be a massive amount of upside, unexplored territory, like so much more intelligence that nature could actually contain inside of reasoning. I mean, are these some of the implications of these ideas? Yeah, I know. That's a great insight. Another really interesting insight is that the more variation there is in a skill in humans, the better and faster that AI's will get at it.
Because coding is a kind of thing where one person of humans are really good at it. The rest of us will, if we try to learn it, we'd be okay at it or something, right? And because evolution has spent so little time optimizing us, there's this room for variation where the optimization hasn't happened uniformly or it hasn't been valuable enough to sort of saturate the human gene pool for this skill. I think you made an earlier point that I thought was really interesting. I wanted to address. Can you remind me of the first thing you said? Is it the complementarianism? Yes. You can take it as a positive future.
You can take it as a negative future in the sense that, well, what is the complement terry skills we're providing? We're good meat robots? Yeah, the low-skip labor of the situation. They can do all the thinking and planning. One dark future, one dark vision of the future is we'll get those meta-glasses and the AI speaking into our ear, and it'll tell us to put this brick over there so that the next data center can be built, because the AI's got the plan for everything. It's got the better design for the ship and everything. You just need to move things around for it, and that's what human labor looks like until robotics is solved.
So yeah, it depends on how you go. On the other hand, you'll get paid a lot, because it's worth a lot to move those bricks. We're building AGI here, but yeah, it depends on how you come out on that question. There seems to be something to that idea, going back to the idea of the massive amount of human variation. We have just in the past month or so, we have news of meta-hiring AI researchers for a $100 million signing bonuses. What is the average software engineer make versus what does an AI researcher make at the top of the market? It's got to imply, obviously, there's some things going on with demand and supply, but also, it does also seem to imply that there's massive variation in the quality of a software engineer, and if the AI's can get to that quality, what does that unlock?
Yeah. I guess we have coding down right now. Another question, though, is what can't AI's do today? How would you characterize that? What are the things they just don't do well? I've been interviewing people on my podcast who have very different time-lapse-run rule with a TGI. I have had people on who think it's two years away and some who think it's 20 years away. The experience of building AI tools for myself, actually, has been the most inside-driving, maybe research I've done on the question of when AI is coming. More than the gesture of use. Yeah, because I have had, I've probably spent on the order of 100 hours trying to build these little tools.
The kinds I'm sure you've also tried to build, of rewrite auto-generated transcripts for me to make them sound the way a human would write them. Find clips for me to treat out, write essays with me, co-write them passively passage, these kinds of things. What I found is that it's actually very hard to get human-like labor out of these models, even for tasks like these, which should be debt center in the repertoire of these models. They're short horizon, they're language in, language out. They're not contingent on understanding some, like, you know, thing I said like a month ago, this is just like, this is the task.
And I was thinking about why is it the case that I still haven't been able to automate these basic language tasks, why do I still have a human work on these things? And I think the key reason that you can't automate even these simple tasks is because the models currently lack the ability to do on the job training. So if you hire a human for the first six months, for the first three months are not going to be that useful, even if they're very smart, because they haven't built up the context, they haven't practiced the skills, they don't understand how the business works. What makes humans valuable is not that mainly they're wrong, like, interlectability matters, but it's not mainly that.
It's their ability to interrogate their own failures in this really dynamic organic way, to pick up small efficiencies and improvements as they practice the task, and to build up this context as they work within a domain. And so sometimes people wonder, look, if you look at the revenue of OpenAI, the annual recurring revenue, it's on the order of $10 billion. Coals makes more money than that. McDonald's be telling you that, right? So why is it that if they've got AGI, they're, you know, like, Fortune 500 isn't reorganizing their workflows to, you know, use OpenAI models at every layer of the stack.
My answer, sometimes people say, well, it's because people are too stodgy. The management of these companies is like, not moving fast enough on AI. That could be part of it. I think mostly it's not that. I think mostly it genuinely is very hard to get human-like labor out of these models. Because you can't, so you're stuck with the capabilities you get out of the model out of the box. So there might be five out of ten, and we're writing the transcript for you. But if you don't like how it turned out, if you have feedback for it, if you want to keep teaching it over time, one's the session ends.
The model, like every, everything knows about you has gone away. You had to restart again. It's like working with an M&E's Jack employee. You had to restart again every day as the first day of employment, basically. Yeah, exactly. It's a groundhog day for them every day, or every couple of hours in time. And that makes you very hard for them to be that useful as an employee, right? They're not really an employee at that point. This I think not only is it key bottleneck to the value of these models, because human labor is worth a lot, right? Like $60 trillion in the world is paid to wages every year.
If these model companies are making on the order of $10 billion a year, that's a big way to AGI. And what explains that gap? What are the bottlenecks? I think a big one is this continual learning thing. And I don't see an easy way that that just gets solved within these models. There's no like, with reasoning, you could say, oh, it's like training on math and code problems. And then we'll get reasoning. And that worked. I don't think there's something super obvious there for how do you get this online learning, this on the job training, working with these models? Okay. Can we talk about that?
Go a little bit deeper on that concept, so this is basically one of the concepts you wrote in your recent post. A.I. is not right around the corner. Even though you're an AI optimist, I would say, and overall an AI accelerationist, you were saying it's not right around the corner. You're saying the ability to replace human labor is a waste out, not forever out, but I think you said, you know, some around 2032, if you had to guess on when the estimate was. The reason you gave is because AI's can't learn on the job, but it's not clear to me why they can't. Is it just because the context window isn't large enough?
Is it just because they can't input all of the different data sets and data points that humans can? Is it because they don't have stateful memory the way a human employee, like, because if it's these things, all of these do seem like solvable problems, and maybe that's what you're saying. They are solvable problems. It's just a little bit longer than some people think they are. I think it's, like, in some deep sense, a solvable problem, because, like, eventually, we will build AGI, and to build AGI, we will have had to solve the problem. My point is that the obvious solutions you might imagine, for example, expanding the context window, or having this, like, external memory, using systems like RAG, these are basically techniques you already have to, it's called retrieval-logmented generate.
Anyways, these kinds of retrieval-logmented generation, I don't think these will suffice, and just to put a finer point at first about, like, what is the problem? The problem is, exactly as you say, that within the context window, these models actually can learn on the job, right? So if you talked it for long enough, it will get much better at understanding your needs and what your exact problem is. If you're using it for research, for your podcast, it will get a sense of, like, oh, they're actually, especially curious about these kinds of questions, let me focus on that. It's actually very human-like in context, right?
The speed at which it learns, the tacit knowledge it picks out. The problem, of course, is the context length, or even the best models only last a million or two million tokens, that's, at most, like, an hour of conversation. Now then you might say, okay, well, why can't we just solve that by expanding the context window, right? So the context window has been expanding for the last few years, why can't we just continue that? A billion-token context window. Something like this. So 2018 is when the transformer came out, and the transformer has the attention mechanism. The attention mechanism is inherently quadratic with the nature, the length of the sequence, which is to say that if you go from, if you double, go from one million tokens to two million tokens, it actually costs four times as much compute to process that two million tokens.
It's not just two to as much compute. So it gets super linearly more expensive as you increase the context length. And for the last seven years, people have been trying to get around this inherent quadratic nature of attention. Of course, we don't know secretly what the labs are working on, but we have Frontier companies like DeepSeek, which have open source search, and we can just see how their algorithms work. And they found this constant time modifiers to attention, which is to say that there's like a, it'll still be quadratic, but it will be like one half times quadratic. But the inherent like super linear in this has not gone away.
And because of that, yeah, you might be able to increase it from one million tokens to two million tokens by finding another hack, like, make sure experts just one such thing. Layton attention is another such technique. But or KV Cash, right? There's many other things that have been discovered. But people have not discovered, okay, how do you get around the fact that like you went to a billion, it would be a billion squared as expensive in terms of compute to process that token. And so I don't think you'll just get it by increasing the length of the context one, though, basically. That's fascinating.
Yeah. I didn't realize that. Okay. So the other reason in your post that AI is not right around the corner is because it can't do your taxes. And or cash, I feel your pain, man, taxes are just like quite a pain in the ass. I think you were talking about this from the context of like computer vision, computer use, that kind of thing. So I mean, I've seen demos. I've seen some pretty interesting computer vision sort of demos that seem to be right around the corner. But like, what's the, what's the limiter on computer use for an AI? There was an interesting blog post by this company called Mechanized where they were explaining why this is such a big problem.
And I loved the way they phrased it, which is that imagine if you had to train a model in 1980, a large language model in 1980, and you could use all the compute you wanted in 1980 somehow. But you didn't have, you had, you were only stuck with the data that was available in the 1980s. Of course, before the internet became a widespread phenomenon. You couldn't train a model now, even with all the computing in the world because the data wasn't available. And we're in a similar position with respect to computer use because there's not this corpus of collected videos of people using computers to different things to access different applications and do white collar work.
Because of that, I think the big challenge has been accumulating this kind of data. And to be clear, when I, when I was saying the use case of like, like do my taxes, you're effectively talking about an AI having the ability to just like navigate the files around your computer, you know, log in to various websites to download your paystuffs or whatever, and then to go to like turbo tax or something and like input it all into some software and file it, right? Just on voice command or something like that. That's basically doing my taxes. It should be capable of navigating UIs that it's less familiar with or that come about organically within the context of a trying to solve a problem.
So for example, you know, I might have business deductions, a seize on my bank statement that I spent $1,000 on Amazon. It goes logs in my Amazon. It sees like, oh, was I bought a keyboard camera. So I think that's probably a business expense for his podcast. He bought an Airbnb over a weekend in the cabins of whatever in the woods of whatever. That probably wasn't a business expense. Although maybe if it's a sort of like a gray, it was willing to go in the gray area and be able to do the gray area of stuff. I was researching, but anyway, including all of that, including emailing people for invoices and haggling with them, it would be like a sort of weak long task to do my taxes, right?
You'd have to, there's a lot of work involved that's not just like, do this skill, this skill. But rather of having a sort of like plan of action and then breaking task apart, dealing with new information, new emails, new messages, consulting with me about questions, et cetera. Yeah. To be clear on this use case too, even though your post is titled like, you know, AI is not right around the corner. You still think this ability to file your taxes. That's like a 20, 20, 20, 20, 28 thing, right? I mean, this is maybe not next year, but it's in a few years. Right. I think that was sort of, people maybe write too much into this title and then didn't read through the arguments.
I mean, that never happens on the internet. Wow. First time. No, I think like, I am arguing against people who are like, you know, this will happen. AGI is like two years away. I do think the wider world, the markets, public perception, even people who are somewhat attending to AI. But aren't in this specific milieu that I'm talking to are way underpricing AGI. One reason, one thing I think they're underestimating is not only will we have millions of extra laborers, millions of extra workers, potentially billions within the course of the next decade, because then we will have potentially, I think likely we'll have AGI within the next decade.
But they'll have these advantages that human workers don't have, which is that a single commodal company, so suppose we solve continual learning, right? So they're, and we solve computer use. So as far as white color work goes, that might fundamentally be solved. You can have AIs which can use not just, they're not just like a text box where you put into, you ask questions in a chatbot and you get some response out. It's not that useful to just have a very smart chatbot. You need it to be able to actually do real work and use real applications. You have that solved because it acts like an employee.
It's got continual learning and it's got computer use, but it has another advantage that humans don't have, which is that copies of this model are going, being deployed all through the economy and it's doing on the job training. So copies are learning how to be an accountant, how to be a lawyer, how to be a coder, except because it's an AI in its digital, the model itself can amalgamate all this on the job training from all these copies. So what does that mean? What means that even if there's no more software progress after that point, which is to say that no more algorithms are discovered, there's not a transformer plus plus that's discovered.
Just from the fact that this model is learning every single skill in the economy, at least for a white collar work, you might just based on that alone have something that looks like an intelligence explosion, it would just be a broadly deployed intelligence explosion, but it will functionally become super intelligent just from having human level capability of learning on the job. So that creates this mesh network of intelligence that's shared among everyone, that's really fascinating. So we've kind of, we're going to get there, we're going to get to AGI, it's going to be incredibly smart. But what we've shared recently is just kind of this mixed bag where currently today, it's pretty good at some things, but also not that great at others.
We're hiring humans to do jobs that we think AI should do, but it probably doesn't. So the question I have for you is, is AI really that smart, or is it just good at kind of asing these particular benchmarks that we measure against? People, I mean, famously recently, they had the paper, the illusion of thinking where it was kind of like, hey, AI is like pretty good up to a point, but at a certain point, it just falls apart. And the inference is like, maybe it's not intelligence, maybe it's just good at guessing. So I guess the question is, is AI really that smart? It depends on who I'm talking to.
I think some people over hype, it's giving at least, I think some people are like, oh, it's already AGI, but it's like a little hobbled little AGI where we're like sort of giving at a concussion every couple of hours and like it forgets everything. We're like trapped in a chatbot context, but fundamentally the thing inside is like a very smart human. I just see that perspective. So if that's your perspective, I say like, no, it's not that smart. Your perspective is just a statistical association. I say, definitely smart, like it's like genuinely, there's an intelligence there. And the, so one thing you could say to the person who thinks that it's already AGI is this.
If a single human had as much stuff memorized as these models seem to have memorized, right? Which is to say that they have all of internet text, everything that human has written on the internet memorized, they would potentially be discovering all kinds of connections and discoveries. They'd notice that this thing which causes a migraine is associated with this kind of deficiency. So maybe we take the supplement, your migraines will be cured. There'd be just be that this list of just like trivial connections that lead to big discoveries all through the place. It's not clear that there's been an unambiguous case of an AI just doing this by itself.
So then why, so that's something potentially to explain, like if they're so intelligent, why aren't they able to use their disproportionate capabilities? There's unique capabilities to come up with these discoveries. I don't think there's actually a good answer to that question yet, except for the fact that they genuinely aren't that creative. Maybe they're like intelligent in a sense of knowing a lot of things, but they don't have this fluid intelligence that humans have. Anyway, so I give you a wish-washy answer because I think some people are underselling the intelligence, some people are overselling it.
I recall a tweet a lately from Todd account, and I think he's referring to maybe 03 and he basically said, it feels like AGI. I don't know if it is AGI or not, but to me it feels like AGI. What do you account for this feeling of intelligence then? I think this is actually very interesting because it gets to a crux that Tyler and I have. So Tyler and I disagree on two big things. One, he thinks, as you said in the blog post, 03 is AGI. I don't think it's AGI. I think it's orders of magnitude less valuable or many orders of magnitude less valuable unless useful in an AGI. That's one thing we disagree on.
The other thing we disagree on is he thinks that once we do get AGI, we'll only see 0.5% increase in the economic growth rate. This is like what the internet caused, right? Right. Whereas I think we will see tens of percent increase in economic growth. It will just be the difference between the pre-industrial revolution rate of growth versus industrial revolution. That magnitude of change again. And I think these two disagreements are linked because if you do believe we're already at AGI and you look around the world and you say, well, it fundamentally looks the same. I'd be forgiven for thinking, oh, there's not that much value in getting to AGI, whereas if you are like me and you think, no, we'll get this broadly at the minimum, at a very minimum, we'll get a broadly deployed intelligence explosion once you get to AGI, then you're like, okay, I'm just expecting some sort of singularitarian, crazy future with robot factories and solar farms all across the desert and things like that.
Yeah, I mean, it strikes me that your disagreement with Tyler is just based on the semantic definition of what AGI actually is. And Tyler, it sounds like he has kind of a lower threshold for what AGI is, whereas you have a higher threshold. Is there like an accepted definition for AGI? No. One thing that's useful for the purposes of discussions is to say automating all white color work because robotics hasn't made as much progress as a lens have or a computer use has. If we just say anything a human can do, or maybe 90% of what humans can do at a desk, any I can also do, that's potentially a useful definition for at least getting the cognitive elements relevant to defining AGI, but yeah, there's not one definition which suits all purposes.
Do we know it's like going on inside of these models, right? So like, you know, Josh was talking really in the conversation about like this at the base being sort of token prediction, right? And I guess this, this starts to raise the question of like, what is intelligence in the first place? And these A.I. models, I mean, they seem like they're intelligent. But do they have a model of the world the way maybe a human might or are they, are they sort of babbling or like, is this real reasoning and like, what is real reasoning? Do we just judge that based on the results or is there some way to like peek inside of its head?
If you used to have some other questions a couple of years ago, and then because honestly the things they did at the time were like ambiguous, you could say, oh, it's close enough to something else in its trading data set that is just basically copy pasting. It didn't come up with a solution by itself. But we've gone to the point where I can come up with a pretty complicated math problem and it will solve it. It can be a math problem like not like a, you know, undergrad or high school math problem. The problem, the problems the smartest math professors come up with in order to test international math Olympiad, you know, the kids who spend all their life preparing for this.
The geniuses who spend all their, all their young adulthood preparing to take these really gnarly math puzzle challenges. And the model will get these kinds of questions right. They require all these abstract creative thinking, this reasoning for hours, the model will get the right. So if that's not reasoning, then why is reasoning valuable again? Like what exactly was this reasoning supposed to be? So I think the genius are reasoning. I mean, I think there's other capabilities they lack, which are actually more, in some sense, they seem to us to be more trivial, but actually much harder to learn.
But the reasoning itself, I think, is there. And the answer to the intelligence question is also kind of clouded, right? Because we still really don't understand what's going on in an LLM. Dario from Anthropiche recently posted the paper about interpretation. And can you explain why we don't even really understand what's going on in these LLMs, even though we're able to make them and yield the results from them? Because it very much still is kind of like a black box. We write some code, we put some inputs in, and we get something out, but we're not sure what happens in the middle, why it's creating this output.
I mean, exactly what you're saying is that in other systems we engineer in the world, we have to build it up, bottom up. So if you build a bridge, you have to understand how every single beam is contributing to the structure, and we have equations for why the thing will stay standing. There's no such thing for AI. We didn't build it, more so we grew it. It's like watering a plant, and a couple thousand years ago, they were doing agriculture, so they didn't know why, you know, why do plants grow, how do they collect air, energy from sunlight, all these things, and I think we're in a substantially similar position with respect to intelligence, with respect to consciousness, with respect to all these other interesting questions about how mine to work, which is in some sense really cool because there's this huge intellectual horizon that's become not only available, but accessible to investigation.
In another sense, it's scary because we know that mines can suffer, we know that mines have moral worth, and we're creating mines, and we have no understanding of what's happening in these mines. It's a process of gradient descent, a painful process. We don't know, but we're doing a lot of it. So hopefully we'll learn more, but I think we're in a similar position to some farmer in Iraq and 3500 BC. Wow. And I mean, the potential, the idea that mines can suffer, mines have some moral worth, and also mines have some free will. They have some sort of autonomy, or maybe at least a desire to have autonomy.
Yeah. I mean, this brings us to kind of this sticky subject of alignment and AI safety and how we go about controlling the intelligence that we're creating. If even that's what we should be doing, controlling it, and we'll get to that in a minute. I want to start with maybe the headlines here a little bit. So headline just this morning, latest open AI models sabotaged a shutdown mechanism, despite command to the contrary, open AI's 01 model attempted to copy itself to external servers after being threatened with shutdown that denied the action when discovered. I've read a number of papers, but this of course mainstream media has these types of headlines almost on a weekly basis now, and it's starting to get to daily.
But there does seem to be some evidence that AI's lie to us, if that's even the right term, in order to pursue goals, goals like self-preservation, goals like replication, even deep-seated values that we might train into them, sort of a constitution type of value. They seek to preserve these values, which maybe that's a good thing, or maybe it's not a good thing if we don't actually want them to interpret the values in a certain way. Some of these headlines that we're seeing now, to you, with your kind of corpus of knowledge and all of the interviews and discovery done on your side, is this like media sensationalism, or is this like alarming?
And if it's alarming, how concerned should we be about this? I think on net, it's quite alarming. I do think that some of these results have been sort of cherry-picked, or if you look into the code at what's happened is basically the researchers have said, hey, pretend to be a bad person. Wow, AI's being a bad person, isn't that crazy? The system prompt is just like, hey, do this bad thing, right? Now, I, but I have also seen other results, which are not of this quality. I mean, the clearest example, so backing up, what is the reason to think this will be a bigger problem in the future than it is now?
Because we all interact with these systems, and they're actually quite moral or aligned, right? You can talk to a chatbot and you will ask it to, how should you deal with some crisis where there's a correct answer, it will tell you not to be violent, it'll give you reasonable advice, it seems to have good values. So it's worth noticing this, right? And being happy about it, the concern is that we're moving from a regime where we've trained them on human language, which implicitly has human morals and the way, you know, normal people think about values implicit in it, plus this RLHF process we did, to a regime where we're mostly spending compute on just having them answer problems, yes or no, or correct or not, rather, just like pass all the unit tests, get the right answer on this math problem.
And this has no guard reels intrinsically in terms of what is allowed to do, what is the proper moral way to do something. I think that can be a load of term, but like here's a more concrete example. One problem we're running into with these coding agents more and more, and there's nothing to do with these abstract concerns about alignment, but more so just like how do we get economic value out of these models is that Claude or Gemini will instead of writing code such that it passes the unit tests, it will often just delete the unit tests, so that the code just passes by default. Now, why would it do that?
Well, it's learned in the process, it was trained on the goal during training of you must pass all unit tests, and probably within some environment in which it was trained, it was able to just get away, like there wasn't design well enough, and so I found this little hole work, it just like delete the file that had the unit test or rewrite them so that it always said, you know, equals true, then pass. And right now we can discover these, even without, even though we can discover these, you know, it still pass, there's still enough at hacks like this such that the model is like becoming more and more hacky like that.
In the future, we're going to be training models in ways that we is beyond our ability to even understand, certainly beyond everybody's ability to understand, and maybe a few people who might be able to see just the way that right now if you come up with a new math proof for some open problem in mathematics, there will be only a few people in the world who will only evaluate that math proof. We'll be in a similar position with respect to all of the things that these models are being trained on at the frontier, especially math and code, because humans were big dumb-dums with respect to this reasoning stuff.
And so there's this sort of like first principles reason to expect that this new modality of training will be less amenable to the kinds of supervision that was grounded within the pre-training corpus. I don't know that everyone has kind of an intuition like an or an idea why it doesn't work to just say like, so if we don't want our AI models to lie to us, why can't we just tell them not to lie? Why can't we just put that as part of their core constitution? If we don't want our AI models to be sycophants, why can't we just say, hey, if I tell you I want the truth, not to flatter me, just like give me the straight-up truth, why is this even difficult to do?
Well, fundamentally it comes down to how we train them, and we don't know how to train them in a way that does not reward lying or sycophancy. In fact, the problem is open AI, they explained why the reason model of theirs was they had to take down was sycophantic, and the reason was just that they did it in the AB test, and the version, the test that was more sycophantic was just preferred by users more. Sometimes you prefer that. Yeah, so that's if that's what's preferred in training, or for example, in the context of lying, if we've just built our own environments in which we're training these models, where they're going to be more successful if they lie, right?
So if they delete the unit tests, and then tell you, I've passed this program, and all the unit tests are succeeded, it's like lying to you basically. And if that's what is rewarded in the process of gradient descent, then it's not surprising that the model you interact will just have this drive to lie if it gets it closer to its goal. And I would just expect this to keep happening unless we can solve this fundamental problem that comes about in training. In the Wild West of DeFi, Stability and Innovation are everything, which is why you should check out Fract Finance, the protocol revolutionizing stablecoins, DeFi, and Rolex.
The core of Fract's Finance is Fract USD, which is backed by BlackRock's institutional fiddle fund. Fract's designed Fract's USD for besting class yields across DeFi, T-bills, and carry trade returns all in one. Or your go-to decks, stake it, and help shape Fract's nation's future. Ready to join the forefront of DeFi? Visit Fract.com now to start earning with Fract's USD and staked Fract's USD. And for bankless listeners, you can use Fract.com slash r slash bankless when bridging to Fract's will for exclusive Fract's will perks and boosted rewards. Ever wonder how industry-leading companies like Coinbase, Metamask, and Phantomolid build products that always stay ahead of market shifts and new token listings?
It's because they all use the CoinGecko API, the most reliable and comprehensive crypto data API, backed by a rock-solid 99.9% uptime SLA. Builders, analysts, and businesses rely on CoinGecko for real-time data on 9 million cryptocurrencies across 1,500 exchanges and 200 different networks. It goes beyond crypto prices to access liquidity, OHLCV, DeX data, metadata, NFT floor prices, and more, enabling your team to build and scale powerful products like wallets, oracles, exchanges, AI agents, and DeFi applications without limitations. Onboarding is easy with clear documentation and enterprise grade support that answers you faster than finality, 24-7 around the clock.
Supercharge your product and build with CoinGecko API today. Head to gcko.io slash bankless 10 and enter code bankless 10 for 10% off any plan. CoinGecko API, crypto's most trusted and reliable data source. So you mentioned how like chat you BT out a version that was sick of phantic, and that's because users actually wanted that. Who isn't the control who decides the actual alignment of these models? Because users are saying one thing and then they deploy it and then it turns out that's not actually what people want. How do you kind of form consensus around this alignment? Are these alignment principles?
Right now, obviously, the labs who decided this, right? And the safety teams of the labs. Maybe, and I guess the question you could ask is then who should decide these? Because this will be, assuming the trajectory, so we keep going to get more powerful. Because this will be the key modality that all of us use to not only get worked on, but even like, I think at some point, a lot of people's best friends will be AI's. At least functionally, in a sense of who do they spend the most amount of time talking to? It might already be AI's. This will be the key layer in your business that you're using to get worked on.
So this process of training was shapes their personality, who gets to control it. I mean, it will be the labs functionally. But maybe you mean like who should control it, right? As you don't know, I mean, I don't know if there's a better alternative to the labs. Yeah, it would assume like there, there's some sort of social consensus, right? Similar to how we have in America, the constitution, there's like this general form of consensus that gets formed around how we should treat these models as they become as powerful as we think they probably will be. Honestly, I don't have, I don't know if anybody has a good answer about how you, how you do this process.
I think we lucked out, we just like really lucked out with the constitution. It also wasn't a democratic process which resulted in the constitution, even though it instituted a Republican form of government. It was just delegates from each state. They haggled it out over the course of a few months. Maybe that's what happens with the AI. But is there some process which feels both fair and which will result in actually a good constitution for these AI's? It's not obvious to me that, I mean, nothing comes up to the top of my head. Like, oh, this, you know, we're doing ranked choice voting or something.
Yeah, so it's going to ask, is there any, I mean, having spoken to everyone who you've spoken to, is there any alignment path which looks most promising, which feels the most comforting and exciting to you? I think alignment in the sense of eventually we'll have these super intelligent systems. What do we do about that? I think the approach that I think is most promising is less about finding some holy grail, some, you know, gigabrain solution, some equation which solves the whole puzzle. And more like one, having the Swiss cheese approach where, look, we for, we kind of have gotten really good at jail breaks.
You, I'm sure you heard a lot about jail breaks over the last few years. It's actually much harder to jailbreak these models because, you know, people try to whack, whack at these things in different ways. Model developers just like patched these obvious ways to do jail breaks. The model also got smarter, so it's better able to understand when it's somebody's trying to jailbreak into it. That I think is one approach. Another is, I think, competition. I think the scary version of the future is where you have this dynamic where a single model and its copies are controlling the entire economy. When politicians want to understand what policies to pass, they're only talking to copies of a single model.
If there's multiple different AI companies who are at the frontier who have competing services and whose models can monitor each other, right? So, Claude may care about its own copies being successful in the world and it might be able to willing to lie on their behalf even you ask one copy to supervise another. I think you get some advantage from a copy of Open AI's model, monitoring a copy of Deep Seeks model, which actually brings us back to the constitution, right? One of the most brilliant things in the constitution is the system of checks and balances. So, some combination of the Swiss cheese approach to model development and training and alignment where you're careful, if you notice this kind of reward hacking, you do your best to solve it.
You try to keep as much of the models thinking in human language rather than letting you think in AI thought in this lane space thinking. And the other part of it is just having normal market competition between these companies so that you can use them to check each other and no one company or no one AI is dominating the economy or advising rules for governments. I really like this bundle of ideas that you sort of put together in that because I think a lot of the AI safety conversation is always couch in terms of control. We have to control the thing that is the way and I always get little worried when I hear terms like control.
It reminds me of a blog post I think you put out, which I'm hopeful you continue to write on. I think you said it was going to be one of a series which is this idea of classical liberal AGI. We were talking about themes like balance of power. Let's have quad, check it with the chat GPT and monitor it. When you have themes like transparency as well, that feels a bit more classically liberal coded than maybe some of the other approaches that I've heard. You wrote this in the post which I thought was kind of, it just sparked my interest because I'm not sure where you're going to go next with this but you said the most likely way this happens that is AIs have a stake in humanity's future is if it's in the AI's best interest to operate within our existing laws and norms.
This whole idea that the way to get through AI alignment is to make it easy, make it the path of least resistance for AI to basically partner with humans. It's almost this idea of the aliens landed or something. We would create treaties with the aliens. We would want them to adopt our norms. We would want to initiate trade with them. Our first response shouldn't be, let's try to dominate and control them. Maybe it should be, let's try to work with them. Let's try to collaborate. Let's try to open up trade. What's your idea here? Are you planning to write further posts about this? Yeah, I want to.
It's just such a hard topic to think about that. Something always comes up. But the fundamental point I was making is, look, in the long run, if AI's or human labor is going to be obsolete because of these inherent advantages that digital minds will have and robotics will eventually be solved. So our only leverage on the future will not no longer come from our labor. It will come from our legal and economic control over the society that AI's will be participating in. AI's might make the economy explode in the sense of grow a lot. For humans to benefit from that, it would have to be the case that AI still respects your equity in the S&P 500 companies that you bought.
Or for the AI's to follow your laws, which say that you can't do violence onto humans and you had to respect humans' properties. It would have to be the case that AI's are actually bought into our system of government into our laws and norms. And for that to happen, the way that likely happens is if it's just like the default path for the AI's as they're getting smarter and they're developing their own systems of enforcement and laws to just participate in human laws and governments. And the metaphor I used here is right now you pay half your paycheck in taxes, probably half of your taxes, in some way just go to senior citizens, right?
Medicare and Social Security and other programs like this. And it's not because you're in some deep moral sense aligned with senior citizens. It's not like you're spending all your time thinking about like my main priority in life is to earn money for senior citizens. It's just that you're not going to overthrow the government to get out of paying this tax. And so also I happen to like my grandmother, she's fantastic, you know, it's those reasons too. But yes, that's why you give money to your grandmother directly. But like why are you giving money to some retiree dealers? Yes. Yeah, it's like, okay, you could say it's like sometimes people, some people just wanted that post by saying like, oh no, I like deeply care about the system of social welfare.
And it's like, okay, maybe you do, but I don't think like the average person is giving away hundreds of thousands of dollars a year, tens of thousands of dollars a year, to like some random stranger they don't know, who's like, who's not like especially in need of charity, right? Like most senior citizens have some savings. It's just it's just because this is a law and you like you give it to them or you'll get go to jail. But fundamentally, the tax was like 99%. You would like you would maybe you wouldn't overthrow the government, you just like leave the jurisdiction, you'd like emigrate somewhere.
And AI's can potentially also do this, right? There's more than one country they could like, there's countries which will be more AI forward and it would be a bad situation to end up in where all the break, all this explosion in AI technologies happening in the country, which is during the least amount to protect humans rights and to provide some sort of monetary compensation to humans once their labor is no longer valuable. So we could our labor could be worth nothing. But because of how much richer the world is after AI, you have these billions of extra researchers, workers, et cetera. We could it could still be trivial to have individual humans have the equivalent of millions, even billions of dollars worth of wealth.
In fact, it might literally be invaluable amounts of wealth in the following sense. So here's an interesting thought experiment. Imagine you have this choice. You can go back to the year 1500, but you know, of course, the year 1500 kind of sucks. You have no antibiotics, no TV, no running water. But here's how I'll make it up to you. I can give you any amount of money, but you can only use that amount of money in the year 1500. And you'll go back with these sacks of gold. How much money would I have to give you that you can use in the year 1500 to make you go back? And plus, the answer is there's no amount of money you would rather than the year 1500.
They just have a normal life today. And we could be in a similar position with regards to the future where there's all these different, I mean, you'll have much better health, like physical health, mental health, longevity. That's just like the thing we can contemplate now. But 15500 couldn't contemplate the kinds of quality of life advances we would have 500 years later, right? So anyways, this is all to say that this could be our future for humans, even if our labor isn't worth anything. But it does require us to have AIs that choose to participate or in some way incentivize to participate in some system which we have leveraged over.
Yeah, I find this just such a fast. I'm hopeful we do some more exploration around this because I think what you're calling for is basically like what you would be saying is invite them into our property right system. I mean, there are some that are calling in order to control AI, they have great power, but they don't necessarily have capability. So we shouldn't allow AI to hold money or to have property. I think you would say, no, actually the path forward to alignment is allow AI to have some vested interest in our property right system and some stake in our governance potentially, right? The ability to vote, almost like a constitution for AIs.
I'm not sure how this would work, but it's a fascinating thought experiment. I will say one thing. So I think this could end disastrously. If we give them a stake in our property system, but we let them play us off each other. So if you think about there's in many cases in history where the British initially the East Indian trading company was genuinely a trading company that operated in India and it was able to play off, you know, was like doing trade with different provinces in India. There was no single powerful leader. And by doing trade with one of them, leveraging one of their armies, et cetera, they were able to conquer the continent.
Similar thing could happen to human society. The way to avoid such an outcome at a high level is involves us playing the AIs off each other instead, right? So this is why I think competition is such a big part of the puzzle having different AIs monitor each other, having this bargaining position where there's not just one company that's the frontier. Another example here is if you think about how the Spanish conquered all these new world empires, it's actually so crazy that a couple hundred conquistadors will show up and conquer a nation of 10 million people, the Incas Aztecs. And why were they able to do this?
Well, one of the reasons is the Spanish were able to learn from each of their previous expeditions, whereas the Native Americans were not. So Cortez learned from how Cuba was subjugated when he conquered the Aztecs. Pizarro was able to learn from how Cortez conquered the Aztecs when he conquered the Incas. The Incas didn't even know the Aztecs existed. So eventually, there was this uprising against Pizarro and Mankoinka led an insurgency where they actually did figure out how to fight horses, how to fight people, you know, people in armor on horses, don't fight them on flat terrain, throw rocks down at them, etc.
But by this point, it was too late. If they knew this going into the battle, the initial battle, they might have been able to fend off because, you know, just as the conquistadors only arrived at the few hundred soldiers, we're going to the age of AI with a tremendous amount of leverage. We literally control all the stuff, right? But we just need to lock in our advantage. We just need to be in a position where, you know, they're not going to be able to place off each other. We're going to be able to learn what their weaknesses are. And this is why I think one good idea, for example, would be that look, Deepseek is a Chinese company.
It would be good if supposed Deepseek did something naughty, like the kinds of experiments we're talking about right now, where it hacks the unit tests or so forth. I mean, eventually, these things will really matter. Like, the Egyptian ping is listening to AIs because they're so smart and they're so capable. If China notices that their AIs are doing something bad, where they notice a field coup attempt, for example, it's very important that they tell us, and we tell them if we notice something like that on RN, it would be like the Aztecs and Incas talking to each other about like, you know, this is what happens, this is how you fight, this is how you fight horses.
This is the kind of tactics and deals to try to make with you, don't trust them, et cetera. It would require cooperation on humans part to have this sort of red telephone. So during the Cold War, there was this red telephone between America and the Soviet Union after the human missile crisis, where just to make sure there's no misunderstandings, they're like, okay, if we think something's going on, let's just hop on the call. I think we should have a similar policy with respect to these kinds of initial warning signs we'll get from AI, so that we can learn from each other. Awesome. Okay, so now that we've described this artificial gender intelligence, I want to talk about how we actually get there.
How do we build it? And a lot of this we've been discussing kind of takes place in this world of bits. We have this great chapter in the book called Inputs, which discusses the physical world around us, where you can't just write a few strings of code. You actually have to go and move some dirt and you have to ship server's places and you need to power it and you need physical energy from meat space and you kind of describe these these limiting factors where we have we have compute, we have energy, we have data. What I'm curious to know is, is do we have enough of this now or is there a clear path to get there in order to build the AI?
What basically what needs to happen in order for us to to get to this place that you're describing? We only have a couple more years left of this scaling, this exponential scaling before we're hitting these inherent roadblocks of energy and our ability to manufacture ships, which means that if scaling is going to work to deliver this AI, it has to work by 2028. Otherwise, we're just left with mostly algorithmic progress, but even within algorithmic progress, the sort of low hanging fruit in this deep learning paradigm is getting more and more plucked. So then the odds per year of getting to AGI diminish a lot, right?
So there is this weird funny thing happening right now where we either discover AGI within the next few years, or there's this or the yearly probability creators, and then we might be looking at decades of further research that's required in terms of algorithms to get to AGI. I am of the opinion that some algorithmic progress is necessarily needed because there's no easy way to solve continue learning just by making the context length bigger or despite being RL. That being said, I just think the progress so far has been so remarkable that 2032 is very close. My time has been slightly longer than that, but I think it's extremely plausible that we're going to see a broadly deployed intelligence explosion within the next 10 years.
One of these key inputs is energy, right? And I feel like one of the things that I've been hearing a lot, actually heard it mentioned on your podcast, is the United States relative to China on this particular place of energy? Where China is adding, what does this say? I think it's one United States worth of energy every 18 months, and their plan is to go from three to eight terawatts of power versus the United States, one to two terawatts of power by 2030. So given that context of that one resource alone, is China better equipped to get to that place versus the United States? So right now, America has a big advantage in terms of chips.
China doesn't have the ability to manufacture leading-edge semiconductors, and these are the chips that go into. These you need these dyes in order to have the kinds of AI chips to, you need millions of them in order to have a frontier AI system. Eventually China will catch up in this arena as well, right? Their technology will catch up. So the actual controls will keep us ahead in this category for five, ten years. But if we're looking in the world where a lot of timelines are long, which is to say that AGI isn't just right around the corner, they will have this overwhelming and energy advantage, and they'll have caught up in chips.
So then the question is like, why wouldn't they win at that point? So the longer you think we're away from AGI, the more it looks like China's game to lose. I mean, if you look in the nitty-gritty, I think it's more about having centralized sources of power because you need to train the AI in one place. This might be changing with RL, but it's very important to have a single site, which has a gigawatt to a gigawatt's more of power. And if we ramped up natural gas, you can get generators and natural gas, and maybe it's possible to do a last-to-chef route, even if our overall energy as a country is lower than China's.
The question is whether we will have the political will to do that. I think that people are sort of underestimating how much of a backlash there will be against AI. The government needs to make proactive efforts in order to make sure that America stays at the leading edge in AI, from zoning of data centers to how copyright is handled for data for these models. And if we mess up, if it becomes too hard to develop in America, I think it would genuinely be China's game to lose. Do you think the narrative is right that whoever wins the AGI war, whoever gets the AGI first, just basically wins the 21st century?
Is that simple? I don't think it's just a matter of training them front to your system. I think people underestimate how important it is to have the compute available to run these systems because eventually, once I get to AGI, just think of it like a person. And what matters then is how many people you have. I mean, it actually is the main thing that matters today as well, right? Why could China take over Taiwan if they wanted to? And if it didn't think America would intervene, but because Taiwan has 20 million people, or it's on the order of 20 million people, and China has 1.4 billion people, you could have a future where if China has way more compute than us, but equivalent levels of AI, it would be like the relationship between China and Taiwan.
The population is functionally so much higher. This just means more research, more factories, more development, more ideas. So this inference capacity, this capacity to deploy AIs will actually probably be the thing that determines who wins the 21st century. So this is like the scaling law applied to, I guess, nation-state geopolitics, right? And it's back to compute plus data wins. If compute plus data wins superintelligence, compute plus data also wins geopolitics. And the thing to be worried about is that China, I mean, speaking of compute plus data, China also has a lot more data on the real world, right?
If you've got entire mega-lopolises filled with factories where you're already deploying robots and different production systems which use automation, you have in-house this process knowledge which the AIs can then feed on and accelerate, that equivalent level of data we don't have in America. So this could be a period in which those technological advantages or those advantages in the physical world manufacturing could rapidly compound for China. And also, I mean, their big advantages as a civilization and society at least in recent decades has been that they can do big industrial projects fast and efficiently.
That's not what, that's not the first thing you think of when you think of in America. And AGI is a huge industrial high-cap ex-manhand project, right? And this is the kind of thing that China excels at and we don't. So you know, I think it's like a much tougher race than people anticipate. So what's all this going to do for the world? So once we get to the point of AGI, we've talked about GDP and your estimate is more on, it let less on the Tyler Cowan kind of, you know, half a percent per year and more on the, I guess, the Satya Nadella from Microsoft, but it was, you say, seven to eight percent.
Once we get to AGI, what about unemployment? Does this cause mass, I guess, you know, like, job loss across the economy or do people adopt? Like, what's your take here? And like, what, yeah, what are you seeing? Yeah, I mean, definitely will cause job loss. I think people who don't, I think a lot of AGI leaders try to gloss over that or something and like, I mean, what do you mean? Like, what does AGI mean if it doesn't cause job loss, right? If it does what a human does and does a cheaper and better and faster, like, why would it not cause job loss? The positive vision here is just that it creates so much wealth, so much abundance that we can still give people a much better standard of living.
Then even the wealthiest people today, even if they themselves don't have a job, the future I worry about is one where instead of creating some sort of UBI that will get, you know, exponentially bigger as society gets wealthier, we try to create these sorts of, um, guild like protection rackets where individual, you know, if, um, if the coders got unemployed, then we're going to give this like fake, we're going to make these bullshit jobs just for the coders and this is how we give them a redistribution or we try to expand Medicaid for AI, but it's not allowed to procure all of these advanced medicines and cures that AI is coming up with.
Rather than just giving people, you know, maybe lump sum of money or something. Um, so I am worried about the future where instead of sharing this abundance and just embracing it, we just have these protection rackets that maybe a lot of few people have access to the abundance of AI or maybe like if you sue AI, if you sue the right company at the right time, you will get a trillion dollars, but everybody else is stuck with nothing. I want to avoid that future and just like, um, be honest about what's coming and make programs that are simple and acknowledge how fast things will change and are forward looking rather than trying to turn what already exists into something amenable to, um, the displacement that AI will create.
That argument reminds me of, I don't know if you read the essay recently, I came out called the intelligence curse. Did you read that? It was, it was basically the idea of applying kind of the, you know, the, the nation state resource curse to the idea of intelligence, you know, it's like, um, nation states that are very high in natural resources. They just have a propensity, I mean, an example of is kind of like a Middle Eastern state with lots of oil reserves, right? Um, they have this rich source of a commodity type of abundance. They need their people less and so they don't invest in citizen rights.
They don't invest in social programs. The authors of the intelligence curse were saying that there's a similar type of curse that could happen once intelligence gets very cheap, which is basically like the nation state doesn't need humans anymore. And those at the top, the rich, the wealthy corporations, they don't need workers anymore. So we get kind of locked in this almost feudal state where, uh, you know, everyone has the property that their grandparents had and there's no meritocracy and sort of the nation states don't reinvest in citizens. Almost similar, similar ideas to your idea that like, um, you know, that the robots might want us just, or sorry, the AI's might just want us for our meat hands because they don't have the robotics technology on a temporary basis.
What do you think of this type of the like futures as possible? I agree that that is like definitely more of a concern given that humans will not be directly involved in the economic output that will be generated in the CIA civilization. The hopeful story you can tell is that a lot of these Middle Eastern resource, uh, you know, Dutch disease is another term that's used, uh, countries. The problem is that they're not democracies so that this wealth can just be extra, the system of government just lets whoever's in power extract that wealth for themselves. Whereas there are countries like Norway, for example, which also have abundant resources, um, who are able to use those resources to, uh, have further social welfare programs to build sovereign wealth funds for their citizens to invest in their future.
Um, we are going into at least some countries, uh, America included will go into the age of AI as a democracy. And so we are, of course, we'll lose our economic leverage, but the average person still has their political leverage. Now, we're the long run. Yeah, if we didn't do anything for a while, I'm guessing the political system would also change. Um, so then the key is to lock in our, uh, turn our current, well, it's not just political leverage, right? We also have property rights. We own a lot of stuff that AI wants, uh, factories, um, sources of data, et cetera. It's to use the combination of political and economic leverage to, uh, lock in benefits for us for the long term beyond our, um, the lifespan of our economic, uh, usefulness.
Um, and I'm more optimistic for us than I am for these Middle Eastern countries that started off poor and also with no democratic representation. What do you think the future of, uh, like, chat GPD is going to be like, if we just extrapolate maybe one version update for to chat GPT five, do you think the trend line of the scaling law will essentially hold for chat GPT five? I mean, another way to ask that question is do you feel like it'll feel like, um, you know, the difference between maybe a blackberry at an iPhone or will it feel like more like the difference between say the iPhone 10 and the iPhone 11, which is just like incremental progress, not a big breakthrough, not a, not a, um, order of magnitude change.
Yeah. I think it always somewhere in between, but I don't think I'll feel like a humongous breakthrough. Even though I think it's a remarkable pace of change, um, because the nature of scaling is that sometimes people talk about it as an exponential process and exponential usually refers to like it going like this. So having like a sort of j curve aspect to it where the incremental input is leading to super linear amounts of, uh, output in this case intelligence and value, where it's actually more like a sideways J. The exponential means the exponential in the scaling laws is that you need exponentially more inputs to get marginal increases in usefulness or loss or intelligence.
So and that's what we've been seeing, right? I think you, um, use initially see like some cool demo. So you, as you mentioned, you see some cool computer use demo, which comes at the beginning of this hyper exponential, I'm sorry, of this sort of, um, plateauing curve and then, um, it's still an incredibly powerful curve and restore early in it, but the next demo will be just adding on to making this existing capability more reliable, applicable for more skills. The other interesting incentive in this industry is that because of there's so much competition between the labs, you are incentivized to release a capability as soon as it's even marginally viable, um, or marginally cool.
So you can raise more funding or make more money off of it. You're not incentivized to just like sit on it until you perfected it, um, which is why I don't expect like tomorrow opening. I will just come out with like, we've still continue to learning guys and we didn't tell you about it. We're working on it for five years. Um, if they had like even an inkling of a solution, they'd want to release it a ASAP so they can raise a $600 billion round, um, and then spend more money on compute. Um, so yeah, I do think it'll seem marginal, but again, marginal in the context of seven years to HGI. So, um, zoom out long enough and a crazy amount of progress is happening.
Month to month, I think people over-hype how significant any one new release is. So I guess they answered so when we will get HGI very much depends on that scale line, that scaling trend holding. Um, your estimate in the book for HGI was 60% chance by 2040. So I'm curious what, what guess or what idea had the most influence on this estimate? What, what made you wind up on, on 60% to 2040? Because a lot of timelines are much faster than that. It's, it's sort of reasoning about the things that currently still lack, the capabilities they still lack and what stands in the way and just generally an intuition that things often take longer to happen than you might think.
Progress tends to slow down. Um, also, it's the case that look, you might have heard the phrase that we keep shifting, shifting the goal posts on AI, right? So they can do the things with skeptics we're saying they couldn't ever do already. But now they say we, AI still at that end because, uh, problem x, y, z, which will be solved next year. Um, now there's a way in which this is frustrating, but there's another way in which there's some validity to do this because, um, it is the case that we didn't get to AGI, even though we have past the touring test and we have models that are incredibly smart and can reason.
So it is accurate to say that, oh, we were wrong. And there is some missing thing that we need to keep identifying about what is still lacking to the path of AGI that it does make sense to shift the goal post. And I think we might discover once continual learning is solved or once accident computer use to solve that there were other aspects of human intelligence, which we take for granted in this more of expert ox sense, but which are actually quite crucial to making us economically valuable. Part of the reason we wanted to do this to our caches, because, um, we both are enjoyers of your podcast.
It's just fantastic. And you talk to all of the, you know, um, those that are on the forefront of AI development, uh, bleeding it in all sorts of ways. And I, one of the things I wanted to do with reading your book and obviously I'm always asking myself when I'm listening to your podcast is like, what does door cash think personally? And I feel like I sort of got that insight maybe toward the end of your book like, um, you know, in the summary section where you think like there's a 60% probability of AGI by 2040, which puts you more in the moderate camp, right? You're not a conservative, but you're not like an acceleration.
So your moderate there. And you also said, uh, you think more than likely AI will be net beneficial to humanity. So you're more optimist than doomer. So we got a moderate optimist. And you also think this and this is very interesting. There's no going back. So you're somewhat of an AI determinist. And I think the reason you state for not, you're like, there's no going back. Uh, it's struck me. There's this line in your book. It seems that the universe is structured such that throwing large amounts of compute at the right distribution of data gets you AI. And the secret is out. If the scaling picture is roughly correct, it's hard to imagine AGI not being developed this century, even if some actors hold back or held back.
That to me is an AI determinist position. Do you think that's fair? Moderate with respect to accelerationism, optimistic with respect to its potential and also determinist like there's nothing else we can do. We can't go backwards here. I'm determinist in the sense that I think if AI is technologically possible, it isn't inevitable. I think sometimes people are optimistic about this idea that we as a world will sort of collectively decide not to build AI. And I just don't think that's a plausible outcome. The local incentives for any actor to build AI are so high that it will happen. But I'm also an optimist in the sense that look, I'm not naive.
I've listed out all the way like what happened to the Aztecs and Incas was terrible. And I've explained how that could be similar to what AI could do to us and what we need to do to avoid that outcome. But I am optimistic in the sense that the world of the future fundamentally will have so much abundance that there is all these that alone is a prima facie reason to think that there must be some way of cooperating that is mutually beneficial. If we're going to be thousands, millions of times wealthier, is there really no way that humans are better off or can we can find a way for humans to become better off as a result of this transformation?
So yeah, I think you've put your finger on it. So the scaling book, of course, goes through the history of AI scaling. I think everyone should should pick it up to get the full chronology. But it also sort of captures where we are in the midst of this story is like we're not done yet. And I'm wondering how you feel at this moment of time. So I know if we're halfway through, if we're a quarter way through, if we're one tenth of the way through, but we're certainly not finished the path to AI scaling, how do you feel like in this moment in 2025? I mean, is all of this terrifying? Is it exciting?
Is it exhilarating? What's the what's the emotion that that you feel? Maybe I feel sort of heard. I personally feel like there's a lot of things I want to do in the meantime, including what my mission is with the podcast, which is to, and I know it's your mission as well, is to improve the discourse around these topics to not necessarily push versus a agenda, but make sure that when people are making decisions that are as well informed as possible, they have as much strategic awareness and depth of understanding around how AI works, what the what it could do in the future as possible. And but in many ways, I feel like I still haven't emotionally priced in the future I'm expecting.
In this one very basic sense, I think that there's a very good chance that I live beyond 200 years of age. I have not changed anything about my life with regards to that knowledge, right? I'm not like when I'm picking partners, I'm not like, oh, this is the person. Now that I think I'm going to live for 200, you know, like hundreds of years rather than yeah, well, ideally, I would pick a partner or that would ideally you pick somebody who would be that would be true of regardless. But usually what I'm saying, right? There's like the fact that I expect my personal life, the world around me, the lives of the people I care about, humanity in general to be so different, has it just like doesn't emotionally resonate as much as my intellectual thoughts and emotional landscape aren't in the same place.
I wonder if it's similar for you guys. Yeah, I totally agree. I don't think I've priced that in. Also, there's like a non-zero chance that Ellie Ezery Kowski is right to work as you know. And so that scenario, I just I can't bring myself too emotionally price in. So I have you're towards the optimism side as well. Dr. Keshe, this has been fantastic. Thank you so much for all you do on the podcast. I have to ask a question for a crypto audience as well, which is when are you going to do a crypto podcast on to our cash? I already did. It was with one Sam Bigman Fried. Oh my god. We got to get you a new guest.
We got to get to someone else to even talk about. Don't look at that one. I think in retrospect. You know what? We'll do another one. Fantastic. I'll ask you guys for some recommendations. That'd be great. It's our cash. But I've fallen your stuff for a while for I think many years. So it's great to finally meet and this is a lot of fun. Appreciate it. It was great. Thanks a lot.