Dwarkesh Podcast

Adam Marblestone is CEO of Convergent Research. He’s had a very interesting past life: he was a research scientist at Google Deepmind on their neuroscience team and has worked on everything from brain-computer interfaces to quantum computing to nanotech and even formal mathematics.

In this episode, we discuss how the brain learns so much from so little, what the AI field can learn from neuroscience, and the answer to Ilya’s question: how does the genome encode abstract reward functions? Turns out, they’re all the same question.

Watch on YouTube; read the transcript.

Sponsors

* Gemini 3 Pro recently helped me run an experiment to test multi-agent scaling: basically, if you have a fixed budget of compute, what is the optimal way to split it up across agents? Gemini was my colleague throughout the process — honestly, I couldn’t have investigated this question without it. Try Gemini 3 Pro today gemini.google.com

* Labelbox helps you train agents to do economically-valuable, real-world tasks. Labelbox’s network of subject-matter experts ensures you get hyper-realistic RL environments, and their custom tooling lets you generate the highest-quality training data possible from those environments. Learn more at labelbox.com/dwarkesh

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) – The brain’s secret sauce is the reward functions, not the architecture

(00:22:20) – Amortized inference and what the genome actually stores

(00:42:42) – Model-based vs model-free RL in the brain

(00:50:31) – Is biological hardware a limitation or an advantage?

(01:03:59) – Why a map of the human brain is important

(01:23:28) – What value will automating math have?

(01:38:18) – Architecture of the brain

Further reading

Intro to Brain-Like-AGI Safety - Steven Byrnes’s theory of the learning vs steering subsystem; referenced throughout the episode.

A Brief History of Intelligence - Great book by Max Bennett on connections between neuroscience and AI

Adam’s blog, and Convergent Research’s blog on essential technologies.

A Tutorial on Energy-Based Learning by Yann LeCun

What Does It Mean to Understand a Neural Network? - Kording & Lillicrap

E11 Bio and their brain connectomics approach

Sam Gershman on what dopamine is doing in the brain

Gwern’s proposal on training models on the brain’s hidden states

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Read the essay here.

Timestamps

00:00:00 What are we scaling?

00:03:11 The value of human labor

00:05:04 Economic diffusion lag is cope00:06:34 Goal-post shifting is justified

00:08:23 RL scaling

00:09:18 Broadly deployed intelligence explosion

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

This is the final episode of the Sarah Paine lecture series, and it’s probably my favorite one. Sarah gives a “tour of the arguments” on what ultimately led to the Soviet Union’s collapse, diving into the role of the US, the Sino-Soviet border conflict, the oil bust, ethnic rebellions and even the Roman Catholic Church. As she points out, this is all particularly interesting as we find ourselves potentially at the beginning of another Cold War.

As we wrap up this lecture series, I want to take a moment to thank Sarah for doing this with me. It has been such a pleasure.

If you want more of her scholarship, I highly recommend checking out the books she’s written. You can find them here.

Watch on YouTube; read the transcript.

Sponsors

* Labelbox can get you the training data you need, no matter the domain. Their Alignerr network includes the STEM PhDs and coding experts you’d expect, but it also has experienced cinematographers and talented voice actors to help train frontier video and audio models. Learn more at labelbox.com/dwarkesh.

* Sardine doesn’t just assess customer risk for banking & retail. Their AI risk management platform is also extremely good at detecting fraudulent job applications, which I’ve found useful for my own hiring process. If you need help with hiring risk—or any other type of fraud prevention—go to sardine.ai/dwarkesh.

* Gemini’s Nano Banana Pro helped us make many of the visuals in this episode. For example, we used it to turn dense tables into clear charts so that’d it be easier to quickly understand the trends that Sarah discusses. You can try Nano Banana Pro now in the Gemini app. Go to gemini.google.com.

Timestamps

(00:00:00) – Did Reagan single-handedly win the Cold War?

(00:15:53) – Eastern Bloc uprisings & oil crisis

(00:30:37) – Gorbachev’s mistakes

(00:37:33) – German unification and NATO expansion

(00:48:31) – The Gulf War and the Cold War endgame

(00:56:10) – How central planning survived so long

(01:14:46) – Sarah’s life in the USSR in 1988

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Ilya & I discuss SSI’s strategy, the problems with pre-training, how to improve the generalization of AI models, and how to ensure AGI goes well.

Watch on YouTube; read the transcript.

Sponsors

* Gemini 3 is the first model I’ve used that can find connections I haven’t anticipated. I recently wrote a blog post on RL’s information efficiency, and Gemini 3 helped me think it all through. It also generated the relevant charts and ran toy ML experiments for me with zero bugs. Try Gemini 3 today at gemini.google

* Labelbox helped me create a tool to transcribe our episodes! I’ve struggled with transcription in the past because I don’t just want verbatim transcripts, I want transcripts reworded to read like essays. Labelbox helped me generate the exact data I needed for this. If you want to learn how Labelbox can help you (or if you want to try out the transcriber tool yourself), go to labelbox.com/dwarkesh

* Sardine is an AI risk management platform that brings together thousands of device, behavior, and identity signals to help you assess a user’s risk of fraud & abuse. Sardine also offers a suite of agents to automate investigations so that as fraudsters use AI to scale their attacks, you can use AI to scale your defenses. Learn more at sardine.ai/dwarkesh

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) – Explaining model jaggedness

(00:09:39) - Emotions and value functions

(00:18:49) – What are we scaling?

(00:25:13) – Why humans generalize better than models

(00:35:45) – SSI’s plan to straight-shot superintelligence

(00:46:47) – SSI’s model will learn from deployment

(00:55:07) – How to think about powerful AGIs

(01:18:13) – “We are squarely an age of research company”

(01:20:23) – Self-play and multi-agent

(01:32:42) – Research taste

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

As part of this interview, Satya Nadella gave Dylan Patel (founder of SemiAnalysis) and me an exclusive first-look at their brand-new Fairwater 2 datacenter.

Microsoft is building multiple Fairwaters, each of which has hundreds of thousands of GB200s & GB300s. Between all these interconnected buildings, they’ll have over 2 GW of total capacity. Just to give a frame of reference, even a single one of these Fairwater buildings is more powerful than any other AI datacenter that currently exists.

Satya then answered a bunch of questions about how Microsoft is preparing for AGI across all layers of the stack.

Watch on YouTube; read the transcript.

Sponsors

* Labelbox produces high-quality data at massive scale, powering any capability you want your model to have. Whether you’re building a voice agent, a coding assistant, or a robotics model, Labelbox gets you the exact data you need, fast. Reach out at labelbox.com/dwarkesh

* CodeRabbit automatically reviews and summarizes PRs so you can understand changes and catch bugs in half the time. This is helpful whether you’re coding solo, collaborating with agents, or leading a full team. To learn how CodeRabbit integrates directly into your workflow, go to coderabbit.ai

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) - Fairwater 2

(00:03:20) - Business models for AGI

(00:12:48) - Copilot

(00:20:02) - Whose margins will expand most?

(00:36:17) - MAI

(00:47:47) - The hyperscale business

(01:02:44) - In-house chip & OpenAI partnership

(01:09:35) - The CAPEX explosion

(01:15:07) - Will the world trust US companies to lead AI?

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

In this lecture, military historian Sarah Paine explains how Russia—and specifically Stalin—completely derailed China’s rise, slowing them down for over a century.

This lecture was particularly interesting to me because, in my opinion, the Chinese Civil War is 1 of the top 3 most important events of the 20th century. And to understand why it transpired as it did, you need to understand Stalin’s role in the whole thing.

Watch on YouTube; read the transcript.

Sponsors

Mercury helps you run your business better. It’s the banking platform we use for the podcast — we love that we can see our cash balance, AR, and AP all in one place. Join us (and over 200,000 other entrepreneurs) at mercury.com

Labelbox scrutinizes public benchmarks at the single data-row level to probe what’s really being evaluated. Using this knowledge, they can generate custom training data for hill climbing existing benchmarks, or design new benchmarks from scratch. Learn more at labelbox.com/dwarkesh

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) – How Russia took advantage of China’s weakness

(00:22:58) – After Stalin, China’s rise

(00:33:52) – Russian imperialism

(00:45:23) – China’s and Russia’s existential problems

(01:04:55) – Q&A: Sino-Soviet Split

(01:22:44) – Stalin’s lessons from WW2

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

The Andrej Karpathy episode.

During this interview, Andrej explains why reinforcement learning is terrible (but everything else is much worse), why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education.

It was a pleasure chatting with him.

Watch on YouTube; read the transcript.

Sponsors

* Labelbox helps you get data that is more detailed, more accurate, and higher signal than you could get by default, no matter your domain or training paradigm. Reach out today at labelbox.com/dwarkesh

* Mercury helps you run your business better. It’s the banking platform we use for the podcast — we love that we can see our accounts, cash flows, AR, and AP all in one place. Apply online in minutes at mercury.com

* Google’s Veo 3.1 update is a notable improvement to an already great model. Veo 3.1’s generations are more coherent and the audio is even higher-quality. If you have a Google AI Pro or Ultra plan, you can try it in Gemini today by visiting https://gemini.google

Timestamps

(00:00:00) – AGI is still a decade away

(00:29:45) – LLM cognitive deficits

(00:40:05) – RL is terrible

(00:49:38) – How do humans learn?

(01:06:25) – AGI will blend into 2% GDP growth

(01:17:36) – ASI

(01:32:50) – Evolution of intelligence & culture

(01:42:55) - Why self driving took so long

(01:56:20) - Future of education

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Nick Lane has some pretty wild ideas about the evolution of life.

He thinks early life was continuous with the spontaneous chemistry of undersea hydrothermal vents.

Nick’s story may be wrong, but I find it remarkable that with just that starting point, you can explain so much about why life is the way that it is — the things you’re supposed to just take as givens in biology class:

* Why are there two sexes? Why sex at all?

* Why are bacteria so simple despite being around for 4 billion years? Why is there so much shared structure between all eukaryotic cells despite the enormous morphological variety between animals, plants, fungi, and protists?

* Why did the endosymbiosis event that led to eukaryotes happen only once, and in the particular way that it did?

* Why is all life powered by proton gradients? Why does all life on Earth share not only the Krebs Cycle, but even the intermediate molecules like Acetyl-CoA?

His theory implies that early life is almost chemically inevitable (potentially blooming on hundreds of millions of planets in the Milky Way alone), and that the real bottleneck is the complex eukaryotic cell.

Watch on YouTube; listen on Apple Podcasts or Spotify.

Sponsors

* Gemini in Sheets lets you turn messy text into structured data. We used it to classify all our episodes by type and topic, no manual tagging required. If you’re a Google Workspace user, you can get started today at docs.google.com/spreadsheets/

* Labelbox has a massive network of domain experts (called Alignerrs) who help train AI models in a way that ensures they understand the world deeply, not superficially. These Alignerrs are true experts — one even tutored me in chemistry as I prepped for this episode. Learn more at labelbox.com/dwarkesh

* Lighthouse helps frontier technology companies like Cursor and Physical Intelligence navigate the U.S. immigration system and hire top talent from around the world. Lighthouse handles everything, maximizing the probability of visa approval while minimizing the work you have to do. Learn more at lighthousehq.com/employers

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) – The singularity that unlocked complex life

(00:08:26) – Early life continuous with Earth's geochemistry

(00:23:36) – Eukaryotes are the great filter for intelligent life

(00:42:16) – Mitochondria are the reason we have sex

(01:08:12) – Are bioelectric fields linked to consciousness?

Ref: 868329

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

I have a much better understanding of Sutton’s perspective now. I wanted to reflect on it a bit.

(00:00:00) - The steelman

(00:02:42) - TLDR of my current thoughts

(00:03:22) - Imitation learning is continuous with and complementary to RL

(00:08:26) - Continual learning

(00:10:31) - Concluding thoughts

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Richard Sutton is the father of reinforcement learning, winner of the 2024 Turing Award, and author of The Bitter Lesson. And he thinks LLMs are a dead end.

After interviewing him, my steel man of Richard’s position is this: LLMs aren’t capable of learning on-the-job, so no matter how much we scale, we’ll need some new architecture to enable continual learning.

And once we have it, we won’t need a special training phase — the agent will just learn on-the-fly, like all humans, and indeed, like all animals.

This new paradigm will render our current approach with LLMs obsolete.

In our interview, I did my best to represent the view that LLMs might function as the foundation on which experiential learning can happen… Some sparks flew.

A big thanks to the Alberta Machine Intelligence Institute for inviting me up to Edmonton and for letting me use their studio and equipment.

Enjoy!

Watch on YouTube; listen on Apple Podcasts or Spotify.

Sponsors

* Labelbox makes it possible to train AI agents in hyperrealistic RL environments. With an experienced team of applied researchers and a massive network of subject-matter experts, Labelbox ensures your training reflects important, real-world nuance. Turn your demo projects into working systems at labelbox.com/dwarkesh

* Gemini Deep Research is designed for thorough exploration of hard topics. For this episode, it helped me trace reinforcement learning from early policy gradients up to current-day methods, combining clear explanations with curated examples. Try it out yourself at gemini.google.com

* Hudson River Trading doesn’t silo their teams. Instead, HRT researchers openly trade ideas and share strategy code in a mono-repo. This means you’re able to learn at incredible speed and your contributions have impact across the entire firm. Find open roles at hudsonrivertrading.com/dwarkesh

Timestamps

(00:00:00) – Are LLMs a dead end?

(00:13:04) – Do humans do imitation learning?

(00:23:10) – The Era of Experience

(00:33:39) – Current architectures generalize poorly out of distribution

(00:41:29) – Surprises in the AI field

(00:46:41) – Will The Bitter Lesson still apply post AGI?

(00:53:48) – Succession to AIs

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Sergey Levine, one of the world’s top robotics researchers and co-founder of Physical Intelligence, thinks we’re on the cusp of a “self-improvement flywheel” for general-purpose robots. His median estimate for when robots will be able to run households entirely autonomously? 2030.

If Sergey’s right, the world 5 years from now will be an insanely different place than it is today. This conversation focuses on understanding how we get there: we dive into foundation models for robotics, and how we scale both the data and the hardware necessary to enable a full-blown robotics explosion.

Watch on YouTube; listen on Apple Podcasts or Spotify.

Sponsors

* Labelbox provides high-quality robotics training data across a wide range of platforms and tasks. From simple object handling to complex workflows, Labelbox can get you the data you need to scale your robotics research. Learn more at labelbox.com/dwarkesh

* Hudson River Trading uses cutting-edge ML and terabytes of historical market data to predict future prices. I got to try my hand at this fascinating prediction problem with help from one of HRT’s senior researchers. If you’re curious about how it all works, go to hudson-trading.com/dwarkesh

* Gemini 2.5 Flash Image (aka nano banana) isn’t just for generating fun images — it’s also a powerful tool for restoring old photos and digitizing documents. Test it yourself in the Gemini App or in Google’s AI Studio: ai.studio/banana

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) – Timeline to widely deployed autonomous robots

(00:17:25) – Why robotics will scale faster than self-driving cars

(00:27:28) – How vision-language-action models work

(00:45:37) – Changes needed for brainlike efficiency in robots

(00:57:59) – Learning from simulation

(01:09:18) – How much will robots speed up AI buildouts?

(01:18:01) – If hardware’s the bottleneck, does China win by default?

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

In this lecture, military historian Sarah Paine explains how Britain used sea control, peripheral campaigns, and alliances to defeat Nazi Germany during WWII. She then applies this framework to today, arguing that Russia and China are similarly constrained by their geography, making them vulnerable in any conflict with maritime powers (like the U.S. and its allies).

Watch on YouTube; listen on Apple Podcasts or Spotify.

Sponsors

* Labelbox partners with researchers to scope, generate, and deliver the exact data frontier models need, no matter the domain. Whether that’s multi-turn audio, SOTA robotics data, advanced STEM problem sets, or even novel RL environments, Labelbox delivers high-quality data, fast. Learn more at labelbox.com/dwarkesh

* Warp is the best interface I’ve found for coding with agents. It makes building custom tools easy: Warp’s UI helps you understand agent behavior and its in-line text editor is great for making tweaks. You can try Warp for free, or, for a limited time, use code DWARKESH to get Warp’s Pro Plan for only $5. Go to warp.dev/dwarkesh

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

00:00:00 – How WW1 shaped WW2

00:15:10 – Hitler and Churchill’s battle to command the Atlantic

00:30:10 – Peripheral theaters leading up to Normandy

00:37:13 – The Eastern front

00:48:04 – Russia’s & China’s geographic prisons

01:00:28 – Hitler’s blunders & America’s industrial might

01:15:03 – Bismarck’s limited wars vs Hitler’s total war

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Jacob Kimmel thinks he can find the transcription factors to reverse aging. We do a deep dive on why this might be plausible and why evolution hasn’t optimized for longevity. We also talk about why drug discovery has been getting exponentially harder, and what a new platform for biological understanding to speed up progress would look like. As a bonus, we get into the nitty gritty of gene delivery and Jacob’s controversial takes on CAR-T cells. For full disclosure, I am an angel investor in NewLimit. This did not impact my decision to interview Jacob, nor the questions I asked him.

Watch on YouTube; listen on Apple Podcasts or Spotify.

SPONSORS

* Hudson River Trading uses deep learning to tackle one of the world's most complex systems: global capital allocation. They have a massive in-house GPU cluster, and they’re constantly adding new racks of B200s to ensure their researchers are never constrained by compute. Explore opportunities at hudsonrivertrading.com/dwarkesh\

* Google’s Gemini CLI turns ideas into working applications FAST, no coding required. It built a complete podcast post-production tool in 10 minutes, including fully functional backend logic, and the entire build used less than 10% of Gemini’s session context. Check it out on Github now!

* To sponsor a future episode, visit dwarkesh.com/advertise.

TIMESTAMPS

(00:00:00) – Three reasons evolution didn’t optimize for longevity

(00:12:07) – Why didn't humans evolve their own antibiotics?

(00:25:26) – De-aging cells via epigenetic reprogramming

(00:44:43) – Viral vectors and other delivery mechanisms

(01:06:22) – Synthetic transcription factors

(01:09:31) – Can virtual cells break Eroom’s Law?

(01:31:32) – Economic models for pharma

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

How will we feed the 100s of GWs of extra energy demand that AI will create over the coming decade? On this episode, Casey Handmer (Caltech PhD, former NASA JPL, founder & CEO of Terraform Industries) walks me through how we can pull it off, and why he thinks a major part of this energy singularity will be powered by solar. His views are contrarian, but he came armed to defend them.

Watch on YouTube; listen on Apple Podcasts or Spotify.

SPONSORS

- Lighthouse helps frontier technology companies like Cursor and Physical Intelligence navigate the U.S. immigration system and hire top talent from around the world. Lighthouse handles everything for you, maximizing the probability of visa approval while minimizing the work you have to do. Learn more at lighthousehq.com/employers

- To sponsor a future episode, visit dwarkesh.com/advertise.

TIMESTAMPS

(00:00:00) – Why doesn’t China win by default?

(00:08:28) – Why hyperscalers choose natural gas over solar

(00:18:01) – Solar's astonishing learning rates

(00:27:02) – How to build 50,000 acre solar-powered data centers

(00:40:24) – Environmental regulations blocking clean energy

(00:44:04) – Batteries replacing the grid

(00:49:14) – GDP is broken, AGI's true value must be measured in total energy use

(00:58:45) – Silicon wafers in space with one mind each

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

A deep dive with Lewis Bollard, who leads Open Philanthropy’s strategy for Farmed Animal Welfare, on the surprising economics of the meat industry.

Why is factory farming so efficient? How can we make the lives of the 23+ billion animals living on factory farms more bearable? How far off are the moonshots (e.g., brainless chickens, cultivated meats, etc.) to end this mass suffering? And why does the meat industry have such a surprising amount of political influence?

For decades, innovation in the meat industry has actually made the conditions for animals worse. Can the next few decades of tech reverse this pattern?

Watch on YouTube; listen on Apple Podcasts or Spotify.

Donation match fundraiser

The welfare of animals on factory farms is so systemically neglected that just $1 can help avert 10 years of animal suffering.

After learning more about the outsized opportunities to help, I decided to give $250,000 as a donation match to farmkind.giving/dwarkesh. FarmKind directs your contributions to the most effective charities in this area.

Please consider contributing, even if it’s a small amount. Together, we can double each other's impact and give a total of $500,000.

Bluntly, there are some listeners who are in a position to give much more. Given how neglected this topic is, one such person could singlehandedly change the game for 10s of billions of animals. If you’re considering donating $50k or more, please reach out directly to Lewis and his team by emailing andres@openphilanthropy.org.

Timestamps

(00:00:00) – The astonishing efficiency of factory farming

(00:07:18) – It was a mistake making this about diet

(00:09:54) – Tech that’s sparing 100s of millions of animals/year

(00:16:16) – Brainless chickens and higher welfare breeds

(00:28:21) – $1 can prevent 10 years of animal suffering

(00:37:26) – Situation in China and the developing world

(00:41:41) – How the meat lobby got a lock on Congress

(00:53:23) – Business structure of the meat industry

(00:57:42) – Corporate campaigns are underrated

Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe