Find shows
you love

The Andrej Karpathy episode.

During this interview, Andrej explains why reinforcement learning is terrible (but everything else is much worse), why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education.

It was a pleasure chatting with him.

Watch on YouTube; read the transcript.

Sponsors

* Labelbox helps you get data that is more detailed, more accurate, and higher signal than you could get by default, no matter your domain or training paradigm. Reach out today at labelbox.com/dwarkesh

* Mercury helps you run your business better. It’s the banking platform we use for the podcast — we love that we can see our accounts, cash flows, AR, and AP all in one place. Apply online in minutes at mercury.com

* Google’s Veo 3.1 update is a notable improvement to an already great model. Veo 3.1’s generations are more coherent and the audio is even higher-quality. If you have a Google AI Pro or Ultra plan, you can try it in Gemini today by visiting https://gemini.google

Timestamps

(00:00:00) – AGI is still a decade away

(00:29:45) – LLM cognitive deficits

(00:40:05) – RL is terrible

(00:49:38) – How do humans learn?

(01:06:25) – AGI will blend into 2% GDP growth

(01:17:36) – ASI

(01:32:50) – Evolution of intelligence & culture

(01:42:55) - Why self driving took so long

(01:56:20) - Future of education

In this lecture, military historian Sarah Paine explains how Russia—and specifically Stalin—completely derailed China’s rise, slowing them down for over a century.

This lecture was particularly interesting to me because, in my opinion, the Chinese Civil War is 1 of the top 3 most important events of the 20th century. And to understand why it transpired as it did, you need to understand Stalin’s role in the whole thing.

Watch on YouTube; read the transcript.

Sponsors

Mercury helps you run your business better. It’s the banking platform we use for the podcast — we love that we can see our cash balance, AR, and AP all in one place. Join us (and over 200,000 other entrepreneurs) at mercury.com

Labelbox scrutinizes public benchmarks at the single data-row level to probe what’s really being evaluated. Using this knowledge, they can generate custom training data for hill climbing existing benchmarks, or design new benchmarks from scratch. Learn more at labelbox.com/dwarkesh

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) – How Russia took advantage of China’s weakness

(00:22:58) – After Stalin, China’s rise

(00:33:52) – Russian imperialism

(00:45:23) – China’s and Russia’s existential problems

(01:04:55) – Q&A: Sino-Soviet Split

(01:22:44) – Stalin’s lessons from WW2

As part of this interview, Satya Nadella gave Dylan Patel (founder of SemiAnalysis) and me an exclusive first-look at their brand-new Fairwater 2 datacenter.

Microsoft is building multiple Fairwaters, each of which has hundreds of thousands of GB200s & GB300s. Between all these interconnected buildings, they’ll have over 2 GW of total capacity. Just to give a frame of reference, even a single one of these Fairwater buildings is more powerful than any other AI datacenter that currently exists.

Satya then answered a bunch of questions about how Microsoft is preparing for AGI across all layers of the stack.

Watch on YouTube; read the transcript.

Sponsors

* Labelbox produces high-quality data at massive scale, powering any capability you want your model to have. Whether you’re building a voice agent, a coding assistant, or a robotics model, Labelbox gets you the exact data you need, fast. Reach out at labelbox.com/dwarkesh

* CodeRabbit automatically reviews and summarizes PRs so you can understand changes and catch bugs in half the time. This is helpful whether you’re coding solo, collaborating with agents, or leading a full team. To learn how CodeRabbit integrates directly into your workflow, go to coderabbit.ai

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) - Fairwater 2

(00:03:20) - Business models for AGI

(00:12:48) - Copilot

(00:20:02) - Whose margins will expand most?

(00:36:17) - MAI

(00:47:47) - The hyperscale business

(01:02:44) - In-house chip & OpenAI partnership

(01:09:35) - The CAPEX explosion

(01:15:07) - Will the world trust US companies to lead AI?

Adam Marblestone is CEO of Convergent Research. He’s had a very interesting past life: he was a research scientist at Google Deepmind on their neuroscience team and has worked on everything from brain-computer interfaces to quantum computing to nanotech and even formal mathematics.

In this episode, we discuss how the brain learns so much from so little, what the AI field can learn from neuroscience, and the answer to Ilya’s question: how does the genome encode abstract reward functions? Turns out, they’re all the same question.

Watch on YouTube; read the transcript.

Sponsors

* Gemini 3 Pro recently helped me run an experiment to test multi-agent scaling: basically, if you have a fixed budget of compute, what is the optimal way to split it up across agents? Gemini was my colleague throughout the process — honestly, I couldn’t have investigated this question without it. Try Gemini 3 Pro today gemini.google.com

* Labelbox helps you train agents to do economically-valuable, real-world tasks. Labelbox’s network of subject-matter experts ensures you get hyper-realistic RL environments, and their custom tooling lets you generate the highest-quality training data possible from those environments. Learn more at labelbox.com/dwarkesh

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) – The brain’s secret sauce is the reward functions, not the architecture

(00:22:20) – Amortized inference and what the genome actually stores

(00:42:42) – Model-based vs model-free RL in the brain

(00:50:31) – Is biological hardware a limitation or an advantage?

(01:03:59) – Why a map of the human brain is important

(01:23:28) – What value will automating math have?

(01:38:18) – Architecture of the brain

Further reading

Intro to Brain-Like-AGI Safety - Steven Byrnes’s theory of the learning vs steering subsystem; referenced throughout the episode.

A Brief History of Intelligence - Great book by Max Bennett on connections between neuroscience and AI

Adam’s blog, and Convergent Research’s blog on essential technologies.

A Tutorial on Energy-Based Learning by Yann LeCun

What Does It Mean to Understand a Neural Network? - Kording & Lillicrap

E11 Bio and their brain connectomics approach

Sam Gershman on what dopamine is doing in the brain

Gwern’s proposal on training models on the brain’s hidden states

Kettle Chips: Cameron Healy. The Wild Bet That Made a Brand

Most founders expand the “right” way: local → regional → national → international.

Cameron Healy totally skipped the “national” part. 

When Kettle Chips was still an upstart regional brand, Cameron made a move that seems almost reckless: he launched his thick-cut, kettle-cooked chips to the United Kingdom — one of the most competitive “crisps” markets on earth — before conquering the U.S.

And that wasn’t his first risky move. 

Before Kettle, Cameron was a turban-wearing Sikh entrepreneur in 1970s Salem, Oregon, building a natural foods business…until he was abruptly fired. He started again from scratch with a $10,000 bank loan.  Inspired by the extra thick, crunchy potato chips that he sampled on a trip to Hawaii, he taught himself how to fry sliced potatoes through trial-and-error.  

Then, just as Kettle started taking off overseas, another trip to Hawaii sparked a second act: Kona Brewing — a craft beer brand that initially lost $20K a month — for years — before Cameron was able to make it work.

Meanwhile, buoyed by its UK success, Kettle chips eventually spread across the US, becoming the top-selling natural chip in the country. 

What you’ll learn

• The hidden details (like cooking-oil quality control) that can make or break a chip
• How curiosity about British “crisp” culture fueled a risky UK rollout
• The decision that turned Kona Brewing from a money pit into a scalable brand

Timestamps

• 07:21 — “You had to get up at 3 a.m.”: building a life in a Sikh community in Salem
• 10:11 — Fired with four kids and no severance: the moment Cameron is forced to rebuild
• 12:04 — The $10K loan (helped along by the offer of ski passes)
• 14:06 — The 1980 peanut crop gamble that suddenly capitalized Cameron’s business
• 23:14 — “Pot Chips” was the original name…until friends told him how bad it was
• 24:48 — Hand-feeding potatoes into vats of oil: inventing a process with zero playbook
• 29:10 — The Safeway disaster: rancid oil, a rejected order, and demand evaporating overnight
• 31:52 — The car crash that jolted Cameron out of despair
• 46:35 — UK word-of-mouth “switches on”--with an extra boost from Lady Di
• 56:03 — Kona Brewing bleeds money…until one decision turns things around

***

Hey—want to be a guest on HIBT?

If you’re building a business, why not get advice from some of the greatest entrepreneurs on Earth?

Every Thursday on the HIBT Advice Line, a previous HIBT guest helps new entrepreneurs work through the challenges they’re facing right now. Advice that’s smart, actionable, and absolutely free.

Just call 1-800-433-1298, leave a message, and you may soon get guidance from someone who started where you did, and went on to build something massive.

So—give us a call. We can’t wait to hear what you’re working on.

***

This episode was produced by Casey Herman with music composed by Ramtin Arablouei. It was edited by Neva Grant with research help from Rommel Wood. Our engineers were Robert Rodriguez and Kwesi Lee.

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

This is March. Zac completed his 36 hour stream, we talk a big College Basketball Saturday slate and we're 2 weeks from brackets. Clean up from the NFL combine and other national sports podcast talk (00:00:00-00:45:40). Who's back of the week including a completely meaningless Sixers/Celtics game that has Max and Hank going to war (00:45:40-00:59:50). Seattle Seahawks Head Coach Mike Macdonald joins the show to talk about winning the Super Bowl, his path in coaching, message to his team next year to try and repeat, when he knew Sam Darnold was the guy, and tons more (00:59:50-01:49:16). We finish the show talking about our 10 year anniversary and some funny memories and gratitude for the best job ever (01:49:16-02:20:42).

You can find every episode of this show on Apple Podcasts, Spotify or Netflix. Prime Members can listen ad-free on Amazon Music. For more, visit barstool.link/pardon-my-take