Find shows
you love

The Andrej Karpathy episode.

During this interview, Andrej explains why reinforcement learning is terrible (but everything else is much worse), why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education.

It was a pleasure chatting with him.

Watch on YouTube; read the transcript.

Sponsors

* Labelbox helps you get data that is more detailed, more accurate, and higher signal than you could get by default, no matter your domain or training paradigm. Reach out today at labelbox.com/dwarkesh

* Mercury helps you run your business better. It’s the banking platform we use for the podcast — we love that we can see our accounts, cash flows, AR, and AP all in one place. Apply online in minutes at mercury.com

* Google’s Veo 3.1 update is a notable improvement to an already great model. Veo 3.1’s generations are more coherent and the audio is even higher-quality. If you have a Google AI Pro or Ultra plan, you can try it in Gemini today by visiting https://gemini.google

Timestamps

(00:00:00) – AGI is still a decade away

(00:29:45) – LLM cognitive deficits

(00:40:05) – RL is terrible

(00:49:38) – How do humans learn?

(01:06:25) – AGI will blend into 2% GDP growth

(01:17:36) – ASI

(01:32:50) – Evolution of intelligence & culture

(01:42:55) - Why self driving took so long

(01:56:20) - Future of education

As part of this interview, Satya Nadella gave Dylan Patel (founder of SemiAnalysis) and me an exclusive first-look at their brand-new Fairwater 2 datacenter.

Microsoft is building multiple Fairwaters, each of which has hundreds of thousands of GB200s & GB300s. Between all these interconnected buildings, they’ll have over 2 GW of total capacity. Just to give a frame of reference, even a single one of these Fairwater buildings is more powerful than any other AI datacenter that currently exists.

Satya then answered a bunch of questions about how Microsoft is preparing for AGI across all layers of the stack.

Watch on YouTube; read the transcript.

Sponsors

* Labelbox produces high-quality data at massive scale, powering any capability you want your model to have. Whether you’re building a voice agent, a coding assistant, or a robotics model, Labelbox gets you the exact data you need, fast. Reach out at labelbox.com/dwarkesh

* CodeRabbit automatically reviews and summarizes PRs so you can understand changes and catch bugs in half the time. This is helpful whether you’re coding solo, collaborating with agents, or leading a full team. To learn how CodeRabbit integrates directly into your workflow, go to coderabbit.ai

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) - Fairwater 2

(00:03:20) - Business models for AGI

(00:12:48) - Copilot

(00:20:02) - Whose margins will expand most?

(00:36:17) - MAI

(00:47:47) - The hyperscale business

(01:02:44) - In-house chip & OpenAI partnership

(01:09:35) - The CAPEX explosion

(01:15:07) - Will the world trust US companies to lead AI?

Read the essay here.

Timestamps

00:00:00 What are we scaling?

00:03:11 The value of human labor

00:05:04 Economic diffusion lag is cope00:06:34 Goal-post shifting is justified

00:08:23 RL scaling

00:09:18 Broadly deployed intelligence explosion

In this lecture, military historian Sarah Paine explains how Russia—and specifically Stalin—completely derailed China’s rise, slowing them down for over a century.

This lecture was particularly interesting to me because, in my opinion, the Chinese Civil War is 1 of the top 3 most important events of the 20th century. And to understand why it transpired as it did, you need to understand Stalin’s role in the whole thing.

Watch on YouTube; read the transcript.

Sponsors

Mercury helps you run your business better. It’s the banking platform we use for the podcast — we love that we can see our cash balance, AR, and AP all in one place. Join us (and over 200,000 other entrepreneurs) at mercury.com

Labelbox scrutinizes public benchmarks at the single data-row level to probe what’s really being evaluated. Using this knowledge, they can generate custom training data for hill climbing existing benchmarks, or design new benchmarks from scratch. Learn more at labelbox.com/dwarkesh

To sponsor a future episode, visit dwarkesh.com/advertise.

Timestamps

(00:00:00) – How Russia took advantage of China’s weakness

(00:22:58) – After Stalin, China’s rise

(00:33:52) – Russian imperialism

(00:45:23) – China’s and Russia’s existential problems

(01:04:55) – Q&A: Sino-Soviet Split

(01:22:44) – Stalin’s lessons from WW2

Jay Paterno is a dynamic force, a leadership coach, change-maker, and author who is boldly redefining what legacy truly means. From the electrifying sidelines of Penn State to the challenging front lines of public service, he inspires leaders to act with unwavering courage, solid character, and genuine conviction. A true visionary, his insights cut through the noise, challenging conventional wisdom and sparking vital conversations nationwide about integrity, impact, and the relentless pursuit of excellence.

Takeaways:

• The Lawlessness of College Football: College football currently operates without true governance, driven by reactions to legal rulings and a lack of unified leadership, creating significant instability and pressure.
• The True Pressure of Leadership: Real pressure in leadership comes from the constant demands, ethical dilemmas, and the 24/7 nature of the role, often leading to mental health challenges for both leaders and those they guide.
• The Enduring Power of Values: Values are crucial anchors that provide stability during adversity and prevent leaders from losing their way during success, serving as an internal compass for integrity and purpose.

Sound Bytes: 

"There's two types of people in the world. There's problem people and solution people."

"Being a leader is not simply lying and never admitting you're wrong. Some of the greatest leaders are people who listen, who admit faults, and then correct them. And those things have all been lost."

"No matter how high or low my estimation in the eyes of the world, my conscience is clear. And that's more important than anything." 

Connect & Discover Jay:

Website: jayvpaterno.com

Facebook: @jaypaternoforpa

X: @JayPaterno

LinkedIn: @jaypaterno

Instagram: @jayvpaterno

YouTube: @nittanygameweek4442

Book: Blitzed! The All-Out Pressure of College Football’s New Era

🔥 Ready to Unleash Your Inner Game-Changer? 🔥 

Mick Hunt’s BEST SELLING book, How to Be a Good Leader When You’ve Never Had One: The Blueprint for Modern Leadership, is here to light a fire under your ambition and arm you with the real-talk strategies that only Mick delivers. 

👉 Grab your copy now and level up your life → Amazon, Barnes & Noble, Books A Million 

FOLLOW MICK ON:

Spotify: MickUnplugged

Instagram: @mickunplugged 

Facebook: @mickunplugged

YouTube:  @MickUnpluggedPodcast 

LinkedIn: @mickhunt 

Website:  MickHuntOfficial.com

Website: howtobeagoodleader.com

Website: Leadloudseries.com

Apple: MickUnplugged

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.