Podcast Summary
The podcast episode features Nat Friedman, the former CEO of GitHub, discussing various topics including the Herculaneum scrolls, AI, and his experiences at Microsoft and GitHub.
Main Topic: The Herculaneum Scrolls
The episode begins with Nat Friedman discussing the Herculaneum scrolls, a collection of ancient papyrus scrolls that were buried under volcanic ash and mud when Mount Vesuvius erupted in 79 AD. The scrolls are estimated to contain approximately 600-800 intact scrolls, which could potentially double the total text available from antiquity if they can be read. Nat and his team have launched the Vesuvius Challenge, a competition to develop a method to read the scrolls without damaging them.
Key Points:
Takeaways:
AI Discussion
Nat Friedman also discusses AI, including the development of large language models and the potential impact of AI on various industries.
Key Points:
Participants:
Takeaways:
GitHub Discussion
Nat Friedman also discusses his experiences as CEO of GitHub, including the acquisition of the company by Microsoft and the development of Copilot, a large language model for software development.
Key Points:
Participants:
Takeaways:
It is said that the two greatest problems of history are: how to account for the rise of Rome, and how to account for her fall. If so, then the volcanic ashes spewed by Mount Vesuvius in 79 AD - which entomb the cities of Pompeii and Herculaneum in South Italy - hold history’s greatest prize. For beneath those ashes lies the only salvageable library from the classical world.
Nat Friedman was the CEO of Github form 2018 to 2021. Before that, he started and sold two companies - Ximian and Xamarin. He is also the founder of AI Grant and California YIMBY.
And most recently, he has created and funded the Vesuvius Challenge - a million dollar prize for reading an unopened Herculaneum scroll for the very first time. If we can decipher these scrolls, we may be able to recover lost gospels, forgotten epics, and even missing works of Aristotle.
We also discuss the future of open source and AI, running Github and building Copilot, and why EMH is a lie.
Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.
Timestamps
(0:00:00) - Vesuvius Challenge
(0:30:00) - Finding points of leverage
(0:37:39) - Open Source in AI
(0:40:32) - Github Acquisition
(0:50:18) - Copilot origin Story
(1:11:47) - Nat.org
(1:32:56) - Questions from Twitter
Dwarkesh Patel
So we have 600 plus kind of roughly intact scrolls that we can't open. And I heard about this and I thought that was incredibly exciting, like the idea that there was information from 2,000 years in the past, we don't know what's in these things, if we could read all of them, then that would give us approximately a doubling of the total text that we have from antiquity. There are thousands more papyrus scrolls in there and we now have the techniques to read them, then there's gold in that mud and, you know, it's got to be dug out. I just fundamentally don't believe the world is efficient. And so if I see an opportunity to do something I used to, but I no longer have a reflexive reaction that says, oh, that must not be a good idea. If it were a good idea, someone would already be doing it.
Okay. Today, I have the pleasure of speaking with Nat Friedman, who was the CEO of GitHub from 2018 to 2021. Before that, he started and sold two companies, Ximian and Xamarin. And he is also the founder of AI Grant and California YIMBY. And most recently, he is the organizer and funder of the Scroll Prize, which is where we'll start this conversation. So, Nat, do you want to tell the audience about what the Scroll Prize is?
Nat Friedman
We're calling it the Vesuvius Challenge. This is just this crazy and exciting thing I feel incredibly honored to have gotten caught up in. A couple of years ago, I was reading, it was the midst of COVID, and I think we were in lockdown. Like everybody else, we were falling into internet rabbit holes. I just started reading about the eruption of Mount Vesuvius in Italy about 2000 years ago.
It turns out that when Vesuvius erupted, it was AD 79. It destroyed all the nearby towns. Everyone knows about Pompeii, but there was another nearby town called Herculaneum. Herculaneum was sort of like the Beverly Hills to Pompeii. Big villas, big houses, fancy people. In Herculaneum, there was one villa in particular that was enormous. It had once been owned by the father-in-law of Julius Caesar. A well-connected guy. It was full of beautiful statues and marbles and art, but it was also the home to a huge library of papyrus scrolls.
When the villa was buried, the volcano actually spit out enormous quantities of mud and ash. It buried Herculaneum in particular in something like 20 meters of material. It wasn't a thin layer. It was a very thick layer. Those towns were buried and forgotten for hundreds of years. No one even knew exactly where they were until the 1700s.
In 1750, a farm worker who was digging a well in the outskirts of Herculaneum struck this marble paving stone of a path that had been at this huge villa. He was pretty far down when he did that, 60 feet down. Subsequently, a Swiss engineer came in and started digging tunnels from that well shaft. They found all these treasures. The spirit at the time was looting. They were taking out incredible bronze statues that had survived. Along the way, they kept encountering these lumps of what looked like charcoal. They weren't sure what they were. Many were apparently thrown away until someone noticed a little bit of writing on one of them. They realized they were papyrus scrolls. There were hundreds, maybe thousands of them. They had uncovered this enormous library. It was the only library ever to have survived in any form, even though it was badly damaged. They were carbonized, very fragile, deformed.
In the open air, papyrus scrolls in a Mediterranean climate rot and decay quickly. They'd have to be recopied by monks every hundred years or so, maybe even less. We only have an estimated less than one percent of all the writing from that period. To find underground hundreds of papyrus scrolls, definitely not in good condition but still present, was an enormous discovery. People immediately, in a well-meaning attempt to read them, started trying to open them. But they're really fragile. They turn to ash in your hand. Hundreds were destroyed. People did things like cut them with daggers down the middle, and a bunch of little pieces would flake off and they'd try to get a few letters off of a couple of pieces.
Eventually, there was a monk named Piaggio, an Italian monk. He devised a machine under the care of the Vatican to unroll these things very, very slowly, like half a centimeter a day. A typical scroll could be 15 or 20 or 30 feet long. He managed to successfully unroll a few of these. They found Greek philosophical texts in the Epicurean tradition by a little-known philosopher named Philodemus. We got new text from antiquity, which is not a thing that happens all the time.
Eventually, people stopped trying to physically unroll these things because so many were destroyed. Some attempts to physically unroll the scrolls continued even into the 1990s and 2000s, and they were destroyed. The current situation is we have 600 plus roughly intact scrolls that we can't open. I heard about this and I thought that was incredibly exciting. The idea that there was information from 2000 years in the past, we don't know what's in these things.
Obviously people are trying to develop new ways and new technologies to open them. I read about a professor at the University of Kentucky, Brent Seales, who had been trying to scan these using increasingly advanced imaging techniques and then use computer vision techniques and machine learning to virtually unroll them without ever opening them. They tried a lot of different things. Their most recent attempt in 2019 was to take the scrolls to a particle accelerator in Oxford, England, called the Diamond Light Source, and to make an incredibly high resolution CT scan of them, a 3D x-ray scan. They needed really high energy photons to do this. They were able to take scans at eight microns, really quite tiny voxels, which they thought would be sufficient.
I thought this was the coolest thing ever. We're using technology to read this lost information from the past. I waited for the news that they had been decoded successfully. That was 2020. Then COVID hit, everybody got slowed down by that. Last year, I found myself wondering what happened to Dr. Seales and his scroll project. I reached out, and it turned out they'd been making really good progress. They'd gotten some machine learning models to start to identify ink inside of the scrolls, but they hadn't yet extracted words or passages. It's very challenging.
I invited him to come out to California and hang out. To my shock, he did. We got to talking and decided to team up and try to crack this thing. The approach we've settled on is to launch an open competition. We've done a ton of work with his team to get the data into a shape where, and the tools and techniques and broad understanding of the materials into a shape where smart people can approach it and get productive easily. I'm putting up together with Daniel Gross a prize, sort of like an X Prize, for the first person or team who can actually read substantial amounts of real text from one of these scrolls without opening them. We're launching that this week. I guess maybe it's when this airs. I don't know.
The stakes are kind of big. What gets me excited are the stakes. The 600 or 800 scrolls that are there, it's estimated that if we could read all of them and the technique works and generalizes to all the scrolls, then that would give us approximately a doubling of the total text that we have from antiquity. This is what historians and classicists tell me. It's not a 5% bump or a 10% bump in the total ancient Roman or Greek text. We get all of the texts that we have again. Multiple Shakespeares is one of the units I've heard. That would be significant. We don't know what's in there. We've got a few Philodemus texts. Those are of some interest. But there could be lost epic poems or God knows what.
I'm really excited. My bet is there's a 50% chance that someone will encounter this opportunity and get the data and get nerd sniped by it, and we'll solve it this year. It is something out of a science fiction novel. It's something you'd read in Neal Stephenson or something.
Dwarkesh Patel
I was talking to Professor Seales before, and apparently the shock went both ways because the first few emails, he was like, this has got to be spam. No way Nat Friedman is reaching out and has found out about this prize. That's really funny because he was really pretty hard to get in touch with. I emailed him a couple times, he just didn't respond. I asked my admin, Emily, to call the secretary of his department and say, Mr. Friedman requests me. Then he knew there was something actually going on there. He finally got on the phone with me and we got on Zoom. He's like, why are you interested in this?
Nat Friedman
I love Brent. He's fantastic. We're like friends now. I think we found that we think alike about this. I think he's reached the point where he just really wants to crack it. They've taken this right up to the one yard line. This is doable at this point. They've demonstrated every key component of putting it all together, improving the quality, doing it at the scale of a whole scroll. This is still very hard work. An open competition seems like the most efficient way to get it done.
Dwarkesh Patel
Before we get into the state of the data and the different possible solutions, I want to make tangible what could be gained if we can unwrap these. So you said there's a few more thousand scrolls. Are we talking about the ones in Philodemus' layer? Are we talking about the ones in other layers?
Nat Friedman
You'd think if you find this crazy villa that was owned by Julius Caesar's father-in-law that we'd just dig the whole thing out. But in fact, most of the exploration occurred in the 1700s through the Swiss engineers' tunnels underground. The villa was never dug out and exposed to the air. You went down 50, 60 feet and then you dig tunnels. Again, they were looking for treasure, not a full archaeological exploration. They mostly got treasure.
In the 90s, some additional excavations were done at the edge of the villa, and they discovered a couple things. First, they discovered it was a seaside villa that faced the ocean. It was right on the water before the volcano erupted. The eruption actually pushed the shoreline out by depositing so much additional mud there, so it's no longer right by the ocean. I've actually never been. They also found that there were two additional floors in the villa that the tunnels apparently had never excavated. At most, a third of the villa has been excavated.
When they were discovering these papyrus scrolls, they found basically one little room where most of the scrolls were. These were mostly these Philodemus texts, at least that's what we know. They found apparently several revisions sometimes of the same text. The hypothesis is this was actually Philodemus' working library. He worked here, this Epicurean philosopher. In the hallways, though, they occasionally found other scrolls, including crates of them. The belief, at least what historians have told me, is that the main library in this villa has probably not been excavated. The main library may be a Latin library and may contain literary texts, historical texts, other things. It could be much larger.
I don't know how prone these classicists are to wishful thinking. It is a romantic idea. But they have some evidence in the presence of these partly evacuated scrolls that were found in hallways. I've since gone and read a bunch of the firsthand accounts of the excavations. There are these heartbreaking descriptions of them finding an entire case of scrolls in Latin and accidentally destroying it as they tried to get it out of the mud. There were maybe 30 scrolls or something in there. There clearly was some other stuff that we just haven't got to.
Dwarkesh Patel
You made some scrolls?
Nat Friedman
Right, yeah. Papyrus is a papyrus. It's a reed, a grassy reed that grows on the Nile in Egypt. For thousands of years, many thousands of years, they've been making paper out of it. The way they do it is they take the outer rind off of the papyrus, then they cut the inner core into strips. They lay the strips out parallel to one another. Then they put another layer at 90 degrees to that bottom layer and they press it together in a press or under stones and let it dry out. That's papyrus essentially. Then they'll take some of those sheets and glue them together with paste made out of flour usually and get a long scroll. You can still buy it. I bought this on Amazon. It's interesting because it's got a lot of texture, those fibrous ridges of the papyrus plant. When you write on it, you really feel the texture. I got it because I wanted to understand what these artifacts are that we're working with.
We made an attempt to simulate carbonizing a few of these. We basically took a Dutch oven. When you carbonize something and make charcoal, it's not like burning it with oxygen. You remove the oxygen, heat it up, and let it carbonize. We tried to simulate that with a Dutch oven, which is probably imperfect. We left it in the oven at 500 degrees Fahrenheit for maybe, I don't know how long these were, but our biggest attempt was like five or six hours. These things are incredibly light. If you try to unfold them, they just fall apart in your hand very readily.
Dwarkesh Patel
I assume these are in somewhat better shape than the ones that were found because these were not in a volcanic eruption and covered in mud.
Nat Friedman
I think the volcano was probably, maybe that mud was hotter than my oven can go. It just flakes. You just squeeze it. It's just dust in your hand. We actually tried to replicate many of the heartbreaking 18th century unrolling techniques. They used rose water, for example, or they tried to use different oils to soften it and unroll it. Most of them are just very destructive. They poured mercury into it because they thought mercury would slip between the layers potentially. This is sort of what they look like. They shrink and they turn to ash.
Dwarkesh Patel
For those listening, by the way, it kind of looks like a black... just imagine the ash of a cigar, but blacker. It crumbles the same way. It's just a blistered black piece of rolled up papyrus.
Nat Friedman
Yeah. And they blister. The layers can separate. They can fuse. This happened in 79 AD, right? So we know that anything before that could be in here.
Dwarkesh Patel
So what could be in there? You and I have speculated about this.
Nat Friedman
I think it would be extremely exciting not to just get more Epicurean philosophy, although that's fine too. Almost anything would be interesting and additive. I think it would maybe have a big impact to find something about early Christianity, like a contemporaneous mention of early Christianity. Maybe there would be something that the church wouldn't want. That would be exciting to me. Maybe there would be some color or detail from someone commenting on Christianity or Jesus. I think that would be a very big deal. We have no such things as far as I know.
Other things that would be cool would be old stuff, even older stuff. There were several scrolls already found in there that they know were hundreds of years old when the villa was buried. The villa was probably constructed about 100 years prior, is my understanding. They can tell from the style of writing. They can date some of these scrolls. There was some old stuff in there. The Library of Alexandria was burned 80 or 90 years prior. Again, it may be wishful thinking, but there's some rumors that some of those scrolls were evacuated. Maybe some of them would have ended up at this substantial, prominent Mediterranean villa. God knows what would be in there. That would be really cool.
I think it would be great to find literature. Personally, I think that would be exciting, like beautiful new poems or stories. We just don't have a ton because so little survived. I think that would be fun. I think you had the best crazy idea for what could be in there, which was text which was GPT watermarked. That would be a creepy feeling.
Dwarkesh Patel
I still can't get over just how much of a plot of a sci-fi novel this is. Potentially the biggest intact library from the ancient world that has been sort of stopped like a debugger because of this volcano. The philosophers of antiquity forgotten, the earliest gospels. There's so much interesting stuff there. But let's talk about what the data looks like. You mentioned that they've been CT scanned and that they've developed these machine learning techniques to do segmentation and the unrolling. What would it take to get from there to understanding the actual content of what is within?
Nat Friedman
Dr. Seales actually pioneered this field of what he calls and is now widely called virtual unwrapping. He did it initially not with these Herculaneum scrolls. These things are like expert mode. They're so difficult. I'll tell you why soon. He did it initially with a scroll that was found in the Dead Sea in Israel. It's called the En Gedi Scroll. It was carbonized under slightly similar circumstances. I think there was a temple that was burned. The papyrus scroll was in a box. It kind of carbonized in the same way. It was not openable. It would just fall apart. The question was, could you non-destructively read the contents of it?
He did this 3D x-ray, the CT scan of the scroll, and was able to do two things. First, the ink gave a great x-ray signature. It looked very different from the papyrus. There's a high contrast. Second, he was able to segment the winds of the scroll throughout the entire body of the scroll and identify each layer, and then just geometrically unroll it using fairly normal flattening computer vision techniques and then read the contents. It turned out to be, I think, an early part of the Book of Leviticus, something of the Old Testament or the Torah. That was a landmark achievement.
The next idea was to apply those same techniques to this case. This is proven hard. There are a couple things that make it difficult. One is that the ink used on the Herculaneum papyri is not very absorbent of X-ray. It basically seems to be equally absorbent of X-ray as the papyrus or very close, certainly not perfectly. You don't have this nice bright lettering that shows up on your tomographic 3D X-ray. You have to somehow develop new techniques for finding the ink in there. That's problem one, and it's been a major challenge.
The second problem is the scrolls are just real messed up. They were long and tightly wound, highly distorted by the volcanic mud, which not only heated them but partly deformed them. The segmentation problem of identifying each of these layers throughout the scrolls, it's doable but it's hard. Those are a couple of challenges. The other challenge is just getting access to scrolls and taking them to a particle accelerator. You have to have scroll access and particle accelerator access and time on those. It's expensive and difficult. Dr. Seales did the hard work of making all that happen.
The good news is, very recently, just in the last couple of months, his lab has demonstrated with a convolutional neural network the ability to actually recognize ink inside these x-rays. I look at the x-ray scans and I cannot, at least in any of the renderings that we've seen, I can't see the ink. But the machine learning model can pick up on very subtle patterns in the x-ray absorption at high resolution inside these volumes in order to identify ink. We've seen that.
You might ask, how do you train a model to do that? You need some kind of ground truth data to train the model. The big insight that they had was to train on broken off fragments of the papyrus. As people tried to open these over the years in Italy, they destroyed many of them, but they saved some of the pieces that broke off. On some of those pieces, you can kind of see lettering. If you take an infrared image of the fragment, then you can really see the lettering pretty well in some cases. They think it's 930 nanometers. They take this little infrared image. Now you've got some ground truth. Then you do a CT scan of that broken off fragment and you try to align it, register it with the image. Then you have data that you can use potentially to train a model. That turned out to work in the case of the fragments.
This is the why now. This is why I think launching this challenge now is the right time. We have a lot of reasons to believe it can work. The core techniques, the core pieces have been demonstrated. It just all has to be put together at the scale of these really complicated scrolls. If you can do the segmentation, which is probably a lot of work, maybe there's some way to automate it, and then you can figure out how to apply these models inside the body of a scroll and not just to these fragments, then it seems like you could probably read lots of text.
Dwarkesh Patel
Why did you decide to do it in the form of a prize rather than just giving a grant to the team that was already pursuing it or maybe some other team that wants to take it up?
Nat Friedman
We talked about that. What we basically concluded was the search space of different ways you could solve this is pretty big. We just wanted to get it done as quickly as possible. Having a contest means lots of people are going to try lots of things and someone's going to figure it out quickly. Many eyes may make it shallow as a task. I think that's the main thing. Probably someone could do it, but I think this will just be a lot more efficient. It's fun too. I think this is fun. It's interesting to do a contest and who knows who will solve it or how. People may not even use machine learning. We think that's the most likely approach for recognizing the ink, but they may find some other approach that we haven't thought of.
Dwarkesh Patel
One question people might have is that you have these visible fragments mapped out. Do we expect them to correspond to the burned off or the ashen carbonized scrolls? Can you do machine learning on the ground truth of one and have it correspond to the other?
Nat Friedman
I think this is a very legitimate concern. They're different. When you have a broken off fragment, there's air above the ink. When you CT scan it, you have ink next to air. Inside of a wrapped scroll, the ink might be next to papyrus because it's pushing up against the next layer. Your model may not know what to do with that. I think this is one of the challenges, how you take these models that were trained on fragments and translate them into the slightly different environment. But maybe there are parts of the scroll where there is air on the inside. We know that to be true. You can sort of see that here. I think it should at least partly work and clever people can probably figure out how to make it completely work.
Dwarkesh Patel
So you said the odds are about 50-50. What makes you think that?
Nat Friedman
It can be done because we've recognized ink from a CT scan on the fragments. I think everything else is probably geometry and computer vision. The scans are very high resolution. They're 8 microns, 8 micrometers. They're taken, if you kind of stood a scroll on end like this, in these slices through it. In the Z axis from bottom to top, they're these slices. The way they're represented on disk is each slice is a TIFF file. For the full scrolls, each slice is like 100-something megabytes. They're quite high resolution. If you stack 100 of these, they're 8 microns, so 100 of these is 0.8 millimeters. A millimeter is pretty small. We think the resolution is good enough or at least right on the edge of good enough that it should be possible. It seemed to be 6 or 8 pixels or voxels, I guess, per across an entire layer of papyrus. That's probably enough.
We've also seen with the machine learning models, Dr. Seales has got some PhD students who have actually demonstrated this at eight microns. I think the ink recognition will work. The data is clearly physically in the scrolls. The ink was carbonized. The papyrus was carbonized. A lot of data actually physically survived. The question is, did the data make it into the scans? I think that's very likely based on the results we've seen so far. I think it's just about a smart person solving this or a smart group of people or just a dogged group of people who do a lot of manual work. That could also be true. You may have to be smart and dogged. That's where most of my uncertainty is, just whether somebody does it.
Dwarkesh Patel
If a quarter of a million dollars doesn't motivate you.
Nat Friedman
I think money is good. There's a lot of money in machine learning these days.
Dwarkesh Patel
Do we have enough data in the form of scrolls that have been mapped out to be able to train a model if that's the best way to go? One question somebody might have is, if you already have this ground truth, why hasn't Dr. Seales' team already been able to just train a model?
Nat Friedman
I think they will. If we just let them do it, they'll get it solved. It might take a little bit longer because it's not a huge number of people and there is a big search space here. If we didn't launch this contest, I'd still think this would get solved. But it might take several years. I think this way it's likely to happen this year.
Dwarkesh Patel
And what happens, let's say the prize is solved. Somebody figures out how to do this and we can read the first scroll. You mentioned that these other layers haven't been excavated. How is the world going to react?
Let's say we get one of these mapped out. That's my personal hope for this. I always like to look for these cheap leverage hacks, these moments where you can do a relatively small thing and it creates – you kick a pebble and you get an avalanche. The theory is – and Brent shares this theory – that if you can read one scroll, just one scroll, and we only have two scanned scrolls, there are hundreds of surviving scrolls. It's relatively expensive to book a particle accelerator. So if you can scan one scroll and you know it works and you can generalize the technique out and it's going to work on these other scrolls, then the money, which is probably low millions, maybe only $1 million to scan the remaining scrolls, will just arrive.
It's too sweet of a prize for that not to happen. The urgency and return on excavating the rest of the villa will be incredibly obvious too, because if there are thousands more papyrus scrolls in there and we now have the techniques to read them, then there's gold in that mud. It's got to be dug out. It's amazing how little money there is for archaeology. For decades, no one's been digging there. So that's my hope: this is the catalyst. It works, somebody reads it, they get a lot of glory, we all get to feel great, and then the diggers arrive in Herculaneum and they dig out the rest.
Nat Friedman
I wonder if the budget for archaeological movies and games like Uncharted or Indiana Jones is bigger than the actual budget to do real world archaeology.
Dwarkesh Patel
I was talking to some of the people before this interview, and that's one thing they emphasized: your ability to find these leverage points. For example, with California YIMBY, I don't know the exact amount you seeded it with, but for that amount of money and for an institution that is that new, it is one of the very few institutions that has had a significant amount of political influence. If you look at the state of YIMBY in California and nationally today, how do you identify these things? How do you see... There are plenty of people who have money who get into history or get into whatever subject, but very few do something about it. How do you figure out where to...
Nat Friedman
I don't know. I'm a little bit mystified by why people don't do more things too. I think, first of all, maybe you can tell me: why are more people doing things? I think most rich people are boring and they should do more cool things. So I'm hoping that they do that now. But I think part of it is I just fundamentally don't believe the world is efficient. If I see an opportunity to do something, I don't have that reflexive reaction that says, "Oh, that must not be a good idea. If it were a good idea, someone would already be doing it. Someone must be taking care of housing policy in California, right?
Or somebody must be taking care of this or that." So I don't have that filter that says the world's efficient. Don't bother. Someone's probably got it covered. The second thing is I've learned to trust my enthusiasm. It gets me in trouble too, but if I get really enthusiastic about something and that enthusiasm persists, I just indulge it. I like doing the things I'm enthusiastic about, so I let myself be impulsive. There's this great image I found and tweeted which said, "We do these things not because they are easy, but because we thought they would be easy." That's frequently what happens.
The commitment to do it is impulsive, done out of enthusiasm. Then you get into it and you're like, "Oh my God, this is really much harder than we expected." But then you're committed and you're stuck and you're going to have to get it done. I thought this project would be relatively straightforward. We're just going to take the data and put it up. But of course, everything is... and truly 99% of the work has already been done by Dr. Seales and his team at the University of Kentucky. I am a kind of carpetbagger. I've shown up at the end here to try to do a new piece of it. But the last mile is often the hardest.
It's turned out to be fractal anyway, all the little bits that you have to get right to do a thing and have it work. I hope we got all of them. So that's part of it: not believing the world's efficient, then allowing your enthusiasm to cause you to commit to something that turns out to be a lot of work and really hard. Then you're stubborn and don't want to fail, so you keep at it.
Dwarkesh Patel
Do you think that efficiency point is particularly true of things like California and this where there isn't a direct monetary incentive?
Nat Friedman
No. Look, certainly parts of the world are more efficient than others. You can't assume equal levels of inefficiency everywhere. But I'm constantly surprised by how even in areas you expect to be very efficient, there are things that are sort of in plain sight. It's not that I see them and others don't. There's lots of stuff I don't see too. I was talking to some traders at a hedge fund recently, and I asked them about the role secrets play in the success of a hedge fund. The reason I was interested is because I think the AI labs are going to enter a new similar dynamic where their secrets are very valuable.
If you have a 50% training efficiency improvement and your training runs cost $100 million, that is a $50 million secret that you want to keep. Hedge funds do that kind of thing routinely. I asked some traders at a very successful hedge fund: if your smartest trader got on Twitch for 10 minutes once a month and described their 30-day-old trading strategies, not your current ones, but the ones that are a month old, how would that affect your business after 12 months? So 12 months, 10 minutes a month, 30-day look back. That's two hours in a year. To my shock, they told me an 80% reduction in their profits.
It would have a huge impact. Then I asked how long the look back window would have to be before it would have a relatively small effect on your business. They said 10 years. That is quite strong evidence that the world's not perfectly efficient. These folks make billions of dollars using secrets that could be related in like an hour or something like that. Yet others don't have them or their secrets wouldn't work. There are different levels of efficiency in the world, but on the whole, our default estimate of how efficient the world is is far too charitable.
Dwarkesh Patel
On the particular point of AI labs potentially starting secrets, you have this strange norm of different people from different AI labs not only being friends but often living together. It would be like Oppenheimer living with somebody working on the Russian atomic bomb. Do you think those norms will persist once the value of these secrets is realized?
Nat Friedman
I was just wondering about that some more today. They seem to be trying to close the valves, but there are a lot of things working against them. One is that the secrets are relatively simple. Two is that you're coming off this academic norm of publishing. The entire culture is based on sharing and publishing. Three, as you said, they all live in group houses. Some are in polycules. There's a lot of intermixing. And it's all in California, which is a non-compete state. We don't have non-competes. You'd have to change the culture, get everybody their own house, and move to Connecticut. Then maybe it would work.
ML engineer salaries and compensation packages will probably be adjusted to try to address this because you don't want your secrets walking out the door. There are engineers like Igor Babushkin, for example, who has just joined Twitter. Is that right? Elon hired him to train. I think that's public. It will be now. Igor is a really great guy and brilliant, but he also happens to have trained state-of-the-art models at DeepMind and OpenAI. That's the set of people who have that set of... I don't know whether that's a consideration or how big of an effect that is, but it's the kind of thing that it would make sense to value if you think there are valuable secrets that have not yet proliferated.
So I think they're going to try to slow it down. Publishing has certainly slowed down dramatically already, but there's a long way to go before you're anywhere near hedge fund or Manhattan Project territory. Secrets will probably still have a relatively short half-life.
Dwarkesh Patel
As somebody who has been involved in open source your entire life, are you happy that this is the way that AI has turned out, or do you think this is less than optimal?
Nat Friedman
I don't know. My opinion has been changing. I have increasing worries about safety issues, not the hijacked version of safety, but some industrial accident-type situations or misuse. I do think there are worlds we should be a little bit concerned about, although I don't know what to do about it where bad things happen. My belief, though, is that it's probably better on the whole for more people to get to tinker with and use these models, at least in their current state. For example, when Georgi Gerganov this weekend did a 4-bit quantization of the LLaMA model and got it inferencing on an M1 or M2, I was very excited. I got that running and it's fun to play with. Now I've got a model that's very good. It's almost GPT-3 quality. It runs on my laptop. I've grown up in this world of the tinkerers and open source folks. The more access you have, the more things you can try. I find myself very attracted to that.
Dwarkesh Patel
I guess that is the scientist and the ideas part of what is being shared. But there's also another part about the actual substance. As different sources of data realize how valuable their data is for training newer models, do you think that these things will become harder to scrape? LibGen, Archive? Are these going to become rate limited in some way?
Nat Friedman
First, there's so much data on the internet. The two primitives that you need to build models are lots of data, which we have in the form of the internet. We digitized the whole world into the internet. And then you need these GPUs, which we have because of video games. You take the internet and video game hardware and you smash them together and you get machine learning models. They're both commodities. I don't think anyone in the open source world is really going to be data limited for a long time. There's so much that's out there. Probably people who have proprietary data sets that are readily scrapable have been shutting those down. So get your scraping in now if you need to do it. But that's just on the margin. I still think there's quite a lot that's out there to work with. Look, there's going to be a ton. This is the year of proliferation. This is a week of proliferation. We're going to see four or five major AI announcements this week. New models, new APIs, new platforms, new tools from all the different vendors. In a way, they're all looking forward. My Herculaneum project is looking backwards. I think it's extremely exciting and cool, but it is a funny contrast.
Dwarkesh Patel
Before we delve deeper into AI, I do want to talk about GitHub. You are at Microsoft, and at some point you realize that GitHub is very valuable and worth acquiring. How did you realize that, and how did you convince Microsoft to purchase GitHub?
Nat Friedman
I had started a company called Xamarin together with Miguel de Icaza and Joseph Hill. We built mobile tools and platforms. Microsoft acquired the company in 2016. I was excited about that. I thought it was great. But to be honest, I didn't actually expect or plan to spend more than a year or so there. When I got in there, I got exposed to what Satya was doing and the quality of his leadership team. I was really impressed. In my first week or so, he asked me, "What do you think we should do at Microsoft?" I said, "I think we should buy GitHub."
Dwarkesh Patel
When would this have been?
Nat Friedman
This was like my first week. March or April of 2016. He said, "Yeah, it's a good idea. We thought about it. I'm not sure we can get away with it or something like that." About a year later, a little more than a year later, I wrote him an email, just a memo. I said, "I think it's time to do this." There was some noise that Google was sniffing around. I think that may have been manufactured by the GitHub team, but it was a good catalyst because it was something I thought made a lot of sense for Microsoft to do anyway. I wrote an email to Satya, a little memo saying, "Hey, I think we should buy GitHub.
Here's why. Here's what we should do with it." The basic argument was that developers are making IT purchasing decisions now. It used to be this IT thing, but now developers are leading that purchase. It's a major shift in how software products are acquired. Microsoft really was an IT company. It was not a developer company in the way most of its purchases were made. But it was founded as a developer company. Microsoft's first product was a programming language. I said, "The challenge we have is there's an entire new generation of developers who have no affinity with Microsoft. The largest collection of them is at GitHub.
If we acquire this and we do a merely competent job of running it, we can earn the right to be considered by these developers for all the other products that we do." To my surprise, Satya replied in like six or seven minutes and said, "I think this is very good thinking. Let's meet next week and talk about it." I ended up in a conference room with him and Amy Hood and Scott Guthrie and Kevin Scott and several other people. They said, "Okay, tell us what you're thinking." I did a little 20-minute ramble on it. Satya said, "I think we should do it. Why don't we run it independently like LinkedIn?
Nat, you'll be the CEO." He said, "Do you think we can get it for $2 billion?" I said, "We could try." Three weeks later, he said, "Okay, go do this. Scott will support you." Three weeks later, we had a signed term sheet and an announced deal. It was an amazing experience for me. I'd been there less than two years. Microsoft was made up of and run by a lot of people who'd been there for many years. They trusted me with this really big project and made me feel really good to be trusted and empowered. I had grown up in the open source world. For me to get an opportunity to run GitHub, it's like getting appointed mayor of your hometown or something.
It felt cool. I really wanted to do a good job for developers.
Dwarkesh Patel
That's one of the things I want to ask you about. Often when something succeeds, we think it was inevitable that it would succeed. But at the time, I remember there was a huge amount of skepticism. I would go on Hacker News and the top thing would be the blog post about how Microsoft's going to mess up GitHub. People have had those concerns alleviated throughout the years, but how did you deal with that skepticism and distrust?
Nat Friedman
I was really paranoid about it. I really cared about what developers thought. There's always this question of who are you performing for? Who do you actually really care about? Who's the audience that's in your head that you're trying to do a good job for, impress, earn the respect of? Though I love Microsoft and care a lot about Satya and everyone there, I really cared about the developers. I'd grown up in this open source world. For me to do a bad job with this central institution of open source would have been a devastating feeling. It was very important to me not to. That was the first thing: I cared.
The second thing is that the deal leaked. It was going to be announced on a Monday. It leaked on a Friday. Microsoft is buying GitHub. The whole weekend there were terrible posts online. People saying we've got to evacuate GitHub as quickly as possible. We're like, "Oh my God, this is terrible." Monday we put the announcement out. We said we're acquiring GitHub. It's going to run as an independent company. And it said, "Nat Friedman is going to be CEO." I don't want to overstate it, but I think a couple of people were like, "Oh, Nat comes from open source. He spent some time in open source. Nat is going to be running independently."
I don't think they were really that calmed down, but at least a few people thought, "Maybe I'll give this a few months and just see what happens before I migrate off." My first day as CEO, after we got the deal closed, 9 a.m., I was in this room and we got on Zoom with all the heads of engineering and product. I don't know what people were expecting, maybe some kind of longer-term strategy. But I came in and said that GitHub had no official feedback mechanism that was publicly available, but there were several GitHub repos that community members had started. Isaac from npm had started one where he'd just been allowing people to give GitHub feedback, and people had been voting on this stuff for years.
I shared my screen and put that up, sorted by votes, and said, "We're going to pick one thing from this list and fix it by the end of the day and ship that. Just one thing." I think people were like, "This is the new CEO strategy?" They were like, "I don't know, we can't... you have to do database migrations, you can't do that in a day." Then someone said, "Well, maybe we can do this. We actually have a half implementation of this." We eventually found something that we could fix by the end of the day. What I was thinking, what I hope I said, was, "What we need to show the world is that GitHub cares about developers, not that it cares about Microsoft."
If the first thing we did after the acquisition was to add Skype integration, developers would have said, "Oh, we're not your priority. You have new priorities now." The idea was just to find ways to make it better for the people who use it and have them see that we cared about that immediately. I said, "We're going to do this today and then we're going to do it every day for the next 100 days." It was cool because it created some really good feedback loops. One was that you ship things and then people are like, "Oh hey, I've been wanting to see this fixed for years and now it's fixed." It's a relatively simple thing, so you get this nice dopaminergic feedback loop.
People in the team feel the excitement of shipping stuff. GitHub was a company that had a little bit of stage fright about shipping previously. Breaking that static friction and shipping a little bit more felt good. The other one is just the learning loop. By trying to do lots of small things, I got exposed to, "Okay, this team is really good. Or this part of the code has a lot of tech debt. Or hey, we shipped that and it was actually kind of bad. How come that design got out?" If the project had been some six-month thing, I'm not sure my learning would have been quite as quick about the company.
There's still things I missed and mistakes I made, for sure. But that was part of how I think... no one knows counterfactually whether that made a big difference or not. I do think that earned some trust.
Dwarkesh Patel
Most acquisitions don't go well. Not only do they not go as well, but they don't go well at all, as we're seeing in the last few months with a certain one. Why do most acquisitions fail?
Nat Friedman
It is true. Most acquisitions are destructive of value. The value of a company in an innovative industry, a lot of it boils down to its ability culturally to produce new innovations. It's some sensitive harmonic of cultural elements that sets that up and makes that possible. It's quite fragile. If you take a culture that has achieved some productive harmonic and you put it inside of another culture that's really different, the mismatch can destroy the productivity of the company. Companies are a little bit fragile. When you acquire them, it's relatively easy to break them. They're also more durable than people think in many cases. Another version of it is that the people who really care leave. The people who really care about building great products and serving the customers maybe don't want to work for the acquirer. The set of people that are load-bearing around the long-term success is small, and when they leave or get disempowered, you get very different behaviors.
Dwarkesh Patel
I want to go into the story of Copilot. Until recently, it was the most widely used application of modern AI models. Whatever part of the story you're willing to share in public.
Nat Friedman
I've talked about this a little bit. GPT-3 came out in May of 2020. I saw it and it really blew my mind. I thought it was amazing. I was CEO of GitHub at that time. I thought, "I don't know what, but we've got to build some product with this. We've got to build something." Sam Altman had, at I think Kevin Scott's urging, already invested in OpenAI like a year before GPT-3 came out.
Dwarkesh Patel
This is quite amazing. He invested like a billion dollars. Do you know why he knew that OpenAI would be worth investing in at that point?
Nat Friedman
I don't know. Actually, I've never asked him. I'm not sure. It's a good question. I think OpenAI had already had some successes that were noticeable. If you're Satya and you're running this multi-trillion dollar company, you're trying to execute well and serve your customers, but you're always looking for the next gigantic wave that is going to upend the technology industry. It's not just about trying to win cloud. It's like, "Okay, what comes after cloud?" You have to make some big bets. I think he thought AI could be one. Kevin Scott deserves a lot of credit for really advocating for that aggressively. Sam Altman did a good job of building that partnership because he knew that he needed access to the resources of a company like Microsoft to build large-scale AI and eventually AGI. I think it was some combination of those three people coming together to make it happen. I still think it was a very prescient bet. I've said that to people and they've said, "Well, a billion dollars is not a lot for Microsoft." But there were a lot of other companies that could have spent a billion dollars to do that and did not. I still think that deserves a lot of credit.
So GPT-3 comes out. I pinged Sam and Greg Brockman at OpenAI. They were like, "Yeah, we've already been experimenting with GPT-3 and derivative models in a coding context. Let's definitely work on something." To me and a few other people, it was not incredibly obvious what the product would be. Now I think it's trivially obvious. Autocomplete. My gosh, isn't that what the models do?
But at the time, actually my first thought was that it was probably going to be like a Q&A chat bot, a Stack Overflow type of thing. And so that was actually the first thing we prototyped. We grabbed a couple of engineers, this guy who had come in from an acquisition that we'd done, and Alex Graveley, and started prototyping. The first prototype was a chatbot.
What we discovered first was that the demos were fabulous. Every AI product has a fantastic demo. You get this sort of wow moment. That turns out to be maybe not a sufficient condition for a product to be good. At the time, the models were just not reliable enough. They were not good enough. I ask you a question, 25% of the time you give me an incredible answer that I love. 75% of the time your answer is useless or wrong. It's not a great product experience.
So then we started thinking about code synthesis. Our first attempts at this were actually large chunks of code synthesis, like synthesizing whole function bodies. We built some tools to do that and put them in the editor. That also was not really that satisfying.
The next thing that we tried was to just do simple small-scale autocomplete with the large models. We used the kind of IntelliSense drop-down UI to do that. That was better, definitely pretty good. But the UI was not quite right. And we lost the ability to do this large-scale synthesis. We still had that, but the UI for that wasn't good.
We had it so that to get a function body synthesized, you would hit a key. And I don't know why this was the idea everyone had at the time, but several people had this idea that it should display multiple options for the function body. The user would read them and pick the right one. The idea was that we would use that human feedback to improve the model. But that turned out to be a bad experience. First you had to hit a key and explicitly request it. Then you had to wait for it. Then you had to read three different versions of a block of code. Reading one version of a block of code takes some cognitive effort. Doing it three times takes more cognitive effort. Most often the result was that none of them were good or you didn't know which one to pick. You're putting a lot of energy in and you're not getting a lot out. It's frustrating.
Once we had that single line completion working, I think Alex had the idea of saying we can use the cursor position in the AST to figure out heuristically whether you're at the beginning of a block in the code or not. If it's not the beginning of a block, just complete a line. If it's the beginning of a block, show in-line a full block completion. The number of tokens you request and when you stop gets altered automatically with no user interaction. Then the idea of using this sort of gray text like Gmail had done in the editor. We got that implemented.
It was really only once all those pieces came together and we started using a model that was small enough to be low latency but big enough to be accurate that we reached the point where the median new user loved Copilot and wouldn't stop using it. That took four months, five months of just tinkering and exploring. There were other dead ends we had along the way.
Then it became quite obvious that it was good because we had hundreds of internal users who were GitHub engineers. I remember the first time I looked at the retention numbers, they were extremely high. It was like 60 plus percent after 30 days from first install. If you installed it, the chance that you were still using it after 30 days is over 60%. It's a very intrusive product. It's always popping UI up. If you don't like it, you will disable it. Indeed, 40-something percent of people did disable it. But those are very high retention numbers for an alpha, first version of a product that you're using all day. I was incredibly excited to launch it. It's improved dramatically since then.
Dwarkesh Patel
Sounds very similar to the Gmail story.
Nat Friedman
Right. It's an incredibly valuable insight and then it becomes obvious that it needs to go outside.
Dwarkesh Patel
Okay, we'll go back to the AI stuff in a second. But some more GitHub questions. By what point will, if ever, will GitHub profiles replace resumes for programmers?
Nat Friedman
That's a good question. I think they're a contributing element to how people try to understand a person now, but I don't think they're a definitive resume. We introduced READMEs on profiles when I was there. I was excited about that because I thought it gave people some degree of personalization. Many thousands of people have done that.
There are forces pushing in the other direction too, where people don't want their activity and skills to be as legible. There may be some adverse selection as well where the people with the most elite skills, it's rather gauche for them to signal their competence on their profile. So there are some weird social dynamics that feed into it too.
I will say, I think it effectively has this role for people who are breaking through today. One of the best ways to break through. I know many people who are in this situation: you were born in Argentina, you're a very sharp person, but you didn't grow up in a highly connected or prosperous network family. Yet you're really capable and you just want to get connected to the most elite communities in the world. If you're good at programming, you can join open source communities and contribute to them. You can very quickly accrete a global reputation for your talent, which is legible to many companies and individuals around the world. Suddenly you find yourself getting a job and moving maybe to the US, or maybe not moving, or you end up at a great startup. I know a lot of people who've deliberately pursued the strategy of building reputation in open source and then the wind catches you and you've got a career.
In other communities, like in machine learning research, this is not how it works. Their reputation is more on arXiv than it is on GitHub. I don't know that it will ever be comprehensive.
Dwarkesh Patel
Are there any other industries for which proof of work of this kind will eat more into the way in which people are hired?
Nat Friedman
I think there's a labor market dynamic in software where the really high-quality talent is so in demand and the supply is so much less than the demand that it shifts power onto the developers. They can require of their employers that they be allowed to work in public. When they do that, they develop an external reputation, which is an asset they can port between companies. If the labor market dynamics weren't like that, if programming well were less economically valuable, then companies wouldn't let them do that. They'd say that's a rule. That used to be the case, in fact. As software has become more valuable, the leverage of a single super talented developer has gone up and they've been able to demand over the last several decades the ability to work in public. I think that's not going away.
Dwarkesh Patel
Other than that, what has been the impact of developers being more empowered in organizations, even ones that are not traditionally IT organizations?
Nat Friedman
Software is kind of magic. You can write a for loop and do something a lot of times. When you build large organizations at scale, one of the things that does surprise you is the degree to which you need to systematize the behavior of the people who are working. When I first was starting companies and building sales teams, I had this wrong idea coming from the world as a programmer that salespeople were hyper aggressive, hyper entrepreneurial, making promises to the customer that the product wouldn't do. The main challenge you had with salespeople was restraining them from aggressively cutting deals that shouldn't be cut.
What I discovered is that does exist sometimes. But the much more common case is that you need to build a systematic sales playbook, which is almost a script that you run on your sales team. Your sales reps know the process they need to follow to exercise this repeatable sales motion and get a deal closed. I just had bad ideas there. I didn't know that was how the world worked. But software is a way to systematize and scale out a valuable process extremely efficiently. The more digitized the world has become, the more valuable software becomes and the more valuable become the developers who can create it.
Dwarkesh Patel
Would 25-year-old you be surprised with how well open source worked and how pervasive it is?
Nat Friedman
Yeah, I think that's true. I think we all have this image when we're young that these institutions are these implacable edifices that are evil and all-powerful and are able to, with master plans, substantially orchestrate the world. That is sometimes a little bit true, but they're very vulnerable to new ideas, new forces, new communications media. Right now, I think our institutions overall look relatively weak. Certainly, they're weaker than I thought they were back then.
I thought Microsoft could stop open source. I thought that was a possibility. They could do some patent move and kind of ring fence open source in. That didn't end up being the case. In fact, Microsoft, when we bought GitHub, we pledged all of our patent portfolio to open source. That was one of the things that we did as part of it. That was a kind of poetic moment for me, having been on the other side of patent discussions in the past, to be instrumental in Microsoft making that pledge. That was quite crazy.
Dwarkesh Patel
It wasn't that there was some business strategic reason? More so it was just an idea whose time had come?
Nat Friedman
GitHub had made such a pledge. In part, in acquiring GitHub, we had to either try to annul that pledge or sign up to it ourselves. There was a moment of a forced choice. Everyone at Microsoft thought it was a good idea too. In many senses, it was a moment whose time had come and the GitHub acquisition was a forcing function.
Dwarkesh Patel
What do you make of critics of modern open source like Richard Stallman or people who advocate for free software? They say corporations might advocate for open source because of practical reasons for getting good code, but the real value of software and the real way the software should be made is that it should be free. You can replicate it, change it, modify it, and completely view it. The ethical values about that should be more important than the practical values. What do you make of that critique?
Nat Friedman
Those are the things that he wants. The thing that maybe he hasn't updated is that maybe not everyone else wants that. He has this idea that people want freedom from the tyranny of a proprietary intellectual property license. But what people really want is freedom from having to configure their graphics card or sound driver. They want their computer to kind of work. There are places where freedom is really valuable, but there's always this thing of "I have a prescriptive ideology that I'd like to impose on the world" versus "I will try to develop the best observational model for what people actually want, whether I want them to want it or not." I think Richard is strongly in the former camp.
Dwarkesh Patel
What is the most underrated license, by the way?
Nat Friedman
I don't know. Maybe the MIT license is still underrated because it's just so simple and bare.
Dwarkesh Patel
Nadia Eghbal had a book recently where she argued that the key constraint on open source software and on the time of the people who maintain it is the community aspect of software. They had to deal with feature requests and discussions and maintaining for different platforms. It wasn't the actual code itself, but rather this sort of extracurricular aspect that was the main constraint. Do you think that is the constraint for open source software? How do you see what is holding up more open source software?
Nat Friedman
By and large, I would say that there is not a problem. Open source software continues to be developed, continues to be broadly used. There are areas where it works better and areas where it works less well. It's winning in all the areas where large scale coordination and editorial control are not necessary. It tends to be great at infrastructure, standalone components, and very horizontal things like operating systems. It tends to be worse at user experiences and things where you need a dictatorial aesthetic or editorial control.
I've had debates with Dylan Field at Figma as to why it is that we don't have lots of good open source applications. I've always thought it had something to do with this governance dynamic. It's such a pain to coordinate with tons of people who all feel like they have a right to try to push the project in one way or another. Whereas in a hierarchical corporation, there can be a head of this product or a CEO or a founder or a designer who just says, "We're doing it this way." You can really align things in one direction very easily.
Dylan has argued to me that it might be because there are just fewer designers, people with good design sense in open source. I think that might be a contributing factor too, but I think it's still mostly the governance thing. That's what Nadia is pointing at also. You're running a project, you gave it to people for free. For some reason, giving people something for free creates a sense of entitlement. They feel like they have the right to demand your time, push things around, give you input. It's very draining. Where that coordination burden is lower is where open source tends to succeed more. Probably software and other new forms of governance can improve that and expand the territory that open source can succeed in.
Dwarkesh Patel
Theoretically those two things are consistent. You could have very tight control over governance while the code itself is open source. This happens in programming languages. Languages can't be designed by committee. They often are eventually set in stone and then advanced by committee. You have these benign dictators of languages who enforce a strong set of ideas. They have a vision master plan. That would be the argument that's most on Dylan's side. It's like, hey, it works for languages. Why can't it work for end-user applications?
Nat Friedman
The thing you need to do to build a good end user application is not only have a good aesthetic and idea, but somehow establish a tight feedback loop with a set of users. You give it to them, they try it, you get feedback. Doing that is so hard, even in a company where you have total hierarchical control of the team, everyone really wants the same thing, and everyone's salary and stock options depend on the product being accepted by these users. It still fails many times in that scenario. Doing that in the context of open source is just slightly too hard.
Dwarkesh Patel
The reason you acquired GitHub, as you said, is that there seemed to be complementarity between Microsoft and GitHub's missions. I guess that's been proven out over the last few years. Should there be more of these collaborations and acquisitions? Should there be more tech conglomerates? Would that be good for the system?
Nat Friedman
I don't know if it's good, but I think there are. It is certainly efficient in many ways. We are seeing agglomeration occur because the math is pretty simple. If you are a large company and you have a lot of customers, the thing that you've achieved is this very expensive and difficult thing of building distribution and relationships with lots of customers. That is as hard or harder and takes longer and more money than just inventing the product in the first place. If you can go and just buy the product for a small amount of money and make it available to all of your customers, then there's often an immediate, obvious gain from doing that.
In that sense, acquisitions make a ton of sense. I've been surprised that the large companies haven't done many more acquisitions in the past, until I got into a big company and started trying to do acquisitions. I saw that there are strong elements of the internal dynamics that make it hard. It's easier to spend a hundred million dollars on employees internally to do a project than to spend $100 million to buy a company. The dollars are treated differently. The approval processes are different. The cultural buy-in processes are different. Many acquisitions do fail. When an acquisition fails, it's somehow louder and more embarrassing than when some new product effort you've spun up doesn't quite work out. There are lots of internal reasons, some somewhat justified and some less so, that they haven't been doing it. Just from an economic point of view, it seems like it makes sense to see more acquisitions than we've seen.
Dwarkesh Patel
Why did you leave?
Nat Friedman
As much as I loved Microsoft and certainly as much as I love GitHub, I truly still feel tremendous love for GitHub and everything that it means to the people who use it. I didn't really want to be a part of a giant company anymore. Building Copilot was an example of this. It wouldn't have been possible without OpenAI and Microsoft and GitHub, but building it also required navigating this really large group of people between Microsoft and OpenAI and GitHub. You reach a point where you're spending a ton of time just navigating and coordinating lots of people. I just find that less energizing. My enthusiasm for that was not as high.
I was torn about it because I truly love GitHub, the product, and there was so much more I still knew we could do. But I was proud of what we'd done. I miss the team. I miss working on GitHub. It was really an honor for me. But it was time for me to go do something. I was always a startup guy. I always liked small teams. I wanted to go back to a smaller, more nimble environment.
Dwarkesh Patel
Okay, so we'll get to it in a second. But first, I want to ask about nat.org and the 300 words, the list there, which is one of the most interesting, and I guess very Straussian list of 300 words I've seen anywhere. I'm just going to mention some of these and get some of your commentary. "You should probably work on raising the ceiling, not the floor." Why?
Nat Friedman
I say "probably." What does it mean to raise the ceiling or the floor? I observed a lot of projects that set out to raise the floor, meaning, "Gosh, we are fine, but they are not. We need to go help them with our superior prosperity and understanding of their situation." Many of those projects fail.
For example, there were a lot of attempts to bring internet to Africa by large and wealthy tech companies and American universities. I won't say they all had no effect. That's not true. But many of them were far short of successful. There were satellites. There were balloons. There were high-altitude drones. There were mesh network laptops. By perfectly well-meaning, incredibly talented people who I think did in some cases see some success, but overall probably much less than they ever hoped.
But if you go to Africa, there is internet now. The way internet got there is the technologies that we developed to raise the ceiling in the richest part of the world, which were cell phones and cell towers. In the movie Wall Street from the 80s, he's got that gigantic brick cell phone. That thing cost like 10 grand at the time. That was a ceiling-raising technology. It eventually went down the learning curve and became cheap. The cell towers and cell phones, we've got hundreds of millions or billions of them in Africa. It was that initially ceiling technology and then the force of capitalism that made it work in the end. It was not any deus ex machina technology solution that was intended to raise the floor.
I say "probably" on my website because there are some examples where people set out to raise the floor and say, "No one should ever die of smallpox again. No one should ever die of Guinea worm again." And they succeed. I wouldn't want to discourage that from happening. But on balance, we have too many attempts to do that that look good, feel good, sound good, and don't matter. In some cases, they have the opposite of the effect they intend to.
Dwarkesh Patel
Here's another one. This is under the image section. "In many cases, it's more accurate to model the world as 500 people than 8 billion." Here's my question. What are the 8 billion minus 500 people doing? Why are there only 500 people?
Nat Friedman
I don't know exactly. It's a good question. I ask people that a lot. The more I've done in life, the more I've been mystified by this. You think, "Oh, somebody must be doing X." Then you hear there are a few people doing X, and then you look into it. They're not actually doing X. They're doing some version of it that's not that. All the best moments in life occur when you find something that to you is totally obvious that clearly somebody must be doing but no one is doing. Mark Zuckerberg says this about founding Facebook. "Surely the big companies will eventually do this and create this social and identity layer on the internet. Microsoft will do this." But no, none of them were, and he did it.
So what are they doing? The first thing is many people throughout the world are optimizing local conditions. They're working in their town, their community, doing something there. The set of people that are thinking about global conditions is just naturally narrowed by the structure of the economy. That's number one.
Number two is most people really are quite memetic. I think we all are, including me. We get a lot of ideas from other people. Our ideas are not our own. We kind of got them from somebody else. It's copy-paste. You have to work really hard not to do that and to be decorrelated. I think this is even more true today because of the internet. I don't know if Albert Einstein, as a patent clerk, wouldn't he have just been on Twitter getting the same ideas as everybody else? Would he have his decorrelated ideas? The internet has correlated us more.
The exception would be really disagreeable people who are just naturally disagreeable. I think the future belongs to the autists in some sense because they don't care what other people think as much. Those of us on the spectrum in any sense are in that category.
We have this belief that the world is efficient and it isn't. The world is so fractal and so interesting. Herculaneum papyri, right? This corner of the world that I find totally fascinating, but I don't have any anticipation that 8 billion people should be thinking about that, or that that should be a priority for everyone.
Dwarkesh Patel
Here's another one. "Large-scale engineering projects are more soluble in IQ than they appear." Does that make you think that the impact of AI tools like Copilot will be bigger or smaller because of engineering? One way to look at Copilot is that its IQ is probably less than the average engineer, so maybe it'll have less impact.
Nat Friedman
I think it increases the productivity of the average engineer to bring them higher up. I think it increases the productivity of the best engineers as well. A lot of the people I consider to be the best engineers tell me that they find it increases their productivity a lot. AI is going to completely change things.
It's really interesting how so much of what's happened in AI has been soft fictional work. You have Midjourney. You have copywriting. You have Claude from Anthropic, which is so literary. It writes poetry so well. Except for Copilot, which is this real hard area where the code has to compile. It has to be syntactically correct. It has to work and pass the tests. We see this steady improvement curve where now already on average more than half of the code is written by Copilot. When it shipped, it was like low 20s. It's really improved a lot as the models have gotten better and the prompting has gotten better. I don't see any reason why that won't be like 95%. It seems very likely to me.
I don't know what that world looks like. It seems like we might have more special purpose and less general purpose software. Right now we use general purpose tools like spreadsheets a lot. Part of that has to do with the cost of creating software. Once you have much cheaper software, do you create more special purpose software? That's a possibility. Every company gets a custom piece of code in a sense. Maybe that's the kind of future we're headed towards. I think we're going to see enormous amounts of change in software development.
Dwarkesh Patel
Another one. "The cultural prohibition on micromanagement is harmful. Great individuals should be fully empowered to exercise their judgment." The rebuttal to this is that if you micromanage, you prevent people from learning and developing their own judgment.
Nat Friedman
Imagine you go into some company. They hire you and you do a great job with the first project they give you. Everyone's really impressed. "Man, Dwarkesh, he made the right decisions. He worked really hard. He figured out exactly what needed to be done and he did it extremely well." Over time, you get promoted into positions of greater authority. The reason the company is doing this is they want you to do that again but at bigger scale. Do it again but 10 times bigger, the whole product instead of part of the product, or 10 products instead of one. The company is telling you, "You have great judgment and we want you to exercise that at a greater scale."
Meanwhile, the culture is telling you, as you get promoted, you should suspend your judgment more and more and defer your judgment to your team. There's some equilibrium there and I think we're just out of equilibrium right now where the cultural prohibition is too strong. Maybe in the 80s I would have felt the other side of this, that we have too much micromanagement. The other problem that people have is that they don't like micromanagement because they don't want bad managers to micromanage.
So you have some bad managers. They have no expertise in the area. They're just kind of people managers and they're starting to micromanage something that they don't understand, where their judgment is bad. My answer to that is: stop empowering bad managers. Don't have them. Just don't have bad managers. Promote and empower people who have great judgment and do understand the subject matter they're working on.
If I work for you and I just know you have better judgment, and you come in and say, "Nat, you're launching the scroll thing and I think you've got the file format wrong. Here's how you should do it," I would welcome that, even though it's micromanagement, because it's going to make us more successful, and I'm going to learn something from that. I know your judgment is better than mine in this case. Or at least we're going to have a conversation about it, and we're both going to get smarter.
So on balance, there are cases where people have excellent judgment, and we should encourage them to exercise it. Sometimes things will go wrong when you do that, but on balance, you will get far more excellence out of it. We should empower individuals who have great judgment. There's a quote about Napoleon that if he could have been in every single theater of every single battle he was part of, he would have never lost a battle.
Dwarkesh Patel
I was talking to somebody who worked with you at GitHub, and she emphasized to me that even the applications being shipped out to engineers, how much of the actual suggestions and design came from you directly. That's remarkable to me that as CEO you would...
Nat Friedman
Yeah, and you can probably find people you can talk to who think that was terrible. But the question is always, does that scale? The answer is it does not scale. It doesn't. But the set of people who really do have great judgment – the experience I had as CEO is I was terrified all the time that there was someone in the company who really knew exactly what to do and had excellent judgment, but because of cultural forces, that person wasn't empowered. That person was not allowed to exercise their judgment and make decisions.
When I would think and talk about this, the fear was that they were in some consensus environment where their good ideas were getting whittled down by lots of conversations with other people, by politeness, and a desire not to micromanage. We were ending up with some kind of average thing. I would rather have more high variance outcomes where you either get something excellent because it is the expressed vision of a really good auteur, or you get a disaster and it didn't work. Now you know it didn't work and you can start over. I would rather have those more high variance outcomes. I think it's a worthy trade.
Dwarkesh Patel
Okay. Let's talk about AI. What percentage of the economy is basically text-to-text?
Nat Friedman
It's a good question. We've done the Bureau of Labor Statistics analysis of this. It's not the majority of the economy or anything like that. We're in the low double-digit percentages. The thing that's hard to predict is what happens over time as the cost of text-to-text goes down. I don't know what that's going to do. But there's plenty of revenue to be got now.
One way you can think about it: we have all these benchmarks for machine learning models – Lambda and this and that. Those are really only useful and only exist because we haven't deployed the models at scale. We don't have a sense of what they're actually good at. The best metric would probably be something like: what percentage of economic tasks can they do? On a gig marketplace like Upwork, for example, what fraction of Upwork jobs can GPT-4 do? My guess is extremely low right now, autonomously. But over time, it will grow.
The question is, what does that do for Upwork? It's probably a $5 billion GMV marketplace, something like that. Does it grow to $15 billion or $50 billion? Does it shrink because the cost of text-to-text tasks goes down? I don't know. My bet would be that we find more and more ways to use text-to-text to advance progress, so overall there's a lot more demand for it. But we'll see.
Dwarkesh Patel
At what point does it happen? GPT-3 has been a rounding error in terms of overall economic impact. Does it happen with GPT-4 or GPT-5, where we see billions of dollars of usage?
Nat Friedman
I've got early access to GPT-4 and I've gotten to use it a lot. I honestly can't tell you the answer to that because it's so hard to discover what these things can do that the prior ones couldn't do. I was just talking to someone last night who told me GPT-4 is actually really good at Korean and Japanese, and GPT-3 is much worse at those. So it's a real step change for those languages. I think people didn't know how good GPT-3 was until it got instruction tuned for ChatGPT and was put out in that format. You can imagine the pre-trained models as unrefined crude oil. Once they've been RLHFed and trained and put out into the world, people can find the value.
Dwarkesh Patel
What part of the AI narrative is wrong in the over-optimistic direction?
Nat Friedman
The probably over-optimistic case from both the people who are fearful and from people who are expecting great economic benefits is that we're definitely in this realm of diminishing returns from scale. For example, GPT-4 is, my guess is, two orders of magnitude more expensive to train than GPT-3, but clearly not two orders of magnitude more capable. Is it two orders of magnitude more economically valuable? That would also surprise me. When you're in these sigmoids where you go up this exponential and then start to asymptote, it can be difficult to tell if that's going to happen.
The idea that we might not run into hard problems, or that scaling will continue to be worth it on a dollar basis, are reasons to be a little bit more pessimistic than the people who have high certainty of GDP increasing by 50% per month or something like that, which I think some people are predicting. But on the whole, I'm very optimistic. You're asking me to make the bear case for something I'm very bullish about.
Dwarkesh Patel
No, that's why I asked you to make the bear case. I want to ask you about these foundation models. What is the stable equilibrium? How many of them will there be? Will it be a sort of oligopoly like Uber and Lyft?
Nat Friedman
I think there will probably be wide-scale proliferation. The structural forces that are pro-proliferation are stronger. The pro-proliferation case: they're actually not that hard to train. The best practices will promulgate. You can write them down on a couple sheets of paper. To the extent that secrets are developed that improve training, those are relatively simple and get copied around easily. Number two, the data is mostly public, mostly from the internet. Number three, the hardware is mostly commodity, and it's improving quickly and getting much more efficient. There are a lot of techniques that overcome the challenges. Some of these labs potentially have 50%, 100%, 200% training efficiency improvement techniques. There's a lot of low-hanging fruit on the technique side.
We're seeing it happen this week and this year – a lot of proliferation. The only case against proliferation is concentration because of training costs. I don't know that that's true. I don't have confidence that the trillion-dollar model will be much more valuable than the $100 billion model, or that it will even be necessary to spend a trillion dollars training it. There might be so many techniques available for improving efficiency. How much are you willing to spend on researcher-defined techniques if you're willing to spend a trillion on training? That's a lot of bounties for new techniques. Some smart people are going to take those bounties.
Dwarkesh Patel
How different will these models be? Will it be everybody chasing the same marginal improvement, leading to the same marginal capabilities, or will they have entirely different repertoires of skills and abilities?
Nat Friedman
Right now, back to the memetic point, they're all pretty similar. Basically the same rough techniques. What's happened is an alien substance has landed on Earth and we are trying to figure out what we can build with it. We're in this multiple overhangs situation. We have a compute overhang where there's much more compute in the world than is currently being used to train models – much, much more. The biggest models are trained on maybe 10-ish thousand GPUs, but there are millions of GPUs.
Then we have a capability and technique overhang where there are lots of good ideas coming out and we haven't figured out how best to assemble them all together, but that's just a matter of time until people do that. Those capabilities haven't reached the tinkerers of the world because many of them are in the hands of the labs. That is where the new discoveries come from – what can this thing actually do? Until you get your hands on it, you don't really know.
I think OpenAI were themselves surprised by how explosively ChatGPT has grown. I don't think they put ChatGPT out expecting that to be the big announcement. I think they thought GPT-4 was going to be their big announcement. It still probably is and will be big. But ChatGPT really surprised them. It's hard to predict what people will do with it, what they'll find valuable, and what works. You need tinkerers. It goes from hardware to researchers to tinkerers to products. That's the cascade.
Dwarkesh Patel
When I was scheduling my interview with Ilya, it was originally supposed to be around the time that ChatGPT came out. Their conference person told me, "Just so you know, this interview will be scheduled around the time. We're going to make a minor announcement. It's not the thing you're thinking. It's not GPT-4. It's just a minor thing." So they didn't expect what it ended up being.
Have incumbents gotten smarter than before? It seems like Microsoft was able to integrate this new technology very well.
Nat Friedman
There have been two really big shifts in the way incumbents behave in the last 20 years that I've seen. The first is that it used to be incumbents got disrupted by startups all the time. You had example after example in the minicomputer, microcomputer era. Then Clay Christensen wrote *The Innovator's Dilemma*. Everyone read it and said, "Disruption is this thing that occurs, and we have this innovator's dilemma where we get disrupted because the new thing is cheaper and we can't let that happen." They became determined not to let that happen, and they mostly learned how to avoid it. They learned you have to be willing to do some cannibalization and set up separate sales channels for the new thing. We've had a lot of stability in incumbents for the last 15 years or so. That's my theory.
The second major change: they are paying a ton of attention to AI. If you look at prior platform revolutions like cloud, mobile, internet, web, PC, all the incumbents derided the new platform and said, "No one's going to use web apps. Everyone will use full desktop apps, rich applications." There was always this sort of laughing at the new thing. The iPhone was laughed at by incumbents. That is not happening at all with AI. We may be at peak hype cycle and about to enter the trough of despair. I kind of don't think so. People are taking it seriously. Every live player CEO is adopting it aggressively in their company. So yes, incumbents have gotten smarter.
Dwarkesh Patel
Let me ask you some questions we got from Twitter. This is from a former guest and mutual friend, Austin Vernon.
"Nat is one of those people that seems unreasonably effective. What parts of that are innate and what did he have to learn?"
Nat Friedman
That's very nice of Austin to say. I think I just have a high willingness to try things and get caught up in new projects, and then I don't want to stop doing it. I have a relatively low activation energy to try something. I'm willing to sort of impulsively jump into stuff. Many of those things don't work, but enough of them do that I've been able to accomplish a few things.
The other thing I would say, to be honest with you, is that I do not consider myself accomplished or successful. My self-image is that I haven't really done anything of tremendous consequence. I don't feel like I have this giant bed of achievements that I can go to sleep on every night. I'm kind of an insecure overachiever. I don't really feel good about myself unless I'm doing good work. I've tried to cultivate a forward-looking view where I try not to be incredibly nostalgic about the past. I don't keep lots of trophies or anything like that. You go into some people's offices and there are things on the wall and trophies of all the things they've accomplished. That always seemed really icky to me.
Dwarkesh Patel
Is that why you took down your blog?
Nat Friedman
Yeah. I just wanted to move forward.
Dwarkesh Patel
Simeon asks for your takes on alignment. Quote: "He seems to invest both in capabilities and alignment, which is the best move under a very small set of beliefs." He's curious to hear your reasoning there.
Nat Friedman
I'm not sure capabilities and alignment end up being opposing forces. It may be that capabilities are very important for alignment. It may be that alignment is very important for capabilities. I believe AI can have tremendous benefits, but there's a small chance of really bad outcomes. Some people might think it's a large chance. The solutions, if they exist, are likely to be technical. They're probably some combination of technical and prescriptive. It's probably a piece of code in a readme file that says, "If you want to build aligned AIs, use this code and don't do this."
I think that's really important, and more people should try to actually build technical solutions. One of the big things that's missing that perplexes me is there's no open source technical alignment community. There's no one actually implementing in open source the best alignment tools. There's a lot of philosophizing and talking, and then a lot of behind-closed-doors interpretability and alignment work. Because the alignment people have this belief that they shouldn't release their work, we're going to end up in a world where there's a lot of open source pure capabilities work and no open source alignment work for a little while. Hopefully that'll change. I wanted to, on the margin, invest in people doing alignment. That seems important.
I thought Sydney was an example of this. Microsoft essentially released an unaligned AI. The world sort of said, "Hmm, it's threatening its users. That seems a little bit strange. If Microsoft can't put a leash on this thing, who can?" I think there'll be more interest in it. I hope there are open communities.
Dwarkesh Patel
That was so enduring for some reason. It threatening you just made it so much more lovable.
Nat Friedman
I think the only reason it wasn't scary is because it wasn't hooked up to anything. If it was hooked up to HR systems or could post jobs on a gig worker site or something like that, it could have been scary.
Dwarkesh Patel
Before we go, where can people learn more about the Vesuvius Challenge?
Nat Friedman
The Vesuvius Challenge is at scrollprize.org – S-C-R-O-L-L-P-R-I-Z-E dot org. Check it out. I think it's very likely that somebody listening to this could be the one who wins the grand prize and decodes the scrolls.
Dwarkesh Patel
Excellent. Okay, well, this was a true pleasure. Thanks so much for coming on the podcast.
Nat Friedman
Thanks for having me.
End of interview
Dwarkesh Patel
Hey, everybody. I hope you enjoyed that episode. Just wanted to let you know that in order to help pay for the bills associated with this podcast, I'm turning on paid subscriptions on my Substack at dwarkeshpatel.com. No important content on this podcast will ever be paywalled. So please don't donate if you have to think twice before buying a cup of coffee. But if you have the means and you've enjoyed this podcast or gotten some kind of value out of it, I would really appreciate your support.
As always, the most helpful thing you can do is to share the podcast. Send it to people you think might enjoy it. Put it on Twitter, in your group chats, etc. It just splits the world. I appreciate you listening. I'll see you next time. Cheers. Thank you.