--:-- Dwarkesh Patel: All right, Mark, thanks for coming on the podcast again.
--:-- Mark Zuckerberg: Yeah, happy to do it. Good to see you.
--:-- Dwarkesh Patel: You too. Last time you were here, you had launched Llama 3.
--:-- Mark Zuckerberg: Yeah.
--:-- Dwarkesh Patel: Now you've launched Llama 4.
--:-- Mark Zuckerberg: Well, the first version.
--:-- Dwarkesh Patel: That's right. What's new? What's exciting? What's changed?
--:-- Mark Zuckerberg: Oh, well, the whole field's so dynamic. A ton has changed since the last time that we talked. Meta AI has almost a billion people using it now, monthly. So that's pretty wild. I think that this is going to be a really big year on all of this because especially once you start getting the personalization loop going, which we're just starting to build in now, really, from both the context that all the algorithms have about what you're interested in, your feed, all your profile information, all the social graph information, but also just what you're interacting with the AI about. I think that's going to be the next thing that's super exciting. So we're really big on that.
--:-- The modeling stuff continues to make really impressive advances too, as you know. The Llama 4 stuff, I'm pretty happy with the first set of releases. We announced four models and we released the first two, the Scout and Maverick ones, which are the mid-size models, mid-size to small. Actually, the most popular Llama 3 model was the 8 billion parameter model. So we've got one of those coming in the Llama 4 Series 2. Our internal codename for it is Little Llama. But that's coming probably over the coming months.
--:-- The Scout and Maverick ones, they're good. They're some of the highest intelligence per cost that you can get of any model that's out there. Natively multimodal, very efficient, run on one host, designed to be very efficient and low latency for a lot of the use cases that we're building for internally. And that's our whole thing. We basically build what we want, and then we open source it so other people can use it too. So I'm excited about that.
--:-- I'm also excited about the Behemoth model, which is coming up. That's going to be our first model that is at the frontier. It's more than 2 trillion parameters. As the name says, it's quite big. So we're trying to figure out how we make that useful for people. It's so big that we've had to build a bunch of infrastructure just to be able to post-train it ourselves. And we're trying to wrap our head around how the average developer out there is going to be able to use something like this, and how do we make it so it can be useful for distilling into models that are of reasonable size to run? Because you're obviously not going to want to run something like that in a consumer model.
--:-- But yeah, there's a lot to go. As you saw with the Llama 3 stuff last year, the initial Llama 3 launch was exciting. And then we just built on that over the year. 3.1 was when we released the 405 billion model. 3.2 is when we got all the multimodal stuff in. So we basically have a roadmap like that for this year too. So a lot going on.
--:-- Dwarkesh Patel: I'm interested to hear more about it. There's this impression that the gap between the best closed source and the best open source models has increased over the last year. I know the full family of Llama 4 models is not yet out, but Llama 4 Maverick is 35 on Chatbot Arena. And on a bunch of major benchmarks, it seems like o4 Mini or Gemini 2.5 Flash are beating Maverick, which is in the same class. What do you make of that impression?
--:-- Mark Zuckerberg: Yeah, well, okay. There's a few things. I actually think that this has been a very good year for open source overall. If you go back to where we were last year, what we were doing with Llama was the only real super innovative open source model. Now you have a bunch of them in the field. And I think in general, the prediction that this would be the year where open source generally overtakes closed source, the most used models out there, is generally on track to be true. I think the thing that's been an interesting surprise, positive in some ways, negative in others, but overall good, is that it's not just Llama. There are a lot of good ones out there. So I think that's quite good.
--:-- Then there's the reasoning phenomenon, which you are alluding to with talking about o3 and o4 and some of the other models. I do think that there is this specialization happening where if you want a model that is the best at math problems or coding or different things like that, these reasoning models with a lot of the ability to consume more test time or inference time compute in order to provide more intelligence is a really compelling paradigm. And we're going to do that too. We're building a Llama 4 reasoning model and that'll come out at some point.
--:-- For a lot of the things that we care about, latency and good intelligence per cost are actually much more important product attributes. If you're primarily designing for a consumer product, people don't necessarily want to wait half a minute to think through the answer. If you can provide an answer that's generally quite good in half a second, then that's great and that's a good trade-off. I think both of these are going to end up being important directions. I am optimistic about integrating the reasoning models with the core language models over time. I think that's the direction that Google has gone in with some of the more recent Gemini models, and I think that's really promising.
--:-- You also mentioned the whole chatbot arena thing, which I think is interesting. It goes to this challenge around how you do the benchmarking. How do you know what models are good for which things? One of the things that we've generally tried to do over the last year is anchor more of our models in our Meta AI product North Star use cases. The issue with both open source benchmarks and any given thing like the LM Arena stuff is that they're often skewed for a very specific set of use cases, which are often not what any normal person does in your product. They are often weighted differently from what people care about in any given product. Because of that, we've found that trying to optimize too much for that stuff has often led us astray and not led towards the highest quality products and the most usage and best feedback within Meta AI as people use our stuff.
--:-- So we're trying to anchor our North Star in the product value that people report to us, what they say they want, and their revealed preferences, using the experiences that we have. Sometimes these things don't quite line up, and I think a lot of them are quite easily gameable. On the arena, you'll see stuff like Sonnet 3.7. It's a great model, and it's not near the top. It was relatively easy for our team to tune a version of Llama 4 Maverick that was way at the top, whereas the one that we released, the pure model, actually has no tuning for that at all. So it's further down. So I think you just need to be careful with some of the benchmarks, and we're going to index primarily on the products.
--:-- Dwarkesh Patel: Do you feel like there is some benchmark which captures what you see as a North Star of value to the user, which can be objectively measured between the different models? And you're like, "I need Llama 4 to come out on top on this."
--:-- Mark Zuckerberg: Well, our benchmark is basically user value in Meta AI. But you can't compare other models.
--:-- Dwarkesh Patel: Well, we might be able to because we might be able to run other models in that and be able to tell.
--:-- Mark Zuckerberg: And I think that's one of the advantages of open source. You have a good community of folks who can poke holes at where your model is not good and where it is good. But I think the reality at this point is that all these models are optimized for slightly different mixes of things. Everyone is trying to create general intelligence, super intelligence, whatever you call it. AI that can lead towards a world of abundance where everyone has these superhuman tools to create whatever they want. That leads to dramatically empowering people and creating all these economic benefits. I think that's what a lot of the labs are going for.
--:-- But there's no doubt that different folks have optimized towards different things. The Anthropic folks have really focused on coding and agents around that. The OpenAI folks have gone a little more towards reasoning recently. And I think there is a space which, if I had to guess, will end up being the most used one: quick, very natural to interact with, very natively multimodal, that fits into your day in the ways that you want to interact with it.
--:-- You got a chance to play around with the new Meta AI app that we're releasing. One of the fun things that we put in there is the demo for the full duplex voice. It's early. There's a reason why we haven't made that the default voice model in the app. But there's something about how naturally conversational it is that is really fun and compelling. Being able to mix that in with the right personalization is going to lead towards a product experience where, if you go forward a few years, we're just going to be talking to AI throughout the day about different things that we're wondering. You'll have your phone, you'll talk to it. You'll talk to it while you're browsing your feed apps. It'll give you context about different stuff. It'll help you as you're interacting with people in messaging apps. Eventually, I think we'll walk through our daily lives and we'll either have glasses or other kinds of AI devices and just be able to seamlessly interact with it all day long. So that's the North Star. Whatever the benchmarks are that lead towards people feeling like the quality is what they want to interact with, that is the thing that is ultimately going to matter the most to us.
--:-- Dwarkesh Patel: I got a chance to play around with both Orion and also the Meta AI app, and the voice mode was super smooth. It was quite impressive. On the point of what the different labs are optimizing for, to steelman their view, I think a lot of them think that once you fully automate software engineering and AI research, then you can kick off an intelligence explosion where you have millions of copies of these software engineers, replicating the research that happened between Llama 1 and Llama 4, that scale of improvement, again, in a matter of weeks or months rather than years. And so it really matters to just close the loop on the software engineer, and then you can be the first to ASI. What do you make of that?
--:-- Mark Zuckerberg: Well, I personally think that's pretty compelling. And that's why we have a big coding effort too. We're working on a number of coding agents inside Meta. We're not really an enterprise software company, we're primarily building it for ourselves. So we go for the specific goal. We're not trying to build a general developer tool. We're trying to build a coding agent and an AI research agent that advances Llama research specifically, and is fully plugged into our tool chain and all this. I think that's important and is going to end up being an important part of how this stuff gets done.
--:-- I would guess that sometime in the next 12 to 18 months, we'll reach the point where most of the code that's going towards these efforts is written by AI. And I don't mean autocomplete. Right today you have good autocomplete, where you start writing something and it can complete the section of code. I'm talking more like you give it a goal, it can run tests, it can improve things, it can find issues. It writes higher quality code than the average very good person on the team already. I think that's going to be a really important part of this for sure.
--:-- But I don't know if that's the whole game. I think that's going to be a big industry, and an important part of how AI gets developed, but I think there are still other things. One way to think about this is this is a massive space. I don't think there's just going to be one company with one optimization function that serves everyone as best as possible. I think there are a bunch of different labs that are going to be doing leading work towards different domains. Some are going to be more enterprise focused or coding focused. Some are going to be more productivity focused. Some are going to be more social or entertainment focused. Within the assistant space, some are going to be much more informational or productivity. Some are going to be more companion focused. There's going to be a lot of stuff that's just fun and entertaining and shows up in your feed.
--:-- So I think there's a huge amount of space. Part of what's fun about this is that going towards this AGI future, there are a bunch of common threads for what needs to get invented. But there are a lot of things at the end of the day that need to get created. I think you'll start to see a little more specialization between the groups, if I had to guess.
--:-- Dwarkesh Patel: It's really interesting to me that you basically agree with the premise that there will be an intelligence explosion and something like superintelligence on the other end. But then if that's the case, tell me if I'm misunderstanding you, why even bother with personal assistance and whatever? Why not just get to superhuman intelligence first and then deal with everything else later?
--:-- Mark Zuckerberg: Well, I think that's just one aspect of the flywheel. Part of what I generally disagree with on the fast takeoff thing is it takes time to build out physical infrastructure. If you want to build a gigawatt cluster of compute, that just is going to take some time. It takes Nvidia a bunch of time to stabilize their new generation of systems. Then you need to figure out the networking around it. Then you need to build the building, get permitting, get the energy. Whether it's gas turbines or green energy, there's a whole supply chain of that stuff. We talked about this a bunch the last time I was on the podcast with you. Some of these are just physical world, human time things. As you start getting more intelligence in one part of the stack, you'll just run into a different set of bottlenecks. That's the way that engineering always works. You solve one bottleneck, you get another bottleneck.
--:-- Another ingredient that's going to make this work well is people getting used to and learning and having a feedback loop with using the system. These systems don't tend to be the type of thing where something just shows up fully formed and then people magically fully know how to use it. There is this co-evolution that happens where people are learning how to best use these AI assistants. On the same side, the AI assistants are learning what those people care about. The developers of those AI assistants are able to make them better. And then you're also building up this base of context. So now you wake up and you're a year or two into it, and the AI assistant can reference things that you talked about a couple of years ago. That's pretty cool, but you couldn't do that if you just launched the perfect thing on day one. There's no way that it could reference what you talked about two years ago if it didn't exist two years ago.
--:-- So my view is there's this huge intelligence growth. There's a very rapid curve on the uptake of people interacting with the AI assistants and the learning feedback and data flywheel around that. And then there is also the build out of the supply chains and infrastructure and regulatory frameworks to enable the scaling of a lot of the physical infrastructure. At some level, all of those are going to be necessary, and not just the coding piece.
--:-- I guess one specific example of this that I think is interesting: even a few years ago, we had a project on our ads team to automate ranking experiments. That's a pretty constrained environment. It's not write open-ended code. It's basically look at the whole history of the company, every experiment that any engineer has ever done in the ad system, and look at what worked, what didn't, what the results were, and basically formulate new hypotheses for different tests that we should run that could improve the performance of the ad system. What we basically found was we were bottlenecked on compute to run tests based on the number of hypotheses. It turns out that even with just the humans that we have right now on the ads team, we already have more good ideas to test than you have either compute or really cohorts of people to test them with. Even if you have three and a half billion people using your products, you still want each test to be statistically significant. So it needs to have hundreds of thousands or millions of people. There's only so much throughput that you can get on testing through that. So we're already at the point where we can't really test everything that we want.
--:-- So just being able to test more things is not necessarily going to be additive. We need to get to the point where the average quality of the hypotheses that the AI is generating is better than what the best humans on the team have been able to do before it'll even be marginally useful. So I think we'll get there, and pretty quickly, but it's not like, "Okay, cool, the thing can write code, all of a sudden everything is just improving massively." There are real world constraints. First it needs to be able to do a reasonable job. Then you need to have the compute and the people to test. And then over time, as the quality creeps up, are we here in five or ten years where no set of people can generate a hypothesis as good as the AI system? I don't know, maybe. In that world, obviously that's going to be how all the value is created, but that's not the first step.
--:-- Dwarkesh Patel: So if you buy this view that this is where intelligence is headed, the reason to be bullish on Meta is obviously that you have all this distribution, which you can also use to learn more things that can be useful for training.
--:-- Mark Zuckerberg: Yeah.
--:-- Dwarkesh Patel: You mentioned the Meta app now has a billion active users.
--:-- Mark Zuckerberg: Not the app. The app is a standalone thing that we're just launching now. I think it's fun for people who want to use it. It's a cool experience. We can talk about that. We're experimenting with some new ideas in there that I think are novel and worth talking through. But I'm talking mostly about our apps. Meta AI is actually most used in WhatsApp.
--:-- Dwarkesh Patel: Got it.
--:-- Mark Zuckerberg: So it's in WhatsApp, mostly used outside of the US. We just passed a hundred million people in the US, but it's not the primary messaging system in the US, iMessage is. So I think people in the US probably tend to underestimate the Meta AI use somewhat. But it's also part of the reason why the standalone app is going to be so important. The US is, for a lot of reasons, one of the most important countries. The fact that WhatsApp is the main way that people are using Meta AI and that's not the main messaging system in the US means that we need another way to build a first class experience that's in front of people.
--:-- Dwarkesh Patel: I guess, to finish the question, the bearish case would be that if the future of AI is less about just answering your questions and more so just being a virtual coworker, it's not clear how Meta AI inside of WhatsApp gives you the relevant training data to make a fully autonomous programmer, remote worker. So in that case, does it not matter that much who has more distribution right now with LLMs?
--:-- Mark Zuckerberg: Well, again, I just think that there are going to be different things. If you were sitting at the beginning of the development of the internet, and it's like, "What's going to be the main internet thing? Is it going to be knowledge work? Or is it going to be massive consumer apps?" It's like, I don't know, you get both. You don't have to choose one. The world is big and complicated. Does one company build all that stuff? I think normally the answer is no.
--:-- But to your question, people do not code in WhatsApp for the most part, and I don't foresee that people starting to write code in WhatsApp is going to be a major use case. Although I do think that people are going to ask AI to do a lot of things that result in the AI coding without them necessarily knowing it. So that's a separate thing. But we do have a lot of people who are writing code at Meta and they use Meta AI. We have this internal thing that we call Metamate, and a number of different coding and AI research agents that we're building around that. And that has its own feedback loop and I think can get good for accelerating those efforts.
--:-- But again, I just think that there are going to be a bunch of things. I think AI is almost certainly going to unlock this massive revolution in knowledge work and code. I also think it's going to be the next generation of search and how people get information and do more complex information tasks. I also think it's going to be fun. I think people are going to use it to be entertained. A lot of the internet is memes and humor. We have this amazing technology at our fingertips. It is sort of amazing and kind of funny when you think about it, how much of human energy just goes towards entertaining ourselves and pushing culture forward and finding humorous ways to explain cultural phenomenon that we observe. I think that's almost certainly going to be the case in the future.
--:-- If you look at the evolution of things like Instagram and Facebook, if you go back 10, 15, 20 years ago, it was text. Then we all got phones with cameras. Most of the content became photos. Then the mobile networks got good enough that if you wanted to watch a video on your phone, it wasn't just buffering. So over the last 10 years, most of the content has moved towards video. Most of the time spent in Facebook and Instagram is video. But do you think in five years we're just going to be sitting in our feeds consuming media that's video? No, it's going to be interactive. You'll be scrolling through your feed and there will be content that maybe looks like a reel to start, but then you talk to it or you interact with it and it talks back or it changes what it's doing, or you can jump into it like a game and interact with it. And that's all going to be AI.
--:-- So my point is there's all these different things. And I guess we're ambitious, so we're working on a bunch of them. But I don't think any one company is going to do all of it.
--:-- Dwarkesh Patel: Okay. So on this point of AI generated content or AI interactions, already people have meaningful relationships with AI therapists, AI friends, maybe more. And this is just going to get more intense as these AIs become more unique and more personable, more intelligent, more spontaneous and funny and so forth. How do we make sure people are going to have relationships with AIs? How do we make sure that these are healthy relationships?
--:-- Mark Zuckerberg: Well, I think there are a lot of questions that you only really can answer as you start seeing the behaviors. So probably the most important upfront thing is just to ask that question and care about it at each step along the way. But I think being too prescriptive upfront and saying, "We think these things are not good," often cuts off value. People use stuff that's valuable for them. One of my core guiding principles in designing products is that people are smart. They know what is valuable in their lives. Every once in a while, something bad happens in a product and you want to make sure that you design your products well to minimize that. But if you think that something someone is doing is bad and they think it's really valuable, most of the time in my experience, they're right and you're wrong. You just haven't come up with the framework yet for understanding why the thing you're doing is valuable and helpful in their life.
--:-- So that's the main way that I think about it. I do think that people are going to use AI for a lot of these social tasks. Already one of the main things that we see people using Meta AI for is talking through difficult conversations that they need to have with people in their life. It's like, "I'm having this issue with my girlfriend, help me have this conversation," or "I need to have this hard conversation with my boss at work, how do I have that conversation?" That's pretty helpful. And then as the personalization loop kicks in and the AI starts to get to know you better and better, I think that will just be really compelling.
--:-- One thing just from working on social media for a long time is the stat that I always think is crazy. The average American, I think, has fewer than three friends, three people they'd consider friends. And the average person has demand for meaningfully more. I think it's like 15 friends or something. I guess there's probably some point where you're like, "I'm just too busy, I can't deal with more people." But the average person wants more connectivity and connection than they have. So there's a lot of questions that people ask about stuff like, "Is this going to replace in-person connections or real life connections?"
--:-- Mark Zuckerberg: And my default is that the answer to that is probably no. I think there are all these things that are better about physical connections when you can have them. But the reality is that people just don't have the connection and they feel more alone a lot of the time than they would like.
--:-- So I think that a lot of these things that today there might be a little bit of a stigma around, over time we will find the vocabulary as a society to be able to articulate why that is valuable, why the people who are doing these things are rational for doing it, and how it is adding value to their lives.
--:-- But I also think the field is very early. There are a handful of companies doing virtual therapist stuff, virtual girlfriend type stuff, but it's very early. The embodiment in the things is pretty weak. A lot of them, you open it up and it's just an image of the therapist or the person you're talking to. Sometimes there's some very rough animation, but it's not like an embodiment.
--:-- You've seen the stuff we're working on in Reality Labs with the Codec Avatars, where it feels like a real person. I think that's where it's going. You'll be able to have an always-on video chat where the AI will be able to use gestures too. More than half of communication when you're actually having a conversation is not the words you speak. It's all the nonverbal stuff.
--:-- Dwarkesh Patel: I did get a chance to check out Orion the other day, and I thought it was super impressive. I'm mostly optimistic about the technology just because generally I'm, as you mentioned, libertarian about if people are doing something, probably they think it's good for them. Although I actually don't know if it's the case that if somebody is using TikTok, they would say they're happy with how much time they're spending on it.
--:-- I'm mostly optimistic about it in the sense that if we're going to be living in this future world of AGI, we need to be upgrading our capabilities as well with tools like this. And generally, there can be more beauty in the world if you can see Studio Ghibli everywhere.
--:-- But I was worried that one of the flagship use cases your team showed me was me sitting at the breakfast table and on the periphery of my vision is just a bunch of reels scrolling by. Maybe in the future, my AI girlfriend is on the other side of the screen. I am worried that we're just removing all the friction between getting totally reward hacked by our technology. How do we make sure this is not what ends up happening in five years?
--:-- Mark Zuckerberg: Again, I think people have a good sense of what they want. That experience you saw was a demo just to show multitasking and holograms. I agree that I don't think the future is having stuff competing for your attention in the corner of your vision all the time. I don't think people would like that too much.
--:-- It's one of the things we're really mindful of as we design these glasses. Probably the number one thing glasses need to do is get out of the way and be good glasses. As an aside, I think that's part of the reason why the Ray-Ban Meta product has done so well. It's great for listening to music, taking phone calls, taking photos and videos, and the AI is there when you want it. But when you don't, it's a great looking pair of glasses that people like and it gets out of the way well. I would guess that's going to be a very important design principle for the augmented reality future.
--:-- The main thing I see here is that it's kind of crazy that for how important the digital world is in all of our lives, the only way we can access it is through these physical digital screens. A phone, a computer, a big TV — it's like this huge physical thing. It just seems like we're at the point with technology where the physical and the digital world should really be fully blended. That's what the holographic overlay allows you to do.
--:-- But I agree, a big part of the design principles around that are going to be about interacting with people and bringing digital artifacts into those interactions seamlessly. If I want to show you something here, here's a screen. You can interact with it, it can be 3D, we can play with it. You want to play a card game? Here's a deck of cards. Two of us are here physically, you have a third friend who's just hologramming in, and they can participate too.
--:-- In that world, just like you don't want your physical space to be cluttered — it wears on you psychologically — I don't think people are going to want the digital physical space to feel that way either. That's more of an aesthetic norm that will have to get worked out. But I think we'll figure that out.
--:-- Dwarkesh Patel: Going back to the AI conversation, you mentioned how big of a bottleneck physical infrastructure can be. Related to other open source models like DeepSeek and so forth — DeepSeek right now has less compute than a lab like Meta, and you could argue it's competitive with the Llama models. If China is better at physical infrastructure, industrial scale-ups, getting more power and more data centers online, how worried are you that they might beat us here?
--:-- Mark Zuckerberg: I think it's a real competition. You're seeing the industrial policies really play out. I think China is bringing online more power. Because of that, the US really needs to focus on streamlining the ability to build data centers and produce energy, or I think we will be at a significant disadvantage.
--:-- At the same time, I think some of the export controls on things like chips are clearly working in a way. There was all the conversation with DeepSeek about how they did all these very impressive low-level optimizations. The reality is they did, and that is impressive. But then you ask, why did they have to do that when none of the American labs did it? It's because they're using partially nerfed chips that are the only thing Nvidia is allowed to sell in China because of the export controls.
--:-- DeepSeek basically had to spend a bunch of their calories and time doing low-level infrastructure optimizations that the American labs didn't have to do. They produced a good result on text — DeepSeek is text only. The infrastructure is impressive, the text result is impressive. But every new major model that comes out now is multimodal — image, voice — and theirs isn't. I don't think it's because they're not capable of doing it. I think they had to spend their calories on these infrastructure optimizations to overcome the export controls.
--:-- When you compare Llama 4 with DeepSeek, our reasoning model isn't out yet, so the R1 comparison isn't clear yet. But we're effectively in the same ballpark on all the tech stuff DeepSeek is doing, but with a smaller model. It's much more efficient per cost per intelligence with what we're doing for Llama on text. And all the multimodal stuff we're effectively leading at, and it just doesn't exist in their stuff. So I think the Llama 4 models, when you compare them to what they're doing, are good. I think people are going to prefer to use the Llama 4 models.
--:-- But there is this interesting contour where it's clearly a good team doing stuff over there, and I think you're right to ask about the accessibility of power, compute, and chips, because the work you're seeing the different labs do is somewhat downstream of that.
--:-- Dwarkesh Patel: So Sam Altman recently tweeted that OpenAI is going to release an open source reasoning model. I think part of the tweet was that they will not do anything silly like say that you can only use it if you have less than 700 million users. DeepSeek has the MIT license, whereas Llama, I think a couple of the contingencies in the Llama license require you to say "built with Llama" on applications using it, or any model that you train using Llama has to begin with the word "Llama." What do you think about the license? Should it be less onerous for developers?
--:-- Mark Zuckerberg: We've basically pioneered the open source LLM things. I don't consider the license to be onerous. When we started pushing on open source, it was a big debate in the industry: Is this even a reasonable thing to do? Can you do something that is safe and trustworthy with open source? Will open source ever be competitive enough that anyone will even care?
--:-- A lot of the hard work from the teams at Meta — the Llama models were the ones that broke open this whole open source AI thing in a huge way. We were very focused on, if we're going to put all this energy into it, then at a minimum, if you're going to have these large cloud companies like Microsoft, Amazon, and Google turn around and sell our model, we should at least be able to have a conversation with them before they do that, around what kind of business arrangement we should have.
--:-- Our goal with the license isn't to stop people from using the model. We just think, if you're one of those companies or if you're Apple, come talk to us about what you want to do and let's find a productive way to do it together. I think that's generally been fine.
--:-- If the whole open source part of the industry evolves in a direction where there are a lot of other great options and the license ends up being a reason why people don't want to use Llama, then we'll have to reevaluate the strategy. But I just don't think we're there. In practice, we haven't had companies come to us and say, "We don't want to use this because your license says if you reach 700 million people, you have to come talk to us."
--:-- It's more something we've heard from open source purists: "Is this as clean of an open source model as you'd like it to be?" That debate has existed since the beginning of open source with all the GPL license stuff versus other things. Does it need to be the case that anything that touches open source has to be open source, or can people just take it and use it in different ways?
--:-- If you're spending many, many billions of dollars training these models, I think asking other companies that are also huge and similar in size, who can easily afford to have a relationship with us, to talk to us before they use it — that seems like a pretty reasonable thing.
--:-- Dwarkesh Patel: If it turns out that other models are also good — there's a bunch of good open source models, so that part of your mission is fulfilled, and maybe other models are better at coding — is there a world where you just say, "The open source ecosystem is healthy, there's plenty of competition. We're happy to just use some other model, whether it's for internal software engineering at Meta or deploying to our apps. We don't necessarily need to build with Llama"?
--:-- Mark Zuckerberg: We do a lot of things. Let's take a step back. The reason we're building our own big models is because we want to be able to build exactly what we want. None of the other models in the world are exactly what we want. If they're open source, you can take them and fine tune them in different ways, but you still have to deal with the model architectures. They make different size tradeoffs that affect latency and inference cost.
--:-- At the scale we operate at, that stuff really matters. We made the Llama Scout and Maverick models certain sizes for a specific reason because they fit on a host and we wanted certain latency, especially for the voice models we're working on that we want to have pervade everything we're doing — from the glasses to all of our apps to the Meta AI app.
--:-- I think there's a level of control of your own destiny that you only get when you build the stuff yourself. That said, AI is going to be used in every single thing that every company does. When we build a big model, we need to choose which use cases internally we're going to optimize for. Does that mean for certain things, maybe Claude is better for building this specific development tool that this team is using? Fine, use that. We don't want to fight with one hand tied behind our back.
--:-- You also asked, would it not be important because other people are doing open source? I'm a little more worried about that. For anyone who shows up now and is doing open source now that we have done it, there's a question: Would they still be doing open source if we weren't doing it? I think there are a handful of folks who see the trend that more and more development is going towards open source. They think, "Oh crap, we need to be on this train or else we're going to lose." They have some closed model API, and increasingly a lot of developers don't want that. So you're seeing a bunch of other players start to do some work in open source, but it's unclear if it's dabbling or fundamental for them in the way it has been for us.
--:-- A good example is what's going on with Android. Android started off as the open source thing. There's not really any open source alternative. I think over time, Android has just been getting more and more closed. So if you're us, you need to worry that if we stopped pushing the industry in this direction, all these other people are maybe only doing it because they're trying to compete with us in the direction we're pushing things. They already have their revealed preference for what they would build if open source didn't exist, and it wasn't open source. I just think we need to be careful about relying on that continued behavior for the future of the technology we're going to build at the company.
--:-- Dwarkesh Patel: Another thing I've heard you mention is that it's important that the standard gets built around American models like Llama. I wanted to understand your logic there. With certain kinds of networks, it is the case that the Apple App Store has a big contingency around what it's built around. But it doesn't seem like if you built some sort of scaffold for DeepSeek, you couldn't have easily switched it over to Llama 4, especially since between generations, Llama 3 wasn't MoE and Llama 4 is. Things are changing between generations. So what's the reason for thinking things will get built out in this contingent way on a specific standard?
--:-- Mark Zuckerberg: I think these models encode values and ways of thinking about the world. We had this interesting experience early on where we took an early version of Llama and translated it into French. The feedback we got from French people was, "This sounds like an American who learned to speak French. It doesn't sound like a French person." It speaks French fine, but the way it thinks about the world seems slightly American.
--:-- There are these subtle things that get built into it. Over time, as models get more sophisticated, they should be able to embody different value sets across the world. That's a not particularly sophisticated example, but it illustrates the point. Some of the stuff we've seen in testing models coming out of China is that they have certain values encoded in them, and it's not just a light fine tune to get that to feel the way you want.
--:-- Language models have a kind of world model embedded into them with more values. Reasoning models are trained on verifiable problems, so do you need to be worried about cultural bias if your model is doing math? Probably not. The chance that some reasoning model built elsewhere is going to incept you by solving a math problem in a way that's devious seems low.
--:-- There's a whole set of different issues around coding, which is the other verifiable domain. You need to be worried about waking up one day and finding that a model tied to another government has embedded all kinds of vulnerabilities in code that the intelligence organizations of that government can then exploit. In some future version where you have some model from another country securing or building out a lot of your systems, and then all of a sudden everything is vulnerable in a way that country knows about but you don't, or it turns on a vulnerability at some point — those are real issues.
--:-- I'm very interested in studying this because one of the main things that's interesting about open source is the ability to distill models. Most people, the primary value isn't just taking a model off the shelf and running it exactly in their application. You're at least going to fine tune it or distill it into a different model. When we get to the Behemoth model, the whole value is being able to take this very high amount of intelligence and distill it down into a smaller model you're actually going to want to run.
--:-- This is the beauty of distillation. It's one of the things that has really emerged as a very powerful technique in the last year. It's worked better than most people would predict. You can basically take a model that is much bigger and take probably 90 or 95% of its intelligence and run it in something that's 10% the size. Do you get 100% of the intelligence? No, but 95% of the intelligence at 10% of the cost is pretty good for a lot of things.
--:-- The other interesting thing is with this more varied open source community where it's not just Llama — you have other models — you have the ability to distill from multiple sources. You can say, "Llama's really good at this, the architecture is really good because it's fundamentally multimodal and more inference friendly and efficient. But this other model is better at coding." You can distill from both of them and build something that's better than either of them for your own use case. That's cool.
--:-- But you do need to solve the security problem of knowing you can distill it in a way that is safe and secure. This is something we've been researching and have put a lot of time into. What we've basically come to is that anything that's language is quite fraught because there are a lot of values embedded in that. Unless you don't care about having the values from whatever model you got, you probably don't want to distill the straight language world model.
--:-- On reasoning, I think you can get a lot of the way there by limiting it to verifiable domains, running code cleanliness and security filters — whether it's the LlamaGuard open source or the CodeShield open source things we've done that allow you to incorporate different input into your models and make sure both the input and output are secure. And then a lot of red teaming to make sure the model isn't doing anything you don't want after distilling from something. With a combination of those techniques, you can probably distill on the reasoning side for verifiable domains quite securely. That's something I'm pretty confident about, and it's something we've done a lot of research around.
--:-- But this is a very big question: How do you do good distillation? There's just so much value to be unlocked. At the same time, I do think there is some fundamental bias in the different models.
--:-- Dwarkesh Patel: Speaking of value to be unlocked, what do you think the right way to monetize AI will be? Obviously digital ads are quite lucrative, but as a fraction of total GDP, it's small in comparison to all remote work. Even if you can increase its productivity and not replace work, that's still worth tens of trillions of dollars. So is it possible that ads might not be it? How do you think about this?
--:-- Dwarkesh Patel: Speaking of value to be unlocked, what do you think the right way to monetize AI will be? Obviously digital ads are quite lucrative, but as a fraction of total GDP, it's small in comparison to all remote work. Even if you can increase its productivity and not replace work, that's still worth tens of trillions of dollars. So is it possible that ads might not be it? How do you think about this?
--:-- Mark Zuckerberg: I mean, like we were talking about before, there's going to be all these different applications, and different applications tend towards different things. Ads is great when you want to offer people a free service, right? Because it's free, you need to cover it somehow. Ads solves the problem of a person not needing to pay for something and they can get something amazing for free.
--:-- And by the way, with modern ad systems, a lot of the time people think that the ads add value to the thing if you do it well. You need to be good at ranking and you need to be good at having enough liquidity of advertising inventory. So if you only have five advertisers in the system, no matter how good you are at ranking, you may not be able to show something to someone that they're interested in. But if you have a million advertisers in the system, then you're probably going to be able to find something pretty compelling if you're good at picking out the different needles in the haystack that that person's going to be interested in.
--:-- So I think that definitely has its place, but there are also clearly going to be other business models as well, including ones that just have higher costs. So it doesn't even make sense to offer them for free. There have always been business models like this. There's a reason why social media is free and ad supported. But then if you want to watch Netflix or ESPN or something, you need to pay for that. It's okay, because the content that's going into that, they need to produce it and it's very expensive for them to produce. They probably could not have enough ads in the service to make up for the cost of producing the content, so you just need to pay to access it. The trade is fewer people do it. They're talking about hundreds of millions of people using those instead of billions. So there's kind of a value switch there.
--:-- I think similar here. Not everyone is going to want a software engineer or a thousand software engineering agents or whatever it is. But if you do, that's something you are probably going to be willing to pay thousands or tens of thousands or hundreds of thousands of dollars for. So I think this just speaks to the diversity of different things that need to get created. There are going to be business models at each point along the spectrum. For the consumer piece, we definitely want to have a free thing, and I'm sure that will end up being ad supported. But I also think we're going to want to have a business model that supports people using arbitrary amounts of compute to do really even more amazing things than what it would make sense to be able to offer the free service. For that, I'm sure we'll end up having a premium service. But I think our basic values on this are we want to serve as many people in the world.
--:-- [Ad break for Lambda]
--:-- Dwarkesh Patel: How do you keep track of all these different projects, some of which we've talked about today? I'm sure there's many I don't even know about. As the CEO overseeing everything, there's a big spectrum between going to the Llama team and saying "here's the hyperparameters you should use" to just giving a mandate like "go make the AI better." And there's many different projects. How do you think about the way in which you can best deliver your value add and oversee all these things?
--:-- Mark Zuckerberg: Well, a lot of what I spend my time on is trying to get awesome people onto the teams. So there's that. And then there's stuff that cuts across teams. It's like, all right, you build Meta AI and you want to get it into WhatsApp or Instagram. It's like, okay, then now I need to get those teams to talk together. And then there's a bunch of questions like, do you want the thread for Meta AI and WhatsApp to feel like other WhatsApp threads? Or do you want it to feel like other AI chat experiences? There's different idioms for those. So I think there's all these interesting questions that need to get answered around how this stuff basically fits into all of what we're doing.
--:-- Then there's a whole other part of what we're doing, which is basically pushing on the infrastructure. If you want to stand up a gigawatt cluster, then that has a lot of implications for the way that we're doing infrastructure build-outs. It has political implications for how you engage with the different states where you're building that stuff. It has financial implications for the company in terms of, all right, there's a lot of economic uncertainty in the world. Do we go double down on infrastructure right now? And if so, what other tradeoffs do we want to make around the company? Those are things that it's tough for other people to really make those kinds of decisions.
--:-- And then I think there's this question around taste and quality, which is when is something good enough that we want to ship it. I do feel like in general I'm the steward of that for the company, although we have a lot of other people who have good taste as well who are also filters for different things. But I think those are basically the areas.
--:-- I think AI is interesting because more than some of the other stuff that we do, it is more research and model led than really product led. You can't just design the product that you want and then try to build the model to fit into it. You really need to design the model first and the capabilities that you want. Then you get some emergent properties, and then you can build some different stuff because this kind of turned out in this way. At the end of the day, people want to use the best model. So that's partially why, when we're talking about building the most personal AI, the best voice, the best personalization, and also a very smart experience with very low latency, those are the things that we need to design the whole system to build. That's why we're working on full duplex voice, why we're working on the personalization to both have good memory extraction from your interaction with AI but also be able to plug into all the other Meta systems, and why we design the specific models that we designed to have the kind of size and latency parameters that they do.
--:-- Dwarkesh Patel: Speaking of politics, there's been this perception that some tech leaders have been aligning with Trump. You and others have donated to his inaugural event and were on stage with him. And I think you settled a lawsuit which resulted in them getting $25 million. I wonder what's going on here. Does it feel like the cost of doing business with the administration? What's the best way to think about this?
--:-- Mark Zuckerberg: My view on this is, he's the president of the United States. Our default as an American company should be to try to have a productive relationship with whoever is running the government. I would do this, and we've tried to offer to support previous administrations as well. I've been pretty public with some of my frustrations with the previous administration, how they basically did not engage with us or the business community more broadly, which I think is going to be necessary to make progress on some of these things. We're not going to be able to build the level of energy that we need if you don't have a dialogue and they're not prioritizing trying to do those things.
--:-- But fundamentally, look, I think a lot of people want to write the story about what direction are people going. I just think we're trying to build great stuff. We want to have a productive relationship with people. That's how I see it, and it is also how I would guess most others see it. But obviously, I can't speak for them.
--:-- Dwarkesh Patel: You've spoken out about how you've rethought some of the ways in which you engage and defer to the government in terms of moderation stuff in the past. How are you thinking about AI governance? Because if AI is as powerful as we think it might be, the government will want to get involved. What is the most productive approach to take there? And what should the government be thinking about here?
--:-- Mark Zuckerberg: In the past, I probably just — most of the comments that I made were in the context of content moderation. It's been an interesting journey over the last 10 years on this. There have been novel questions raised about online content moderation. Some of those have led to productive new systems getting built, like our AI systems to be able to detect nation states trying to interfere in each other's elections. I think we will continue building that stuff out, and that has been positive.
--:-- Other stuff we went down some bad paths. I just think the fact-checking thing was not as effective as community notes because it wasn't an internet scale solution. There weren't enough fact checkers, and people didn't trust the specific fact checkers. You want a more robust system. So I think where we got with community notes is the right one on that.
--:-- But my point on this was more that I think historically I probably deferred a little bit too much to either the media and their critiques or the government on things that they did not really have authority over, but just as a central figure. We tried to build systems that maybe we could not have to make all of the content moderation decisions ourselves. I just think part of the growth process over the last 10 years is just okay, we're a meaningful company. We need to own the decisions that we need to make. We should listen to feedback from people but shouldn't defer too much to people who do not actually have authority over this, because at the end of the day, we're in the seat and we need to own the decisions that we make. So I think it's been a maturation process, and in some ways painful, but I think we're probably a better company for it.
--:-- Dwarkesh Patel: Will tariffs increase the cost of building data centers in the US and shift buildouts to Europe and Asia?
--:-- Mark Zuckerberg: It is really hard to know how that plays out. I think we're probably in the early innings on that, and it's very hard to know.
--:-- Dwarkesh Patel: Got it. What is your single highest leverage hour in a week? What are you doing in that hour?
--:-- Mark Zuckerberg: I don't know. Every week is a little bit different. It's probably got to be the case that the most leveraged thing that you do in a week is not the same thing each week, or else by definition, you should probably spend more than one hour doing that thing every week. It's part of the fun of both this job and the industry being so dynamic. Things really move around. The world is very different now than it was at the beginning of the year, than it was six months into the middle of last year. A lot has advanced meaningfully, and a lot of cards have been turned over since the last time that we sat down. I think that was about a year ago, right?
--:-- Dwarkesh Patel: Yeah, yeah. You were saying earlier that recruiting people is a super high leverage thing you do.
--:-- Mark Zuckerberg: It's very high leverage. Yeah, yeah.
--:-- Dwarkesh Patel: What would be possible if, you know, you talked about these models being mid-level software engineers by the end of the year. What would be possible if software productivity increased like 100x in two years? What kinds of things could we build that we can't build right now?
--:-- Mark Zuckerberg: That's an interesting question. I think one theme of this conversation is that the amount of creativity that's going to be unlocked is going to be massive. If you look at the overall arc of human society and the economy over 100 or 150 years, it's basically people going from being primarily agrarian, with most of human energy going towards just feeding ourselves, to that becoming a smaller and smaller percent. The things that take care of our basic physical needs are a smaller and smaller percent of human energy. That has led to two impacts: one is more people are doing creative and cultural pursuits, and two is that people in general spend less time working and more time on entertainment and culture.
--:-- I think that is almost certainly going to continue as this goes on. This isn't the one to two year thing of what happens when you have a super powerful software engineer, but over time, if everyone is going to have these superhuman tools to be able to create a ton of different stuff, I think you're going to get this incredible diversity. Part of it is going to be solving things that we hold up as hard problems, like curing diseases or solving different things around science or different technology that makes our lives better. But I would guess that a lot of it is going to end up being cultural and social pursuit and entertainment. I would guess that the world is going to get a lot funnier and weirder and quirkier in a way that the memes on the internet have gotten over the last 10 years. I think that adds a certain richness and depth as well. In funny ways, I think it actually helps you connect better with people because now, all day long I just find interesting stuff on the internet and send it in group chats to the people I care about who I think are going to find it funny. The media that people can produce today to express very nuanced, specific cultural ideas — it's cool. I think that'll continue to get built out, and I think it does advance society in a bunch of ways, even if it's not the hard science way of curing a disease.
--:-- I guess this is the meta social media view of the world: I think people are going to spend a lot more time doing that stuff in the future, and it's going to be a lot better and it's going to help you connect because it's going to help express different ideas. The world's going to get more complicated, but our cultural technology to express these very complicated things in a very funny little clip or something are going to just get so much better. So I think that's all great.
--:-- I tend to think that for at least the foreseeable future, this is going to lead towards more demand for people doing work, not less. Now people have a choice of how much time they want to spend working. But let me give you one interesting example of something we were talking about recently. We have almost three and a half billion people who use our services every day. One question that we've struggled with forever is how do we provide customer support? Today, you can write an email. But we've never seriously been able to contemplate having voice support where someone can just call in. That's maybe one of the artifacts of having a free service. The revenue per person isn't so high that you can have an economic model where people can call in. But also with three and a half billion people using your service every day, there'd be a massive number of people — the biggest call center in the world type of thing. It would be $10 to $20 billion a year to staff that. So we've never really thought too seriously about it because it was always just, no, there's no way this makes sense.
--:-- But now as the AI gets better, you're going to get to this place where the AI can handle a bunch of people's issues. Not all of them — maybe 10 years from now it can handle all of them. But when we're thinking about a three to five year time horizon, it'll be able to handle a bunch. Kind of like self-driving cars can handle a bunch of terrain, but in general they're not doing the whole route by themselves yet in most cases. People thought truck driving jobs were going to go away. There are actually more truck driving jobs now than there were when we started talking about self-driving cars, whatever it was, almost 20 years ago.
--:-- Going back to this customer support thing: it wouldn't make sense for us to staff out calling for everyone, but let's say the AI can handle 90% of that. Then if it can't handle it, it kicks it off to a person. If you've gotten the cost of providing that service down to one tenth of what it would have otherwise been, then maybe that actually makes sense to go do, and that would be kind of cool. So the net result is, I actually think we're probably going to go hire more customer support people. The common belief that people have is that this is clearly just going to automate jobs and all these jobs are going to go away. That has not really been how the history of technology has worked. You can create things that take away 90% of the work, and that leads you to want more people, not less.
--:-- Dwarkesh Patel: Final question. Who is the one person in the world today who you most seek out for advice?
--:-- Mark Zuckerberg: Oh man. I feel like it's part of my style that I like having a breadth of advisors, so it's not just one person. We've got a great team. There are people at the company, people on our board. There are a lot of people in the industry who are doing new stuff. There's not a single person. But it's fun. And when the world is dynamic, just having a reason to work with people you like on cool stuff — to me, that's what life is about.
--:-- Dwarkesh Patel: All right. Great note to close on. Awesome. Thanks for doing this.
--:-- Mark Zuckerberg: Yeah. Thank you.
--:-- [Outro by Dwarkesh Patel]
