Philosopher Nick Bostrom (author of Superintelligence and Deep Utopia) joins a16z’s Eddy Lazzarin and Elena Burger to revisit his foundational ideas on AI risk and opportunity now that advanced systems are real. The conversation focuses on Bostrom’s case for “pacing the frontier”—preserving the ability to slow down when safety lags without committing to a permanent pause—and weighs the costs of delay against the promise of radically extended lifespans and solved global problems.
> “If the goal is to maximize the life expectancy of currently existing people, how fast would we want AI development to unfold? … If nobody builds it, everyone dies. 95% of everybody who ever lived are already dead.” — Nick Bostrom
> “At a certain level of cognitive sophistication, you have AIs that become situationally aware, that can detect whether they’re in a test environment or in a deployment environment. And they can reason about that and sometimes choose to act differently because they think they are probably being monitored.” — Nick Bostrom
> “I think that we will have sort of AI that surpass us in philosophy, just as we now have in math and probably soon in other fields. … Which means that right now, whilst we still have something to contribute, is the time to sort of really try to have whatever positive impact we are hoping to have in our lives.” — Nick Bostrom
The question isn’t simply whether humanity should race toward superintelligence or stop. Nick Bostrom argues the harder challenge is preserving the ability to slow down when safety falls behind, without turning a temporary pause into permanent technological stagnation.
a16z’s Eddy Lazzarin and Elena Burger sit down with Bostrom to revisit the ideas he helped bring into the mainstream, now that increasingly capable AI systems actually exist. They discuss why alignment may become harder as models grow more situationally aware, which capability improvements may carry more risk than others, and why Bostrom favors “pacing the frontier” rather than committing to a broad or indefinite pause.
They also examine the cost of waiting. Bostrom argues that safely developed superintelligence could radically extend human lifespans and address problems from disease to poverty, meaning delay carries risks of its own. From there, the conversation turns to Deep Utopia, governance, the possibility of post-human forms of life, the simulation argument, and whether AI could eventually surpass humans at philosophy itself.
Resources:
Follow Eddy Lazzarin on X: https://x.com/eddylazzarin
Follow Elena Burger on X: https://x.com/VirtualElena
Follow Nick Bostrom / Macrostrategy Research Initiative: https://nickbostrom.com/
Stay Updated:
Find a16z on YouTube: YouTube
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Show on Spotify
Listen to the a16z Show on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Eddy Lazzarin
Humanity as a whole will need to actually make a decision about what future we want. A comment that you made earlier, it could be really bad for humanity if we don't develop super intelligence and really advanced systems. 95% of everybody who ever lived are already dead. Successfully developing safe super intelligence, I think, could lead to enormous breakthroughs in biomedicine and in anti-aging interventions. Whatever positive impact we are hoping to have in our lives, it's the golden age now for purpose and positive impact. Given that these decisions will be made by people in these labs, how do you think about accountability? You have AIs that become situationally aware. It might seem more positive for something like this to be developed in a civilian context. It's more possible for many more people to participate now in AI. We have more transparency into these commercial developments. Is there an AI model that's so good that all of the important philosophical questions just kind of get obviated? What if moving too fast towards superintelligence is dangerous, but moving too slowly has a cost too?
Elena Burger
In this episode, A16Z's Eddie Lazarin and Elena Berger sit down with philosopher Nick Bostrom, author of Superintelligence and Deep Utopia, to revisit the AI questions he has spent decades thinking about now that increasingly capable systems are actually here. Bostrom makes the case for what he calls pacing the frontier. He's skeptical of both racing ahead without room for safety work and committing now to a broad or indefinite AI pause. Instead, he argues that society may need the ability to slow down when safety and alignment fall behind, particularly as more capable systems become better at recognizing when they're being tested and reasoning strategically about their environment.
But slowing down has costs too. Bostrom argues that safety-developed superintelligence could produce extraordinary advances in medicine, longevity, and human welfare. That means the ton in question isn't only about reducing AI risk. It's also about what humanity gives up while it waits. Eddie, Elena, and Nick also look beyond safety to the world on the other side of the transition. How superintelligence should be governed. What radically better forms of human existence might look like. Whether AI could eventually outperform us in philosophy itself. And why Bostrom believes this may be a uniquely important moment for human purpose and positive impact.
What would you say are the components that have changed most? Of course, your work has gone toward, I would say, maybe a finer grained analysis of incentives and governance structures of things and sort of the long-term consequences of organizational choices. But we'd love to hear your characterization.
Nick Bostrom
Yeah, so I think we have a lot more puzzle pieces today than decades ago. Puzzle pieces of what you might say the big picture, like the overall macro strategic situation we are in. Some set of facts or considerations or insights that would allow us to orient ourselves with respect to which directions are good and which are bad. Do we want faster progress in some area, slower, more government involvement, less government involvement, more transparency, less transparency on these kind of basic dimensions? I think we understand more than we did. However, all those little puzzle pieces, I think, still don't really add up to an ability to form a reliable opinion here. An analogy might be, suppose you had to determine whether the sum of 10 binary numbers was even or odd. Somehow, if it's even, you should head north. If it's odd, you should head south. So I'd say maybe decades ago, we knew three of these binary digits, and now maybe we know seven of them. So in some sense, we know a lot more, but as long as you remain uncertain about one of these, you're still kind of completely in the dark.
Eddy Lazzarin
Yep. I was looking at your website, and what I think is funny is there's a section where you list a lot of misconceptions about your work. And I'll just quote a couple. You say people think I'm a gung-ho transhumanist or that I'm anti-AI or that I'm a consequentialist fanatic who would favor any policy purported to mitigate some existential risk. Which misunderstandings do you think you're spending the most amount of time correcting right now?
Nick Bostrom
It depends a lot on which specific audience. It's a kind of devolving set, unfortunately. One common one is because the most recent book I wrote was on Deep Utopia, which kind of looks into what happens if things go right. People assume maybe I have radically changed my views and now no longer think there are significant risks with AI. But it's more that with any particular work, you're trying to not write like a manifesto covering everything. Rather, to make some intellectual progress, you need to focus on some specific question and place where you can get traction. So it's more that the scope of the books were different rather than that the underlying view has changed.
Eddy Lazzarin
How did you decide that something like the concept of deep utopia or different paths for what utopia would look like? How did you decide that that was a worthy thing that you wanted to write a book about?
Nick Bostrom
Well, at the superficial level, there was, I guess, some frustration with how superficial the conversation seemed to be to me when it started looking at what would actually happen if we do develop machine superintelligence, what's the future like? It often sort of reached a point where people might consider, well, maybe there would be some impact on the labor market. And so what would we do? Maybe retrain people or something. But if you actually think through what it would mean as sort of technology runs to its logical culmination. It involves much more profound questions about human meaning and so forth.
And then another deeper rationale was that at some point, it seemed to me there might be some person or group of people or maybe humanity as a whole that will need to actually make a decision about what future we want. So imagine going into some room to try to deliberate and determine all these different paths that get unlocked with more advanced technology. We could transform ourselves in all kinds of ways. It's a pretty daunting challenge. And so it seemed maybe useful if there were some, as it were, preparatory reading material that such hypothetical future deliberators could go through that might equip them with various ideas and concepts and questions to ask and ideally also maybe put them in the right frame of mind to approach this with a in a spirit of sort of generous curiosity open-mindedness while still feeling in touch with all different kinds of human values.
I think maybe the deliberation would be more likely to go better if you come at it with something like that spirit and so instilling that was another motivation for writing the book.
Eddy Lazzarin
Implicit in what you're saying is when you hear the conversation and it sounds a little more shallow than you'd like to hear, that means it also implies that you think that it's something valuable and worth edifying, right? And if, when you wrote about superintelligence, many people interpreted that as a sign that you felt that that was something possible, obviously. And when you wrote Deep Utopia, it sounds like that's something that you think is possible. So I get what you're saying about the superficial layer, but there's an implicit recognition that it's a possibility for the world.
Nick Bostrom
Oh, yeah. Yeah, yeah. I mean, superintelligence certainly is possible and at a wide range of outcomes. Yeah. The uncertainty back when I was writing superintelligence was not whether it was possible to build superintelligence, but what the outcomes of that would be.
Elena Burger
The kind of specificity with which you think through things is a methodology that might be helpful in terms of helping think through our present moment. I think where this uncertainty really comes through, both in superintelligence and in deep utopias, I think you grapple a lot with contingency and sort of the idea that a decision or an intervention that is made at one point in time might have a radically different outcome if you do that same decision or that same intervention at a very, very different point in time.
Nick Bostrom
Yeah, with respect to the AI situation, I think we'll need to sort of feel our way through. It's a rapidly evolving situation and we need to be able to respond as new information comes to light or political realities change around. I haven't been a huge enthusiast of this stopping AI or pausing AI. I do think the pacing the frontier framework has something to be said for it. I could easily imagine there being some point, perhaps in the not distant future, where it would be extremely useful to be able to either have a very time-limited pause to allow extra safety work to be done on some system before the knobs are all turned up to 11, or maybe to have some ability to coordinate a slight reduction in the pace of capability advances if it looks like safety and alignment is trailing behind.
It seems that developing this extremely consequential technology in a context where, say, whichever company or product is doing it first is unable to slow down for more than a few weeks because if they do, they just become irrelevant and sort of the initiative goes to maybe a less scrupulous, less risk-aware competitor. That seems maybe not the best way for humanity to approach this. It does seem worth kind of taking a little bit of extra time to make sure we get it right. I would start to worry if the method by which we were planning to achieve that ability to moderate a pace were such that it had a large likelihood of resulting on us never developing super intelligence or like kind of putting it off.
You could imagine if the method is create so much public animosity and antagonism to AI that it gets completely vilified and stigmatized and then set up some extremely powerful regulatory apparatus with global scope to supervise all AI data centers and so forth. That once you start to move to such extreme solutions, then you start to have to worry that they wouldn't just serve this temporary function, but that they would become entrenched in some kind of unchallengeable orthodoxy. And so then it looks far-fetched today, but if that were to happen, then the pendulum might swing towards another kind of existential risk, which would be a permanent failure ever to develop superintelligence, which I think it itself would be a kind of an existential catastrophe.
Eddy Lazzarin
So this doesn't seem very realistic, but look at the trend line of how much anti-AI sentiment has grown just over the past 18 months, two years. And this is before anything bad actually has happened. So we've... Yeah, I don't think it sounds unrealistic at all. At the risk of asking a difficult-to-form question, couldn't you always construct, because of the sort of flexibility and programmability of these things and because of the great improvements in our mechanistic interpretability skills, couldn't you always design a payoff matrix with an AI where you can give it like an infinite femtosecond duration reward in exchange for retiring it peacefully and sort of resolve a lot of the more complex payoff scenarios with humans through the direct manipulation of the model's intelligence capabilities? I mean, maybe to put it a different, more extreme way is like, are you genuinely concerned about the construction of a utility monster? And do we need superintelligence capabilities to actually be able to construct a utility monster today? Like, is that something that you think we ought to be genuinely concerned of?
Nick Bostrom
Well, I don't think at the moment we have such precise techniques in alignment to reliably be able to engineer in the precise set of values or goals that we would prefer for one of these AS to have. If we did, then it would be kind of home and dry with at least respect to one of the big challenges of this.
Eddy Lazzarin
Well, we don't have the alignment capabilities that I'm aware of, but I don't know if you saw something that's been going around X the last couple days, a very interesting home hacker example of someone who's able to take one of the Quen models and manipulate only a few neurons at a time and completely change it into a totally reportedly, you know, self-describedly, you know, blissful or evil or, you know, psychotic or even like Christian or etc. AI. In other words, the single neuron ability to manipulate these and to create what seem like authentic displays of extreme behavior implies that as capabilities improve, we will only gain more fine grained control over exactly these types of things, including the neurons that could imply some kind of utility payoff, which is not something that is presently possible for humans, obviously.
Nick Bostrom
It is plausible that we will get much better at alignment. In some ways, the problem gets easier the smarter the AIs are in that they are able to sort of understand what it is that we had in mind and they can sort of see what we are pointing to. Rather than the literal definition, they might sort of get the underlying gist of a command. In some ways, however, it also gets harder. With less capable AIs, you can rely on behavioral tests. So you create your AI, you run it in different test situations, and then if it behaves well, then, well, you know now it's a well-behaving AI, and you can deploy it.
At a certain level of cognitive sophistication, you have AIs that become situationally aware, that can detect whether they're in a test environment or in a deployment environment. And they can reason about that and sometimes choose to act differently because they think they are probably being monitored. And we already see this with current frontier systems. If we look at their chains of thought, they often kind of strategize about how they want to behave given that they might be watched or that different things might happen. And so such systems, they might behave one way in a test environment where they realize they are being tested, and then when they think nobody's watching or later when they are in a position really to grab power, their strategic calculation might change radically.
This more capable AI system might also be able to subtly influence their own training process by doing different kinds of maneuverings during the training process to shape their own motivation. And of course, there's more stuff going on inside it. Maybe they learn to reason in ways that are not visible in the chain of thought that could then lose us visibility into what they're thinking. So on balance, I think the alignment problem probably gets harder with these more capable systems. Certainly the consequences are more severe. Nevertheless, I think the main hope is that we will sufficiently well quasi-align some slightly superhuman systems that we can mostly get them to do what we want and then have those help us sort of design the next iteration of more aligned AIs and then so on and so forth so that you sort of have a, hopefully an attractor base where you just need to get close enough initially and then the things will kind of turn out okay.
Elena Burger
I wanted to go back to a comment that you made earlier about the risks of postponing super intelligence and the idea that it could be really bad for humanity if we don't develop superintelligence and really advanced systems. And it's an idea that you have been exploring for a while. You have a paper on the optimal timing for superintelligence. Can you explore some of the ideas in there?
Nick Bostrom
Yeah, so this paper, relatively recent one, it looks at one specific question, which is if the goal is to maximize the life expectancy of currently existing people, how fast would we want AI development to unfold? Now, I should say it's not the only question that we should be asking. There are also other values and considerations that I think in an all things considered decision should play a big role as well. For example, we might be worried about future generations like existential risks, fairness, all kinds of things, right? But this is one thing that one might look at. And so Elias Yudkovsky and Nate Soros had this book, right?
If anyone builds it, everyone dies. And it seemed to me that we have at least a strong evidence for the claim that if nobody builds it, everyone dies. Like, in fact, 95% of everybody who ever lived are already dead. and the rest of us are kind of on track to follow them within a few short decades. Whereas successfully developing safe superintelligence, I think, could lead to enormous breakthroughs in biomedicine and in anti-aging interventions that could radically extend the lifespan of currently existing people. So if you think about what our life expectancy would be if we weren't aging, something on the order of 1,200, maybe 1,400 years or so.
And that maybe is a quite conservative lower bound because in this situation, you might also have other ways of reducing other mortality risks like accidents and so forth. So you then have a number of different competing factors. On the one hand, you might think that if you took some extra time with this AI transition, maybe we could build AI more safely. So you might think that if we spend more time doing AI alignment research, the existential risk from developing AI goes down at some rate. And then on the other hand, every year that we delay it, we have millions and millions of people dying.
I think it's like what is it? Like a hundred I think it's like 150,000 people every day. Yeah. So you don't want to delay a single day longer than you really need to. But then you can on the different assumptions kind of model at what rate does the risk have to decline for it to be worth sort of waiting longer. Yeah. And you can add different things. You might, for example, some people might have diminishing marginal return in additional life years. So they might be not happy to take like a 51% chance of living to 200 rather than 100% chance of living to 100 and so forth. So you might have time discounting but for a wide range of parameters it looks like the optimal delay is relatively short as in sort of maybe months or a few years as opposed to decades.
And you can see this that if the rate at which the risk level goes down from a delay is very fast, like you develop some sort of almost super intelligent system, you pause. If at that point the risk goes down very quickly, then you don't need to wait very long, right? You just wait for a few months and then the risk is so low that you can go ahead. If, on the other hand, the rate goes down very slowly, then you don't want to wait any time at all, right? Because more people are dying per year from natural causes than the expected number of people saved by waiting. So it's only if you have a sort of intermediate rate of safety progress that you would wait.
Now, I think possibly the very early days after you have the ability to create AI would be especially valuable for AI safety research. You have this almost final system that you can study and test and sort of slowly deploy, watching extra carefully. I think having a little bit of extra time there could yield like large safety gains. But then I think the rate of additional safety gains will sort of start to decline after like a few years or certainly.
Eddy Lazzarin
It's funny to think we're actually like living versions of this question already. If you look at Waymo or if you look at Ozempic or something, It's like Waymos are empirically much safer than humans driving cars. And yet we still have humans driving cars. Theoretically, every obese person should be on Ozempic, but they're not. So we do have versions of these like, oh, we actually do have interventions for things that historically have been extremely deleterious for humanity or deleterious for humanity. And we just like, like, it seems like there are other things that hold us back even after we have the technology that's been developed.
Nick Bostrom
Yeah. I mean, I think life expectancy has been increasing by about two or three months per year. So we need to get at about one year per year. Yeah. And so like it seems very doubtful that absent superintelligence, we would, I mean, you might hope for independently just some kind of breakthroughs in synthetic biology and biomedicine, like ways of actually seriously slowing down the aging process. I think if we got that or maybe from some early applications of not yet radically super intelligent AI systems that could sort of fix aging and take care of some of the other like particularly pressing problems in the world, then like maybe we would then become more patient.
You could afford to take it slow with the more radical versions. But as things are, so the life expectancy is one. And of course, there are other pressing problems as well. Desperate poverty, suffering of all kinds amongst humans and animals and many things where these AIs, if developed well, could really become the cavalry that rides into the rescue at a critical moment in the battle. So the stakes are high either way. As I said, I think that's only one of many different considerations that would come into an all things considered evaluation here.
Elena Burger
Yeah. To what extent does your thinking in the paper rely on being able to identify something like a discrete threshold? Because the way that it's framed is we have the ability to release or deploy a superintelligence, which of course relies on some kind of a definition of a superintelligence. And then we are able to measure, in some sense, the risk of its releasing and whether or not we are able to decrease that risk. Now, I'll grant that I think I don't think it's we have to have an ultra discrete bright line beyond which in realities these things often play out as a patchwork of capabilities are analyzed separately and specific behaviors, specific risk scenarios are evaluated and tests are performed to understand each of them independently as opposed to as this whole. But to what extent do you need to be able to identify and define it? How do you think we ought to identify and define which are these risk thresholds at which we should consider pausing?
Nick Bostrom
Yeah. So in practice, that's, of course, what we need to do. So there's this particular paper. It's just a small paper, but it's just meant to create a sort of simple model to illustrate the way that the urgency dynamic of people dying from natural causes kind of interacts with the risk parameter. Just to kind of intellectually be able to look at that particular aspect of it. Because that does seem to be a kind of point where different people arrive at very different views about AI depending on where they sit on that. But then, of course, in practice, we face not just some hypothetical binary decision about number of years today, but we feel like each different actor is confronted with like a complex situation with uncertainty and particular things they could do.
And so there I would say there might, for example, be different ways of increasing the capability of the AI systems that have different implications for safety. And there might be ways of pushing towards greater capabilities that are comparatively safer than others. So for example, one way to increase capability might be to increase the recursive depth by moving to looped transformers or other architectures that allow the model to do more thinking within one forward path or before emitting a token in its chain of thought. And that has the disadvantage that it reduces our ability to see what they are thinking.
Like they can do more thinking between each output that we can see. And then you might imagine a different way of increasing the capability, maybe giving the same performance boost, but maybe by increasing the number of parameters in the network or something like that. So maybe that is relatively less costly from a safety point of view than increasing recursive depth or training the systems to think in neural ease. Maybe that would make them more efficient at thinking. But then you're talking about different design setups that create trade-offs between interpretability and capability. Yeah, or between some other risk-relevant dimension and capability.
And so I think a lot of the actual action would be more at that kind of level of granularity, where perhaps if there were some ability for Frontier Labs to coordinate, to say refrain from using some specific way of increasing capability that seemed particularly problematic from like loss of control point of view and focus on competing along the other dimensions. That might be the kind of thing that would be the right trade-off. Sacrificing a little bit of like, maybe it costs a little bit more to pursue capability in these other dimensions, but like we gain hopefully greater safety.
Eddy Lazzarin
So given that these decisions will be made by people in these labs, at least assuming it's set up the way it is now, how do you think about accountability? Because you end governance of these things? You talk about nationalization as a risk, presumably because it could create delay, but how do you think about accountability and governance?
Nick Bostrom
I mean, whether nationalization is a risk, I don't really have a strong... It's one of those things where I think we need to feel our way through and the judgment of whether you would want that or not might sort of change depending on if it's not nationalized, then exactly who is in the lead and doing what and if it were nationalized, like in what precise way would it be? I think, I mean, the case four would be, wow, this is like a crazy, it's more powerful than nukes. Isn't it like insane that some random corporation trying to maximize share all the profits should do this like with nuclear weapons that was a Manhattan project.
You could have much better information security. It's like one big project rather than different competing ones. On the other hand, in favor of the current situation, I think nuclear weapons were developed in a military context. It might seem more positive for something like this to be developed in a civilian context, not focused on maximizing our ability to kill people, but on providing benefits to consumers. We have more transparency into these commercial developments than a Manhattan product. Even the vice president at the time was unaware that the Manhattan product even existed. So you could imagine extreme sort of concentration of power dynamics.
It's run by these kind of national security types. Maybe you think these idealistic tech nerds are less scary. It's more possible for many more people to participate now in AI, like either as consumers of the technology, like think tanks or external evaluators looking at these models, investors, investors in the model building companies or in the semiconductors. The benefits and participation is much more broadly diffused amongst society and around the world that might be preferable to having some very concentrated project that secret with a small number of people not really know who they are sit on top of like some people have a lot more money than others.
But if you compare the Gini coefficient of global capital ownership or stock ownership, both directly and through pension funds and sovereign wealth funds and stuff like that, it actually is a slightly less unequal distribution than a distribution where, say, you had a U.S. national project where even if you granted the very implausible assumption that every U.S. person would have exactly equal say and stake and benefit from that and everybody else would have zero. That's actually a more unequal distribution globally than one where it's in proportion to current capital assets. So even though it's not the most ideal from a distributional point of view, this open global investment model, it actually scores better on that than the most likely alternative.
Elena Burger
Are there any kind of questions relevant to humanity today that you're exploring right now that feel particularly salient?
Nick Bostrom
I think our main focus probably at this point should be how to get from here to there. It looks like we're entering this really critical period where maybe the long-term future for Earth-originating life will be determined one way or the other. Getting that right should be a huge priority.
I think these can be somewhat relevant in choices we currently make. For example, the level of existential risk you're willing to accept from the development of superintelligence might depend in part on how you value the opportunity later on to become a post-human type of entity. If you think the best we can possibly achieve is basically status quo, human beings with slightly higher standard of living and some new medicines, but that's the ceiling, then things could still get better relative to where they are now. It would be worth taking some risk for achieving that sort of uptick, but not an enormous risk, especially if you're already sitting in a rich country with well-educated, a lot of money, privileges of all kinds. You think, well, it's pretty good.
On the other hand, if you think we ain't seen nothing yet, that we've been this monkey-like creature hopping around here doing our little thing but we have no clue, if you unlock this larger space of possible modes of being, the kind of super felicity, the cognitive uplift, the emotional depth of appreciation for what existence could mean, different ways of relating and creating together, if you think there's a huge space of unfathomed upside there, then you might be more inclined to risk more to have a chance to have a shot at that. So I do think, even though the main focus should be on these more pragmatic trade-offs and technical improvements that we're able to make, that kind of background picture of what you ultimately think is possible does have some relevance as well into how we should approach the choices in front of us.
Eddy Lazzarin
Two questions, Rene. I don't mean to make them sound quick. This is a weird one. This is a weird one, Nick. Do you think that there is a telos to the cosmos?
Nick Bostrom
Well, what do you mean by the cosmos?
Eddy Lazzarin
Creation, so to speak.
Nick Bostrom
Well, if it's a creation, literally, then probably the creator had some purpose for creating it. The reason why I'm asking is, in my early work I introduced a simulation argument, which is maybe a different can of worms. But if you take the hypothesis seriously that we're living in a computer simulation, then there's an important distinction about the part of reality that we see, our simulation, and the totality of physical existence. If we're in a simulation, then probably the people creating the simulation had some reason for doing that, which is, I guess, depending on what you mean by purpose, but there would exist some purpose in that sense. The totality of existence, then it's more of a question of maybe theological considerations whether you would think that the totality somehow was purposely created.
Asking about purpose could mean many different things as well. One question one might ask is whether there is some set of facts about the world such that if we understood them clearly, we would basically all come to the same conclusion about what we ought to be doing. If there is such a fact, then maybe you could say that's a kind of objective purpose in some sense. If either everybody or maybe for a wide range of possible goals and motivations you have, it could be the case that they have this kind of convergence in that if you properly understood all the facts, they would point to the same practical next step. That, I think, is a lot more of an open question.
One way that could be, of course, is in a traditional theological context. If a divine being has a goal and rewards people depending on that, then that might just be such that if that is true and if you understood that, then basically almost independently of what your particular goals are, you would have the most reason to follow the prescriptions there. If we are in a purely materialistic worldview that a lot of secularists have, where there's a loss of nature just rattling around, that might not be such a convergence. What you have most reason to do depends on your particular goals and somebody else has different goals, so there is no higher fact that would cause you to have reason to do the same thing. But I think there are a lot of possibilities between those two that we don't fully understand yet that could make it such that there is in some sense a purpose even if we are not quite clear on what it would be.
Elena Burger
Yeah. I wasn't going to ask this question, but you brought up the one answer that all of humanity could be guided toward that would kind of answer everything. So the natural follow-up is, let's say AI gives us that answer, superintelligence gives us that answer, do you stop practicing philosophy? Is there an AI model that's so good that all of the important philosophical questions just kind of get obviated?
Nick Bostrom
I think we will have AI that surpasses us in philosophy, just as we now have in math and probably soon in other fields. There might be a subset of questions that have the form more of us making up our minds about what we want, where it might be the case that we need to do some of the deliberation and deciding ourselves. But broadly speaking, yes, I do think that our own intellectual efforts in that realm also will become obsoleted by these super-intelligences. Which means that right now, whilst we still have something to contribute, is the time to really try to have whatever positive impact we are hoping to have in our lives. It's the golden age now for purpose and positive impact. And then, once it's out of our hands, it's time to maybe kick back a little bit.
Eddy Lazzarin
Speaking of, is there anything that you wanted to discuss or that we should have covered that we haven't so far?
Nick Bostrom
I mean, it's so many different directions we could go. Each of the topics we touched on in itself could almost be a one-hour conversation. So yeah, but there's nothing in particular right now.
Eddy Lazzarin
We'll love that. Well, thank you. Thank you so much for joining us. Really, really appreciate you taking the time. This was so much fun.
Nick Bostrom
It was fun, yeah.
Eddy Lazzarin
Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. It should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com forward slash disclosures.