AI is here to stay, but at what cost? Connor Leahy is the CEO of Conjecture, a mission-driven organization that’s trying to make the future of AI go as well as it possibly can. He is also a Co-Founder of EleutherAI, an open-source AI research non-profit lab.
In today’s episode, Connor and David cover: 1) The intuitive arguments behind the AI Safety debate 2) The two defining categories of ways AI could end all of humanity 3) The major players in the race towards AGI, and why they all seem to be ideologically motivated, rather than financially motivated 4) Why the progress of AI power is based on TWO exponential curves 5) Why Connor thinks government regulation is the easiest and most effective way of buying us time
------ 🚀 Unlock $3,000+ in Perks with Bankless Citizenship 🚀 https://bankless.cc/GetThePerks
------ 📣 CYFRIN | Smart Contract Audits & Solidity Course https://bankless.cc/cyfrin
------ BANKLESS SPONSOR TOOLS:
🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE https://k.xyz/bankless-pod-q2
🦊METAMASK LEARN | HELPFUL WEB3 RESOURCE https://bankless.cc/MetaMask
⚖️ ARBITRUM | SCALING ETHEREUM https://bankless.cc/Arbitrum
🧠 AMBIRE | SMART CONTRACT WALLET https://bankless.cc/Ambire
🦄UNISWAP | ON-CHAIN MARKETPLACE https://bankless.cc/uniswap
🛞MANTLE | MODULAR LAYER 2 NETWORK https://bankless.cc/Mantle
----------- TIMESTAMPS
0:00 Intro 3:12 AI Alignment Importance 9:40 Finding Neutrality 14:16 AI Doom Scenarios 21:06 How AI Misalignment Evolves 25:56 The State of AI Alignment 32:07 The AI Race Trap 41:49 Motivations of the AI Race 56:18 AI Regulation Efforts 1:14:28 How AI Regulation & Crypto Compare 1:21:44 AI Teachings of Human Coordination 1:36:53 Closing & Disclaimers
----------- RESOURCES
Connor Leahy https://twitter.com/NPCollapse
Conjecture Research https://www.conjecture.dev/research/
EleutherAI Discord https://discord.com/invite/zBGx3azzUn
Stop AGI https://www.stop.ai/
----------- Related Episodes:
We’re All Gonna Die with Eliezer Yudkowsky https://www.youtube.com/watch?v=gA1sNLL6yg4
How We Prevent the AI’s from Killing us with Paul Christiano https://www.youtube.com/watch?v=GyFkWb903aU
----------- Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.
Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here: https://www.bankless.com/disclosures
No summary for this episode yet.
Even if you, dear listener, don't totally buy the existential risk thing, you know, maybe, you don't buy it. It's fine. But it is the case that the leaders of all the top labs, anthropic, deep-mind, open-air have been on the record saying clearly that they do think that there is a realistic possibility that these technologies will kill literally everybody, and they're doing it anyways. Welcome to Bankless, where we explore the frontier of internet money and the internet finance. This is how to get started, how to get better, and how to front-run the opportunity. This is David Hoffman here without my co-host Ryan Trin Adams, but regardless, we are here to help you become more bankless.
Today, on the episode, we're talking AI alignment and AI safety once again. We're talking to Connor Leahy, the CEO at Conjecture, a mission-driven org trying to make AI go well. We wanted to do one last episode on the AI alignment and AI safety conversation, because we think Connor can really deliver a very compelling and easy articulation as to why AI safety is real and why it needs to be treated as such. Some main benefits and takeaways that you're going to get from this episode. First, the intuitive arguments behind the AI safety debate, the things that you can take to your friends to convince them that AI safety is a real issue.
Second, the two defining categories of ways AI could end humanity, and third, the major players that are playing in the race towards AGI and why they all seem to be ideologically motivated, rather than financially motivated. Fourth, why the progress of AI power is based on two exponential curves, and lastly, fifth, why Connor thinks government regulation is the easiest and most effective way of buying us time. Here on bankless, we've had the AI alignment, AI safety conversation a handful of times with different players from the industry. Ever since we had that Eleazar episode, which we were hoping would have been an AI crypto conversation, but turns out it was an AI is going to kill us all conversation.
We're bringing on Connor Leahy in June of 2023, a number of months after we first went down this rabbit hole because the world of AI kind of feels like the world of crypto in 2021, it is moving so fast. So Connor gives us the lay of the land, a snapshot in time of the AI alignment conversation as it stands here in late June of 2023. As well as also articulates in the easiest and most simple terms possible why AI alignment is such a big deal. If you bankless listener are not convinced of AI alignment going into this episode and you remain unconvinced after the end of this episode, I have nothing left for you.
So let's go ahead and get right into that conversation with Connor Leahy, but first a moment to talk about these fantastic sponsors that make this show possible, especially Kraken, our preferred crypto exchange for 2023, assuming that we get past the AI alignment issue, we will still need to buy crypto assets using our Fiat dollars. So perhaps use Kraken to get that done. There's a link in the show notes to get started. Kraken Pro has easily become the best crypto trading platform in the industry, the place I use to check the charts and the crypto prices, even when I'm not looking to place a trade.
On Kraken Pro, you'll have access to advanced charting tools, real-time market data, and lightning fast trade execution. Kraken's new customizable modular layout lets you tailor your trading experience to suit your needs, pick and choose your favorite modules and place them anywhere you want in your screen. With Kraken Pro, you have that power. Whether you are a seasoned pro or just starting out, join thousands of traders who trust Kraken Pro for their crypto trading needs. Visit pro.kraken.com to get started today. Mantle, formerly known as BitDow, is the first Dow-led Web3 ecosystem, all built on top of Mantle's first core product, the Mantle Network.
A brand new, high-performance Ethereum layer 2 built using the OP stack, but uses eigenlayers' data availability solution instead of the expensive Ethereum layer 1. Not only does this reduce Mantle Network's gas fees by 80%, but it also reduces gas fee volatility, providing a more stable foundation for Mantle's applications. The Mantle Treasury is one of the biggest Dow-owned treasuries, which is seeding an ecosystem of projects from all around the Web3 space for Mantle. Mantle already has subcommunities from around Web3 on board, like Game 7 for Web3 Gaming, and Buybit for TVL and liquidity and on-rounds.
So if you want to build on the Mantle Network, Mantle is offering a grants program that provides milestone-based funding to promising projects that help expand, secure, and decentralize Mantle. If you want to get started working with the first Dow-led Layer 2 ecosystem, check out Mantle at Mantle.xyz and follow them on Twitter at 0xMantle. If you haven't experienced the superpowers that a smart contract wall it gives you, check out Ambiere. Ambiere works with all the EVM chains that are out there, the Layer 2's like Arbitrum, Optimism, and Polygon, but also the non-aetherium chains like Avalanche and Phantom.
Because of the power of smart contract wallets, Ambiere lets you pay for DAS and stablecoins, meaning you'll never have to spend your precious ETH again. The Web app has numerous fiat on-ramps to make it easy to dump your fiat for crypto, and if you like self-custody but you still want training wheels, you can recover a lost Ambiere wallet using an email and password, but without giving the Ambiere team any control over your funds. Check it out at Ambiere.com for the Web app experience, but also the Ambiere mobile wallet is coming soon for both iOS and Android, and if you want to be a beta tester, you can sign up at Ambiere.
com slash app. And since you stayed to the very end of this ad read, you should know that Ambiere is air-dropping its wallet token to early users for simply just using the wallet. So if you want to get started with Ambiere, all the links that you need are in the show notes. BanklessNation, I'm excited to introduce you to Connor Leahy, the CEO at Conjecture, a mission-driven organization trying to make AI go well. He's also the co-founder of Eluther AI and open-source AI research nonprofit laboratory, which interestingly operates mostly inside of a Discord server, much like our company and so many of us in the BanklessNation Connor, welcome to Bankless.
Thank you so much for having me. Connor, the crypto world has more or less collided with AI, and Bankless, we had our introduction with that. Surprisingly, when we had our LA's or Yudkowski episode, which we had to pivot mid-episode from a crypto AI intersection episode into an AI is going to kill us episode. Since then, we've continued to go down that AI alignment rabbit hole. I think a decent number of people in the BanklessNation and the broader crypto landscape accept the AI alignment problem, but others completely reject it. And it's interesting to see some people just have a spinal reflex rejection of the AI alignment problem.
So in this conversation, I hope we can kind of just talk about the conversation of AI alignment in the outside world, the companies that are playing here, the game board that is laid out in front of us. But first, I really just want to dive into the very basics of the AI alignment problem and see if we can, once again, articulate clearly why the AI alignment problem exists, what it is, and why it's important. And you've gotten a lot of practice at articulating this, so I'm wondering if you can kind of hand hold us through some of the very basic premises behind the AI alignment problem. Yeah, absolutely.
So to start things off, the AI alignment problem or existential risk is really what I care about. This is that something could be so dangerous that an accident or a misuse could occur of such magnitude that it could threaten the continued existence of all humanity or curb our potential forever in some sense. This is, of course, be extremely terrible. I think this is pretty uncontroversial that if such a thing were possible, and if it did happen, that would be pretty terrible. And so the way I like to think about this is first, kind of from like an outside perspective is if you look at a history of technology, you know, we see that technology has gotten better and better all throughout our history as given us way more power, way more fordance.
A lot of this is great, right? You know, we have medicine and, you know, I have air conditioning in my office, thank God, you know, all these, you know, great things that are wonderful and I love. But also as technology increases, as your power increases your ability to control the environment, you have more capacity for things to go wrong or for destruction to occur. So, you know, back on the stone ages, you know, the worst possible thing that I could do with stone age technology would kill, you know, I don't know, maybe like 10 people or something. You know, if I'm like a pretty big guy or, you know, maybe if I'm like super smart about it, I could kill a few hundred.
You know, if I have my whole war band with me, I could kill maybe more, but like an individual person just using stone age level tools, you know, not going to really be an existential risk or anything of that sort. As our technology gets better, you know, we develop, you know, more sophisticated weaponry, we develop gunpowder, we develop, you know, stuff like this, the number, the damage that can be caused both on purpose and accidentally increases. You know, sometimes from benign reasons, you know, if you have bigger ships, more people can drown, you know, that's pretty benign. We don't think that, you know, that's a risk that we are willing to take.
But in other greetings, you know, there was no such thing as, you know, a TNT factory is blowing up before we had explosives. And now that we had TNT, when an accident occurred, the collateral damage was suddenly of a type and a degree that didn't exist previously. And this is only continuing, you know, so we went from, you know, okay, you can kill like five guys to you can kill like 50 guys to kill 500. And now you know, you press the button and drop a new guy out of an airplane and you can kill 50,000 people or even more than that. And so if you like would graph, you know, over time, the increase in like blast radius of technology of like, you know, a misuse of a technology or thing, you would see an exponential.
You would see that our technology is growing extremely fast, extremely quickly towards larger and larger blast radius, not all technologies. You know, many technologies are very safe and they're very good. And I think we should invest in them and we should build them. But there are technologies that have larger and larger blast radiuses and eventually this blast radius will encompass earth. If our technology keeps improving, at some point, we will have technology that is so powerful that it can destroy earth, that it can destroy all humans. And even accidentally, it would of course be easier if you do it on purpose.
But at some point, if we have powerful enough technology, we should expect things where even accents are pro. And so I would make the claim that AGI is in this category. It's in the category of a trend in technology towards more and more powerful systems where even an accident during the development of the system has larger and larger blast radius. And sometimes this is what we accept as a society. We accept that sometimes a clinical trial might go wrong. This is something that sometimes we accept to some degree, not to arbitrary degrees. We don't let anyone do any clinical trial without any oversight of course not.
We do generally as a society, we think very highly of human welfare and life and that it shouldn't be endangered recklessly. But as this goes on, as we're dealing with these more and more powerful technologies, we have to be very... What do you do when you have a technology where the blast radius is everybody, including you? How do you develop a technology where getting it wrong ends everything, ends you, ends your experiment? This is not something humanity has experienced dealing with. This is not something that we are generally set up to do. The way we usually do technology is we build something and we fail a bunch of times.
We mess it up. If your lab assistants lose their hands, a bunch of stupid things happen. Government gets angry at you and 20 years later you maybe have something. This is fine, but predictably at some point this stops working. Maybe people could argue the time is not now and I would agree. I don't expect GPT 3 or 4 to kill all people. But GPT 5, 6, 7, 8, you know, combined with some modern RL-agentic systems, that's much less clear to me. And happy to go into a bit more about why I think this technology has this kind of black radius, but just pause there for a second. One thing I really want to emphasize in this argument is the neutrality of it.
It's not saying that AI is good or bad. It's just saying that AI is. And this is a continuation of the arc of technology. Technology is not good nor bad. You're just really putting it into very neutral terms that technology has an arc of goodness but has turbulence with associated blast radiuses along the way. And that we've all accepted these because the blast radiuses has been sufficiently small that it doesn't end all humans. But what you're saying is that, you know, continue that arc, get it to AI. AI, it doesn't want to kill us, it doesn't want to make our lives better. Humans will make choices with this technology, but it just so happens that in the advent of a bad outcome, that bad outcomes, bad radius contains all of us.
So it's not even like a political stance or an opinion stance about the goodness or badness of AI. It's merely just a statement about the magnitude of what could go wrong if something does go wrong. Yeah, exactly. I think a lot of discourse around this kind of stuff is very bad. I think a lot of it is completely politicized and like psychologist is about is open source good or bad. Is this guy a good person or a bad person? And I think this is just a terrible way to think about this. We should think about this very neutrally. The question is not, is AI good or bad? It's not, is open source good or bad?
It's not, is Sam Altman a good person or a bad person? These aren't the, the question is just what are the outcomes of the various choices we make? How can we do our best to predict what the consequences of taking various actions are? And how do we feel about those consequences? How do we feel about taking these risks? How risk tolerant are we? Who should be consulted? You know, I was never consulted to have, you know, a GPT-4 released onto the public internet, which I use as well to, for it to be released on my family and my friends. I was never asked if this kind of experimental new technology should be unleashed into the commons that I also inhabit.
And, you know, maybe that's fine, you know, we don't generally seek consent when someone, you know, wants to write a book. You know, someone's writing a book, you know, I don't have to read it, not my problem. But if someone is to, you know, releasing, I don't know, a new, you know, substance into the water supply. Well, I sure hope that I would be consulted about whether I think that's a good idea or not, because I'm drinking that water. But like, I feel like I have a say in this. And so the, a lot of the problem with the current state of discourse around the stuff is that there is some good technical discourse.
But then there's also a lot of discourse which focuses way too much on intentions. It focuses way too much on ideologies, like just like, you know, that we can't derive from an ideology what it's true. You know, if you believe, you know, open source is good. You know, maybe you believe that, right? But that's not a statement about reality. It's a mood affiliation, you know, sometimes open source is great. I worked a lot on open source. I think in many cases open source is fantastic. I'm so glad that Linux exists. I'm so glad that we have so much great open source software. I think this was really good.
But should the genetic sequence of smallpox be public? Like, I don't know, man. Like, you know, I love science. I love academia. I think it's great that people are getting funded to do all kinds of, you know, cutting edge research. A group of researchers in, I think, Canada used, you know, government funds to reconstruct an extinct form of horsepox or smallpox virus and published how to do it. I think they should have done that. Like, I think this is bad. I think people shouldn't do that. I think that as much as I love open access science, as much as I love science, we have to be practical here.
I don't expect the upsides of this being public to be worth it. So this is kind of like how I like to think about these kind of things. I like to think about these things way more is just like, look, what do we want? Like, what are the scenarios? What are the outcomes? And let's work from there. Let's work at this completely neutrally. Let's be just like, let's be realistic here. I like to talk about strategy. I don't like to talk about, I don't like to talk about like, you know, good or bad. If that makes any sense. Right. There's a bunch of conversations to be had here about the players in the world of AI.
We have chat GPT and open AI and Tim Altman. We have stable diffusion with E-Mod. There's a bunch of people in this game. There's the regulators are also in this game. I still want to continue a little bit down the defining the alignment problem conversation because there's something I want to parse apart. You're talking about the sheer neutral AI and its blast radius, right? What could go wrong if something went, where to go wrong? When we had our conversation with Eliezer, Eliezer gave us a very strict prescription of how this will go wrong, that your explanation of the AI alignment problem contains, but his is more narrow.
And I want to parse apart that because he calls this or we call this the AI alignment problem as in how do we align the goals of AI with the human goals? That's a more narrow conversation than I think what you're presenting. You're just saying, hey, AI is powerful. It could go wrong and it could go wrong for any subset of ways, some of which may be the AI alignment problem, but there are also other ways that it could go wrong. I'm wondering how do you parse apart the ways that are there categories of AI like Doom that are worth parsing apart? One of them is the AI alignment. Others are like maybe humans go rogue, and then we've seen humans do this, right?
What would happen if Ted Kaczynsi got their hands on a very powerful AI? That's another conversation. How do you parse apart the actual sources of destruction here? Yeah, so I like to use two, maybe three or four categories, is that the first one is misalignment. It's just you don't have control. I'm saying does something random, that's it. There's no, you know, doesn't even need a human to be involved. And that random things blast radius contains us is what you're saying. Yes, so the claim is that a accidents can happen and that accidents can have blast radius of this side. So this is like accidents.
The second one is misuse. So misuse is you have a system which does what you say, and someone tells it to do something bad, or for there to be conflict of some kind. You know, maybe there's multiple actors who go to war using this kind of technology or like fight. And then in the eyes you can, I can barely imagine something more horrific than multiple, you know, super intelligent systems fighting. I could you imagine, like it's unimaginable what horrors such systems could unleash in the terms of war? And that scenario, that scenario is like kind of interesting because it implies that the AI alignment problems actually solved.
As in we've been able to align human humans and AI together to achieve the same goals, but human alignment is not solved. And so using our AI superpowers we now commit war and our war blast radius once again contains everyone. Exactly. So it's like it's even worse than that. It's like there's so many ways in which this problem is very, very hard. We are, this is not a super narrow specific problem. There are specific narrow aspects to problem specific narrow technical problems, which are very, very important. And we can talk about those. But importantly, this is a generalized problem of society.
This is a general problem of the human condition. Is a general problem of how do we responsibly deal with powerful technology? This is the meta problem that we have to actually have to solve. And this is a problem that people have been talking about, at least since like World War One, you know, there is a Polish novel man named Alfred Krožipski, who after the horrors of World War One noticed in this like 1920s, he was like, wait, if technology keeps increasing, but our wisdom and our, you know, our control as a society increased much slower. Well, then all humanity will end. He figured out ex-risk in the 20s.
And yeah, and he wrote, and so his solution was he had a, he had to figure out how to improve the art of human rationality. Sound familiar? So there was an LESer in the 1920s, actually named Alfred Krožipski. So this is not a new thought. This is not something that, you know, I'm sure people before Alfred has also come up with variations on this thought. And this is continuing this problem. And we have not yet solved this problem. We already in the problem, you know, the last time we had a new level of powerful technology, you know, we did drop two nukes on purpose. And after that, there were several really, really close calls where, you know, nukes almost did stop flying, you know, at least two where it was just one single person each time who stopped the nukes from actually getting fired.
So our track record here is like decent, but it's not good. It is not good. And it's going to be much worse when we're dealing with AGIs and AGIm, especially like imagine if these things are open source. Imagine if every person in the world had a nuke during the Cold War. I expect we would have gotten nuked, you know, I expect it wouldn't have gotten well. If we didn't have chains of command, you know, several people signing off on something. If we didn't have sensible people that took their responsibility extremely seriously, I think it just would have not gone well. And the way things currently are, there is currently more regulation on selling a sandwich to the public.
Then there is to building unprecedented AGI level technology and releasing it to the general public. There is no oversight. There is no, you know, processes, you know, general processes here is or controls or like, you know, stakeholder or something. There is nothing. It's, it's just these private companies. It's a small number of private companies to be clear about this. It's basically currently mostly just deep mind, anthropic and open AGI, and a few others who are like trying to catch up that are really pushing forward to these high end AGI level, dangerous technologies, and are racing completely out of control.
So, like, how would you expect this to go well at this current pace? It's like, it's like, we're in the worst possible scenario we could be in kind of. I just want to pin down the categories that we were talking about. I interrupted you and I just want to make sure that we clearly identify them. The categories of how A.I. progress would go wrong. One of them is misalignment. This is the LEAzer conversation, the paperclip maximizer conversation. We create the super intelligent A.I. and then we can't figure out how to harness its goals and align them with humans, and so it accidentally turns us into paperclips.
That's one category. Another category that you'd defined was misuse. We do somehow do figure out the A.I. alignment problem, but we just abuse A.I.s to kill us all. One superpower fights with another superpower. One of them has A.I. Maybe both of them have A.I. and then we all die because we're using A.I. to have misaligned human goals. That's misuse. Another one is accidents. We have super powerful A.I. It accidentally does something that we don't like, and we're inside of that blast radius. That's a third category. Are those all of them or are there others that are worth unpacking? I think you can even fold accidents into misalignment in a sense.
I truly align system. If you tell it to shoot you in the foot, it will say, no, that's not what you intended, and you know, bring you flowers or something. If you have a truly, it's like there's like, and maybe like three categories is, or like maybe four is like, you know, like a good hierarchy, maybe four categories. Technology, there's so dangerous that no one should do it. You know, just like, there is nothing, like you just turned on, it bulls up everything. Then there's like technology, which is, which is controllable, if you're really careful, if you're very sensible, if you're very careful, it's fine.
Then there is technology that is safe for general use, except if you misuse it, except if you're specifically trying to do something bad, and then there is technology that is good, no matter who uses it. So that you can give it to the most evil sociopath in the world, and it's fine. Okay, and AI, I can see AI fitting into all those categories. It concludes those dangerous ones. Exactly. I think we will start at one, and then we can develop technology to move to two, and then we can develop technology to move to three, and then we can develop technology to move to four. But by default, we get the category one AI.
By default, we elevate AI as it progresses, finds itself inside of the category of technology you almost never ever want to even open up at all. Yes. And this is what I expect, like the shortest path to an AGI gets you this type of AGI, a system which is misaligned, you know, paperclip maximizer has some random values. It's very intelligent, very capable. It's very deceptive. And if you just turn it on, it doesn't matter who turns it on. It doesn't matter if this is, you know, the USA or China. It doesn't matter if it's open AI or an apthropic. It doesn't matter who does it. It just blows up everything.
There is no, it doesn't matter. You said the shortest path to get to a super intelligent AI. Can you unpack why you emphasize the word short there? What does, what is implied under a longer path to an AI? What does that mean? So importantly, this is not a rule of nature. There is no law of physics with states. It is impossible to have a safe AI that does good things for you. This is completely allowed by the laws of physics and computer science. We just don't know how to do it. And this is a very narrow target. Then as each of these, you know, categories one to four, are like a narrower and narrower target.
You need to know more and more about control, about intelligence, about, you know, safe practices, about human psychology, about like, you know, about, you know, values and game theory and whatever, to narrow down on these more and more complex systems. So by default, if you just want to think which is just smart, it's just powerful. It just succeeds at goals. Well, you know, just big larger models, you know, just throw a computer at it. Man, you know, just, you know, continue doing what we're doing right now. Basically, all the research currently being done at AI companies is of the kind, which gets you to type one.
There is a very little research that goes into getting you to type two, three or four. This is a great, a great quote, it's from Wilbur Wright. So this is the, one of the brothers who built the first airplane. And what he said was, is that when once a machine is under proper control under all conditions, the motor problem will be quickly solved. A failure of a motor will then mean simply a slow descent and safe landing instead of a disastrous fall. So this is the man himself, built an airplane, built an airplane, who realized the first step to building a good airplane was to solve the safety problem.
It was to build a safe glider that if something goes wrong, it carefully, slowly descends instead of killing the pilot. Because before this, there was a lot of attempts at flying machines and they always killed the pilot. So they couldn't develop them. He recognized that we had to first build a safe glider and then we can worry about the motor part. You know, and he even thought it's like almost dismissive of the motor. He was like, you know, it's like a real deal. And so, basically all of AI currently works on engines. They work on motors. There's very few people working on gliders. And there's a, basically, everyone is working on building bigger and bigger turbojet engines.
And you know, just like as fast as possible, as quickly as possible. And if you get a bigger and bigger turbojet engine, without improving a glider design, without improving your wing control design, without, you know, doing the necessary experience for this, by default, you get a type one engine. You get an engine that just explodes that just like, you know, just like does something stupid. And you know, if it's like an aircraft engine, the risk, the blast radius is contained as before. But AI is not just an airplane engine. Right. Yeah. The AI engine is focused on elevating humanity to as high of an elevation as possible.
And then all of the AI safety people are like, hey, we also need to make sure that we have a spaceship that is containing us that can take us back down in the inevitable case that eventually that engine runs out. Connor, I'm hoping I can just get your lay of the land of the state of the AI alignment conversation because as we've been saying, as you've been talking about, AI progress seems to be moving really fast. It is now June 2023 and it kind of seems that we have to timestamp these podcasts because if we were doing this podcast in just May or April of the same year, it would be a slightly different conversation about what is the state of the AI alignment conversation?
Last week checked, there was this letter signed by many world leaders and AI experts asking for a pause on AI progress beyond chat to be before. I'm wondering, like, oh, if you can just update us on the last few months and the mainstreaming of the alignment problem, like, is there more reasons to be optimistic or people bearing their heads deeper in the sand? Where is the world with regards to AI alignment and the AI safety? Yeah, I mean, what man has the world changed? The world's changed insanely over the last, you know, six to 12 months, I mean, to say it's likely AI alignment has gone to a large degree mainstream.
I just earlier today was talking to a member of Parliament who didn't even have a smartphone, didn't know what up an AI was, but he heard about his AI thing and he wanted me to tell him about it. And he got quite furious when I explained to him some other brists and he's like, no one taking this seriously? What? This is outrageous. Of course, we have to do something about this. So it's quite fun. For me, it's also been a very enlightening experience. I just got into this field, you know, I came in from like a pretty classic, you know, kind of like less wrong, you know, a laser adjacent, kind of viewpoint.
Very technical, very nerdy, very philosophical, perspective on things. Not much content. There's a lot of weird social memes in like that sphere around politics. That like, politics is bad. You should never do it. Don't, you know, don't talk to the public. Don't talk to the government. They're all crazy. Can't talk to them. And I feel it's completely gaslit and it's like, uh, they're going to be wrong. Politics is hard. You know, politicians or have their incentives. A blah, blah, blah. All these things are true. But this is things you can do. You can't talk to them. I think that's just been incredibly positively shocking to me.
Is that my talk to normal people who don't work in tech. They really get it. I'm so used to talking to people in tech. And they just totally dismiss these things. They're like, oh, yeah, I can never do anything bad. You know, blah, blah, blah. I can't hear you. And I talked to normal people who are really intelligent. And they can do more and more things and we don't know how to control them. And they're like, holy shit. Well, that's, that's terrible. Of course, this is going to go wrong. What do you mean? And like, you know, and then they, you know, I'll, I'll, I'll, repeat an argument that like some ultimately something and like, that's not convincing at all.
Like, what do you mean? He doesn't understand how his system works. It's all black boxes. This is, this is madness. And this is the correct reaction. The correct reaction. Like, let me be blunt here. Even if you, your listener, don't totally buy the existential risk thing. You know, maybe, you don't buy it. It's fine. But it is the case that the leaders of all the top labs, anthropic, deep mind, have been on the record, saying clearly that they do think that there is a realistic possibility that these technologies will kill literally everybody. And they're doing it anyways. Even if you disagree about the risk being real, I'm quite shocked that someone would like admit that they believe this, you know, state that they would, that they do think this.
And then also that they are willing to do it without the necessary safety and repercussions. And there are arguments. And man are there arguments. And we can get into those arguments, the counter arguments. But why actually this is fine? I don't find them convincing. Obviously. And we live in this world. And I don't think these people are malicious to be clear or like, they're lying per se. I think they're being inconsistent. Like, you know, Sam Altman will often go on the record and say, oh, you know, he thinks, AI is the biggest risk to humanity. Cool. Great. Thank you, Sam. That's really great.
That's like, I'm not being fachitious. This is actually fantastic that he has the honor, the honor and the bravery to say this publicly as someone, you know, who is ultimately a businessman, but still willing to go back to one deserve credit. But then he keeps racing. And then he keeps still building technology. There is this incredibly funny interaction on Twitter where, young like, the head of safety and alignment at OpenAI, tweeted something like, you know, maybe we should be careful and like, you know, slow down a bit before we integrate all this AI technology into, you know, all facets of our life.
And then six days later, Sam Altman tweets about ChatGPT plugins, you know, plug in ChatGPT into whatever you want. And I'm like, man, wow. Like if this was in a movie, like, you know, I can imagine just like the cut and then like, you know, the laugh track playing. I'm like, this is, this is, this is truly shocking. And this is a consistent feature is that this is something I've been pushing on a lot in the current discourse. Is that a lot of the discourse right now is people are starting to wake up to it and they're be confused. They're confused about like, well, is this risk real? Which is a good thing to be confused about.
This is a fair thing to be confused about. And there's other things about like, so this is a very funny thing that I've seen a lot, for example, on Twitter. People see like, say, Sam Altman call for regulation. And they'll be like, wait, this is sus. Like if he wants to be regulated so bad, and he thinks this risk is so big, why is he doing this? Just stop. Just don't race. If you think this is an extra interest, just stop. My honest opinion, yeah, that's pretty sus. Like some people criticize me by proxy. They'll be like, Conner, you're one of these doomer people. But, you know, you do mer people, you know, if you take it so seriously, why don't you start?
Well, first of all, I don't race. And yeah, that's a really good question. This is a question that I have for the head of all these labs. Why? Like, stop. I'm happy to go into the arguments that I expect their strong end version too. In case you have any comments. Maybe we can actually just define race. The whole like race condition, sort of things. A decent part of the bankless audience will be familiar with MoLock and MoLock traps. But it's been a while since we've had a MoLock episode. So maybe you can kind of talk about just like what this term race means. And why we are in a race trap. Yeah.
So race conditions or, you know, it's kind of like in a sense of like a race to the bottom. Is that the idea is you and other people don't want to go somewhere. You don't want a certain technology to exist. But other people are heading towards it. You think you're better than these people. You're more responsible, nicer, whatever. And so you think, well, you have to get there first before they get to it. And so you start, you know, trying to get there as fast as possible. They notice that now you're trying to get there as fast as possible. And then they're like, well, shit, I don't want that guy to get it.
Because I'm the good guy. And then they start going as fast as possible. So you get this game and now it's the race to the bottom. This is not unique to AI. This example, for example, with like safety regulation. The reason we have regulations for safety standards on the government side, instead of letting the market regulate itself, is by default, if you have a market and no regulation, there is an incentive to cut as much safety measures as you can possibly get away with. You want to do as little safety as possible. If you don't get in trouble for your employees getting killed, well, then just let them die.
You know, it's like it's not that big of a deal. And this is exactly what happened, like they're in the industrial revolution. Is that the pricing of a risk and of dangers and these kind of things can be unlike an economic scale. It's very easy to be mispriced from what we as a culture might want as a society might want. And this is not supposed to be a statement that regulation is good or bad. Again, it's not about good and bad. This is how we started this podcast. Not saying regulation is good or regulation is good. It's not about what we want. What do we want? What do we as a society want?
And how can we get that? And if we don't like where we are currently, what can we do to move to something that we like? So with the race, what we're seeing here is as Pete is companies undercutting themselves on safety and speed and timelines. Sometimes I like to sometimes it call it burning our runway. So humanity has a runway until AGI arrives. It will arrive sooner later. If before that time we don't have a safe spaceship. We die. And they're burning the runway. They're making it shorter by pushing forward this technology by investing more into it by building bigger engines. There's some engine size that when you get to that size, it blows up everybody.
Unless you have a safe spaceship built around it. And currently we're not building a spaceship. We're just building bigger engines. Because well, I don't want the other guy to build engines. He's really unsafe. He doesn't take it seriously or what if what if China gets it? There's some other country against it. No, America, number one, we have to get it first. And I understand these arguments wrong. And I'm happy to go into some of the details about why they're misleading. But it doesn't matter. There's nothing to win. So the real problem here is the simple counter argument is there's nothing to win.
You just lose. You don't get a type. This would work if we were talking about type two AIs. So AIs that are safe if you're careful. If we get to this and I think this is one of the things that someone would claim. Give a claim. Oh, no, no. We're going to go to type two AIs. Don't worry. We're not going to do those type one ones. Don't worry about that. And I would strongly disagree. I don't think that that is supported by our level of scientific progress on the alignment problem whatsoever. And but if we were super on track to get like a type two, three or four AIs then I'm like, okay, fine. You know, fair enough.
That's that's a reasonable thing to believe. I would still be more careful than that because I would never be that certain. I don't I wouldn't want to risk it. Even if I believe we're probably going to get a type two or three, I wouldn't want to risk all of humanity on it. I like take some time. But this is kind of the situation written out is that, of course, the people who are the most optimistic will be the ones who end up in the position to push this kind of phrase forward. It's not a coincidence. And they're going to be the ones who get billions of dollars of investment. It's not a coincidence.
It's not a coincidence that, you know, Sam Altman is the head of opening high and not Ellie Ezra. Ellie Ezra is not the kind of guy that lead an opening. I, of course, not. That's not what you would do. And so that's exactly the kind of scenario we're in right now. Who would you say are the main players in this race? Sam Altman, certainly of open AI, China as a techno country, as a whole maybe. Totally disagree. Totally disagree. Who's in the race? So just to say a word in China, I want to go too much into this. But like, there is a meme that exists, which I would like to dispel. Of like, China is this massive rival here.
They're going to, you know, overtake us. They're going to build AI, whatever. I think this is really ludicrous for anyone who actually knows about China. What the Chinese Communist Party wants is stability and to stay in power more of anything. Do you think they want a crazy, uncontrollable technology that could topple governments? No. And China has been very clear multiple times that they're willing to take massive economic burdens to censor and stamp out their own tech industry. They've done this multiple times of all the countries that I think is like most likely to regulate AGI away its China.
This is like the, this is completely in their interest. It is not in the Chinese Communist Party's interest. What's so ever for AGI to exist? It is completely counter to their incentives. And so, and also they're very far behind technologically. But this is just like a comment I decide. I think a lot of people are perhaps actually benefiting from holding up China as a bogeyman. So that's, you know, but the truth is basically 100% of the risk comes exclusively from the United States of America. There is no, in my opinion, appreciable risk from non-Western countries. What's whatever. Maybe this will change in you 20 years or 50 years or something.
But at the current point in time, all these models, all these things are being built on US soil, basically exclusively. Like, you know, even model like there's a recent model called Falcon, which was created by the UAE. But it was trained in US data centers. This was done on US hardware, on US data centers. This is this by, as far as I'm aware, US programmers. So they were working with the UAE on these kind of things. So like, this is very much an American and partially UK European issue. But that being said. So players. The main players as a geopolitically, it's the USA. I mean, to some degree, the UK, the US, the US, the US, the US, the US, the US, the US, the US, the US, the US, the US, I mean, to some degree, the UK, and the EU, a little bit as well.
But not deeply so. A little bit. And not non-trivially so. But not really. It's the USA. And within the USA, it's generally a small number of private companies. This is Google slash deep blind. Open AI, slash Microsoft, and Anthropic. These are the main ones. There are other people who are trying to catch up. There's a bunch of other startups trying to, you know, race forward to that kind of technology. But they're all very, very far behind. And most of the other technologies, groups in these field are not newly as ideologically devoted to super intelligence as these three companies are. The founders of all of these companies have been very clear.
And publicly so, that their interest is God-like super intelligence. They want to build system that can reshape, you know, all of humanity that can, you know, make, you know, upload us all into the cloud that can, you know, turn the whole world into nanobots that can. This is, this is not me saying this. Like this is, like, what actually these people have been quite publicly about. This is what they're trying to build. They're trying to build a better chatbot. They're trying to make the most shareholder return. The trying to do is to build God-like super intelligence. And then unleash it upon the world with, you know, they believe we'll bring utopia.
And so, this is, you know, people like Sam Altman, people like Demis Osabis, Daru Ahmeday, from Anthropic. These are, like, some CEOs of these various companies. And again, I would like to state, I'm not saying these are bad people. I really don't want to say this. I've talked to all of these people. And they're, for the most part, really great. Like, they're really pretty nice and very, very smart and hardworking and mostly, you know, trying to do what they think is right. Some more than others. But it should be clear what they are and what they are as trans humanists. They have an ideological interest in doing this.
They have strong incentives to be very optimistic and not think too hard about the dangers. Or to find excuses for why, well, it's a race. You know, my hands are tied. Nothing I can do about it. And I understand. I don't want to criticize and, like, say, like these are, like, evil people. There are evil people involved in this system, maybe. But for the most part, these are people of certain beliefs and of certain incentives who are trying to make the world a better place. But I, from my perspective, find what they're doing unimaginably. Unimaginably, reckless and cannot continue. Yeah. I'd like to unpack a little bit more around that just because I, so I can understand where the disposition of all of these founders are.
You said that the tech industries more or less in denial about this, at least in comparison to the outside world. And you've also said that people like Sam Altman and maybe the other founders as well have said that yes, AI is perhaps the most, the largest existential threat that we have. And yet they continue. And so I'm wondering if you can just like, diagnose that. Like is Sam Altman and all these other founders, they're just, oh, we're in a race. My hands are tied. I guess I'll just keep on racing until some sort of external force stops me. Like, how would you actually define why the disposition of these people are the way that they are?
The truth is, of course, I do not know. And this is basically psychoanalysis. I could do psychoanalysis. I know some of these people, you know, at least a little bit. But I think that's the only thing that some of these people, you know, at least a little bit. I have studied pretty extensively what they do and why they do it. I've read everything they've written and so on. I have, you know, guesses about their internal lives, which is, I don't know their internal lives. Maybe, you know, maybe some of them are total evil, you know, scheming bond villains. I don't think so. But like, maybe, I don't know, you know, maybe some of them have really good reasons to believe that AI is super duper, super safe.
I haven't gotten those reasons out of them, even after talking to all of them many times and pressuring them on this. So I don't know what their emotional motivation is. But what I can describe is their stated opinions, their stated beliefs, and their stated actions. My usual thing, I would say, and I recommend this to everyone. I think this is a really, really, really important skill that a lot of people neglect is watch the hands not the mouth. These people all say very, very nice things that make you feel very, very safe. And then their hands do something very different. This is a very consistent feature I found across dealing with, I mean, not just these, like, you know, generally powerful people, you know, politicians CEOs, you know, billionaires watch the hands not the mouth.
That it's, if you're very smart, it's very easy to come up with explanations why it is actually good to do the thing that you want to do anyways. And to the matter is, you can go back to archives sometimes from like the 90s, from some of these people talking openly on, like, email lists or on their blogs about how they want to build AGI. How they want to bring in the transhumanist future. They want to, you know, create this beautiful, you know, you, you know, immortal world of, you know, transhumanist cyborgs or uploads or whatever. And they want to do this fast as possible. You know, they want to save all this, you know, even Ellie Ezra is, was, at least in this camp and was, you know, in the 90s and 2000s was accelerationist.
You thought that being building AGI was the best thing we could do. And we should do this fast as possible. He changed his mind, which is fantastic. And speaks to his character and his like ability to think about things rationally and reasonably that Ellie Ezra did change his mind when he was quite young. I think it was like 21 or something. But for other people, I feel like, I think a lot of them have absorbed part of the arguments, but like, watch the hands, not the mouth. I think a truly damning example of this is the effective altruist movement, which is deeply ingrained with basically all three of these organizations to various degrees.
And look, again, the fact of altruists, for the most part, are really well-meaning, good-hearted, smart people trying to do the right thing. Are there bad apples? Of course there are. You know, no, no question about it. Are there weird, culty dynamics? Of course there are. There exists in any large movement of this kind. Are there, you know, weird untolds? Absolutely. But most individuals are good people. My truth of the matter is, is that, you know, effective altruists and people like Eliezer are speaking out against these corporations for racing. Eliezer was one of the people who helped Dario found deep-night.
He was one of the people who introduced Dario and Shane Legg to Peter Teal for initial funding. I don't know how much it helped them, but it was something I was thinking about at the time. How to, you know, push on. Open AI was founded by people many of which were effective altruists or effective altruist adjacent and open philanthropy. The largest funder of effective altruist goals gave open AI a very large early grant. It is, I think, the second largest grant they've ever made to an AI organization. I think it was $30 million roughly. It was given to open AI in the early days to make it a safe AGI lab.
In fact, from what I hear, it was Dario Amode later, CEO of Anthropic, who suggested to Elon Musk to create open AI as a, you know, counter lab to deep-night. And as you probably know, like early opening AI was also very open source focused. They changed their tune about that. And then later, you know, after Sam got involved a lot of drama there about Elon and Sam and such that I have not privy too. I don't know exactly what happened, but some drama occurred later on. Dario led the project for GPT-2. And then the project for GPT-3. It was the one pushing scaling laws. It was the one making these systems stronger and more powerful.
And pushing forward and pushing forward on those access. And then after GPT-3, citing security concerns and disagreements with Sam and the direction of opening AI, which seems, you know, completely plausible to me and have a reason to doubt this per se. That he and a bunch of other people, taught people of the GPT-3 authors and people left to defend Anthropic. And what did they do in Anthropic? They build larger models and other racing. And the recent pitch deck was leaked from Anthropic. They are talking about how they are going to build a 10x more powerful model in GPT-4. And to this day, you will have people in the AI safety community where vigorously defend Anthropic or even opening AI or deep-mind.
As the safety warrant to know this is good, actually, you know, there are friends. But like, you know, there's all these weird incentives going on there. Like, you know, like the president of Anthropic, Daniela is a Dario sister, is married to Holden Karnowski, who was the CEO of Open Philanthropy. Like, there's like, and this is a great example of watch the hands. What has happened is that these people stated very clearly that they think AI risk is real. This is a huge problem, et cetera, et cetera. But what has happened? What has actually occurred? Is they have accelerated the race like no one else?
And this is, I'm talking about this publicly for one of the first times, I think, because I think this is so important to make this clear. What is going on here? I've gotten quite shunned by the EA community for a lot of these, you know, angry comments. But this is, this is a common feature. This is, and the same thing with politicians. I've talked to many politicians. They come to me and I'd say them, oh, this is dangerous. And I'd say, well, Open AI told me that they tested their model and it was safe. And I'm like, man, okay, where do we start with this? So like, none of this is surprising.
It's corporations doing corporation things. It's Microsoft lobby is doing their Microsoft lobbying thing. I'm not mad. I feel like, this is just, this is what any, I think, reasonable political analyst would have predicted about how the discourse on AI safety would go before we say it. If I talked to, like, an old school, you know, like, I know, like, you know, environmental activists who was there for like the oil lobbying stuff and I asked them, how do you predict the current state of AI discourse is? You would probably make the same predictions for where it currently is. Or like, yeah, people are not unbored with like, oil pollution.
I think it's bad. Of course, companies have lawyers. They have lobbyists. And they have a lot of great excuses. And they have a lot of great comments about, like, oh, we should do self-regulation. Oh, let's do the oil companies, you know, regulate themselves. That's it. The oil companies set the safety standards for their oil, you know, extraction, which is, you know, a lot of people at open AI pushing for evals, you know, for safety evals, which is a great incentive. But this obviously has to be nonpartisan. This obviously has to done with people that are, you know, neutral parties. This is currently not the case.
You know, a lot of the people doing the evaluations are people who either, you know, work at one of these organizations or used to work at one of these organizations. And this is just kind of, it's kind of silly. I'm kind of just getting the intuition that if we re-rolled the dice on this universe and we plopped out Sam Altman in open AI, something else would be there, instead. And so, like, we could go down the rabbit hole of just like, hey, let's talk about the personality disposition of each of these founders. But I think it's really just about the nature of the problem itself that it doesn't really matter.
Like, take out Google, take out Anthropic, take out open AI, and you'll just find three other companies that will take that place, take their place in the race. Like, the race conditions are race conditions that expand beyond the current set of players. Do you agree with this intuition? I'm more skeptical about that than, I think, you are. I think it is true in the limit. Like, eventually, someone will figure out AGI. But I think you're underestimating how much ideology is actually at work here. GPT-4 was not a coldly calculated, you know, business decision. It was an ideological decision. GPT-4 costs like $100 million or $1 billion to build.
It's not meant to make that much revenue. It's extremely expensive. It's extremely risky. It could have blown up at any time. This is not what a Goldman Sachs does. No Goldman Sachs is going to build a GPT-3 or GPT-4. No coldly rational organization like this is going to take risks like this. And I'm saying the rational economic actor would not have chosen to produce chat GPT-4. Yeah. After they have chat GPT, sure they might have liked it. But no rational actor would have let, at least not now. You know, maybe when the cost comes down, you know, 10 more years of progress, maybe then. So I do think eventually would have happened.
But we do have people who are accelerating. It's not just that, like, oh, this is the, this is the one time where it happens. No, it could have happened in 10 years. It probably could have happened a few years earlier, you know. It could, but like not many years earlier. I think, you know, maybe other people would have aroused it's possible. I'm not saying it's impossible. Like, you know, we can't re-roll history anymore. Yeah, I don't like these arguments too much about, like, oh, it's just incentive is just a race. Also, because this, one of my maybe controversial beliefs is that I think that, um, people have way more control over reality than they think they do.
I think that actually the world is way more plastic and way more controllable than people think it is. I think that individuals with high agency can get a lot more done than people think they can. I think that individual actors matter a lot. Individual great, great people, you know, or politicians or activists or whatever can make a huge difference, actually. There's a saying forgot who was from, and he's like, never underestimate the ability of a small group of dedicated people to change history. In fact, it's the only thing that ever has. So, what you're saying is that, what I hear you saying is that these, the, because of the commitment to building chat GBD4, even though it was economically just non-rational, you're saying, there is ideological as in, therefore, these people are motivated by something, something else.
An ideology, maybe some sort of glory. I don't know, maybe the idea of just, like, creating the AGI is so attractive that they want to, they want to be the ones. And so, that is a little bit more of a pernicious problem, because that is, like, harder to tinker around with with external incentives, correct? Yeah. And to be clear, it's not just that, actually state this on their own personal blocks. Right. Like, I am not doing just pure psychoanalysis here. Like, this is literal actual statements. You can actually read written by these people. And, like, you know, you know, that will be confirmed by their friends and so on.
Like, you know, not all of it, but, like, these are not wild speculations that I'm pulling out of nowhere. You know, maybe they've changed their minds. You know, maybe there's some subtlety to it. I'm not dismissing these possibilities to be easier. But I'm saying, ignoring that there is an ideological component, I think, is not correct. I think it is very reasonable. There is an ideological or glory or a aesthetic preference, in some sense. There is a, in retrospect, really rather chilling, interview with Joffrey Hinton, one of the, you know, Godfather's of AI from, I think, like, 2015 or 20.
I don't remember, like, I don't remember. And he, the article ends on, I'm asking, like, well, you know, I think these things could be dangerous. Like, why would you do this? And Hinton, basic answer as well, sometimes the, the, the lure of discovery is just so sweet, you can't resist. And this is very grimly prophetic in the, in the sense that now, Joffrey Hinton has disavowed his life's work and has now come out in favor of that, actually, this is an existential risk. This could kill everybody. I was in a lecture with him that he gave in Cambridge a few weeks ago. And he's a wonderful lecturer.
And it was, it was a very lovely lecture. But it basically ended on this note of just, like, well, it turns out, yeah, probably, that this is, yeah, this is going to kill everybody. And I really don't know what to do about it. Yeah. Damn. It was just, like, really this, like, weird note, like, even Yashua Bangu, one of the other, like, touring award winners. So one of the, the most senior, most respected people in the entire field of AI has said that he feels lost after having come to realize just how, it's regret his life's work. Imagine that. Imagine one of the most senior professors, one of the most senior scientists to ever, to exist in this field, who has built this field, saying that he regrets his life work.
Like, this is unprecedented. This is an, this is unprecedented, basically, in history. I mean, you know, there's some cases. But this is, this is a truly extraordinary scenario for something to be this, so I think a lot of it is just curiosity, beauty, fun, glory, et cetera. And you're correct. This makes purely rational incentives harder to control. I think this is, this is a part of it. If we try to model this as a purely capitalist problem, this is a purely money problem, I think we would be not get the correct solution. I think this is, again, where we just have to be pragmatic. I'm not being just mental.
I'm just being like, okay, let's be pragmatic. What can we do? Like, where did things need to come? And like, the truth is just this is where government has to step in. Like at this point, like these people are not going to stop. They've had many opportunities too, and they will not do so. Uh, government just has to make them. Metamask has something new. Introducing Metamask Portfolio. Metamask Portfolio is the best way to view your crypto portfolio from a holistic level. See everything across all the chains all at once. In your portfolio, Metamask will report the aggregate value of all the assets in your Metamask wallets.
And even the other wallets you import too. But Metamask Portfolio isn't just a passive portfolio viewer. It is a place to do all of the money verbs that make DeFi so powerful. You can buy, swap, bridge, and stake your crypto assets. So not only is Metamask the easiest place to see your wallets in aggregate, but it's also a powerful battle station for all of your DeFi moves. So go check out your Metamask Portfolio, because it's waiting for you to open it up. Check it out at portfolio.metamask.io. You know Uniswap? It's the world's largest decentralized exchange, with over $1.4 trillion in trading volume.
You know this because we talk about it endlessly on Bayquus. It's Uniswap, but Uniswap is becoming so much more. Uniswap labs just released the Uniswap mobile wallet for iOS. The newest, easiest way to trade tokens on the go. With a Uniswap wallet, you can easily create or import a new wallet. Buy crypto on any available exchange with your debit card, with extremely low fiat on-ramp fees, and you can seamlessly swap on mainnet, polygon, arbitrum, and optimism. On the Uniswap mobile wallet, you can store and display your beautiful NFTs, and you can also explore Web3 with the in-app search features, market leaderboards, and price drives, or use wallet connect to connect to any Web3 application.
So you can now go directly to DeFi with the Uniswap mobile wallet. Safe, simple custody from the most trusted team in DeFi. Download the Uniswap wallet today on iOS. There's a link in the show notes. Arbitrum 1 is pioneering the world of secure Ethereum scalability, and is continuing to accelerate the Web3 landscape. Hundreds of projects have already deployed on Arbitrum 1, producing flourishing DeFi and NFT ecosystems. With a recent addition of Arbitrum Nova, gaming and social dApps like Reddit are also now calling Arbitrum home. Both Arbitrum 1 and Nova, leverage the security and decentralization of Ethereum and provide a builder experience that's intuitive, familiar, and fully EVM compatible.
On Arbitrum, both builders and users will experience faster transaction speeds with significantly lower gas fees, with Arbitrum's recent migration to Arbitrum Nitro, it's also now 10 times faster than before. Visit Arbitrum.io, where you can join the community, dive into the developer docs, bridge your assets, and start building your first app. With Arbitrum, experience Web3 development, the way it was meant to be. Secure, fast, cheap, and friction-free. I was going to ask you, like, what are the next steps here? Because I resonated with the quote that you said where the man feels lost. Because at the end of any sort of AI podcast, and again, our AI podcast are primarily alignment and safety podcasts.
I feel lost. I feel a little bit hopeless. And so I think what you just suggested is like the easiest thing, the lowest hanging fruit that you see possible, which is that government has to step in. What does that look like to you? How do we get that process started? Like, what are your ideas here? Yeah. And so again, I want to call back to the beginning of this podcast and just be like, this is not about government you know, you're from a crypto background. I assume many of your viewers are quite skeptical of the government. I think there's very good reasons to be very skeptical about material distaste.
Yeah. Totally understood. You know, I run a business. You know, nothing makes you a libertarian faster than founding a business. You know, and noticing all the red tape you have to go through. Like, I understand. I understand. But this is not about good about. It's not about ethics judgment. This is not about, oh, you know, they've mess up housing policy but it's, it's way more about, okay, let's be practical here. What are the options? What can we do? So, this is the kind of thing the government exists for. You know, if other companies are polluting, they're just, you know, putting poisonous the river or whatever.
And you ask them nicely. They don't care. And they keep doing it. You know, maybe they cite some reasons about, you know, acceleration or, like, races or whatever. Then you send police officers to make them stop. You know, like, they give the polite knock and the polite letter and you tell them, look, you knock this off or you're in deep trouble. So, this is going to have to happen. This is not a solution to be clear. I want to be very clear. None of this is a solution. This buys you time. This doesn't solve the whole problem of technology and, you know, the future of mankind. This is, no one will claim that.
No one will claim that, you know, us, you know, shutting down some of these dangerous experiments is, you know, going to solve us. This is the last time. The, we can talk about this in a second. But the longer story, of course, is, is that if we really want to solve this, if you want a good future for humanity, I'm going to have to do a lot more than this. I'm going to have to do a lot more than this. I'm going to have to solve a lot of technical problems and a lot of political problems, a lot of philosophical problems, if we want to get there. And it's unfortunately not optional. But the first, most clearer step from my perspective is that I, I mean, to be blunt, I don't currently see any good timeline, in which there is not a pause.
If we don't buy more time, we just continue accelerating, we're just pushing as hard as possible, as fast as possible, with all the resources our economy can muster, we're not going to make it. We're going to, type one system is going to pop out, and that's just going to be it. You know, and, you know, we're not even going to make it to a type two system. If we make it to a type two system, Metas can immediately make it open source, and then we die that way. But, you know, I don't even think we're going to get to type two systems. So, the government needs to step in, and slow these things down.
Luckily, there are actually very good levers that the government does have access to in current legal frameworks that can be used with quite small impacts on the wider economy. There is, luckily, we are in a scenario where really 99% of AI is completely fine. You know, we love, you know, medical research, you know, AI, or, you know, using particle physics, or, you know, doing like, you know, picture generation stuff, even so that has some copyright issues. Okay, let's not open that can, worms. But, like, 99% of AI is great. It's like, you know, it's progress as technology, it will provide, you know, great benefit to society, you know, fantastic.
You know, there will be ups and downsides, but the blast radius is contained. 99% of AI things has a small blast radius. Not necessarily small, not zero, not saying zero, but, you know, the kinds of blast radiuses that are society knows how to deal with, like we can handle, you know, if the internet had a huge blast radius, we're still here, we can handle the internet. And, so we can handle 99% of AI, just fine. There's only really a teeny tiny percentage of these like, hard, you know, huge, you know, GPT-4, GPT-4 plus, you know, large language models, general-purpose reasoning, agents, RL, you know, like far-end stuff that were basically all of the risks comes from, that, you know, all of the existential risks come from, not that there's no risk from other things, there are other risks but it's not existential.
The existential risk is really focused on this teeny, teeny tiny subset of like, you know, three, maybe, you know, companies in the United States, that's kind of it. And the government totally can just stop that. This is, we don't even need international treaties. The United States government can just unilaterally tomorrow, put out an executive order and just put it into this. It's that easy. And this exact suggestion I would give to the US government is to simply ban or require strict oversight and regulation of any individual training run which takes more than 10 to the power of 24 floating-point operations.
This is a unit of measurement for the amount of computing power used. And luckily, this is very easy to measure. We know GPUs have this in this much. No. Computing power run for this in this long. You can calculate how much computing power is going to run. This is very easy to measure. This is, it's very clear. There's very few companies that have access to computers this big. They are known to the US government. They are easily tracked down and they are, you know, settled in the US. These are, you know, a large cloud provider is Microsoft Google Amazon, et cetera. And they will comply. Of course they will.
Like, you know, they're not going to risk their entire multi-billion-dollar business for these, you know, few, you know, eccentric clients. So if the US government tells, you know, Amazon, no more large training runs. No more things above 10 to the power of 24. I expect to Amazon will just instantly comply. And that seems actually to be kind of a long-term solution with my limited understanding of AI, constraining how powerful our computation can be. Constraint how powerful the AI's can get universally across the board. And so while that law would be in effect, we have as, so long as no one's violating that law, we have as much time as we need.
Is that correct? That's my intuition. Fortunately, no. I wish, though. Yeah. Oh, yeah. Indeed. The truth is, is that we are not an exponential. We're on a double exponential. There's two exponentials. Is it hardware plus software? Yes. That's correct. We are both on an exponential and hardware and an exponential in software. If we can cap the exponential and hardware, again, you know, if people want access to their small, AI's at home, they want to do stable diffusion, they want, they're completely fine. You know, you know, maybe the government gets involved for copyright reasons or something, but like, from my existential risk perspective, I'm quite fine with that.
But it's not these frontier runs. So, you know, for, for calibration, 10 to the 23 is roughly how much is estimated GPT-3 used. GPT-4 probably 10 to the 24 or 10 to the 25, and we expect that, you know, GPT-5 will probably take another, you know, factor of 10 on top of them. And so, if we cap a 10 to the 24, I personally am confident that algorithms will improve, and you could still have an existential risk in that regard. I actually expect it's possible 10 to the power of 23. It's probably powerful, you know, capable of even less than that. It's probably possible. I expect to do that. You need algorithms that we don't currently have.
Like, I don't think currently, you can make GPT-3 existential interest. I think it's currently possible. But I'm not confident it's going to be true in 10 years, or in 20, or in 30. We will develop better algorithms for bootstrapping for, you know, more efficient, you know, more sample efficient learning for better RL planning things for bootstrapping. And like, we have a lot of things that we will discover and that we will continue to improve. There's lots of low-hanging fruit here still. So, but this does buy us time. This does buy us a lot of time. This buys us a lot of time, where we can do...
You can cut down the acceleration by half. Exactly. It makes a huge difference. And this is like, it's not as safe as it makes a huge difference. It gives us the time to do safety research, to build gliders, to build rockets. It gives us time to, you know, actually study these systems, like, actually understand, to integrate them to society, to figure out what is the right regulation. To integrate these things into society. Like, a thing that like, you know, opening, I like to say, is like, oh, we think the safest thing is to do iterative deployment and so they can integrate to society, which is code word for, lowly, we release it immediately, and just, you know, throw it down the market.
If open AI, had built a GPT-3, and then completely stopped, and, you know, and done, you know, only say GPT-3 until they fully understood it, they had full safety control of it. And, you know, all society had, like, totally integrated it, culturally, and we had, like, regulation that, like, totally handles it, and we're all comfortable with it. And then they had produced GPT-4. Yeah, you know what? Fine. That's fair. Honestly, totally fair. If this was the way our society handled AI, fantastic. I think this is great. You know, I want technological progress, too. Of course. I just want it to be, go well.
And, you know, this is not the shape of a technological path that goes well. Yes. What you're saying is that, instead of testing chat GPT-3 or 4 inside of a contained environment in the lab, so that we can fully unpack it and understand it, open AI is just eating it out into the world, and using the world as the test bed, not the contained environment as the test bed. It's even worse than that. Nevermind a contained environment. They're eating out, you know, the one thing, and then eating out the second one before we've even, you know, recovered from whiplash from the first one. Yeah. And so just to really unpack these two exponential curves, we have the hardware curve and the software curve.
The hardware curve is the thing that you're saying that we can easily and objectively place a cap on a government-inforced cap on that curve with that specific certain to the power of a number that we can totally measure. And so that seems like a solved problem if we can actually get the governments to do that. Then there's the other equation which is like, we are still in the early days of these AI models in the first place. And so even if we do cap the hardware, we can still have at least a handful of orders of magnitudes, efficiency in the algorithms that have access to that hardware. And so we have like Nvidia, the GPU supplier on one side.
Maybe there's others in the game as well. I don't know how relevant AMD is in the world of AI. But Nvidia is the big player that I think most people know. On one side, supplying GPUs into the market. And then we have the consumers of those GPUs which are the big people racing, right? The OpenAI's. And so we need to, in order to have a complete containment of the problem, there needs to be control on both sectors. And you're saying the hardware side of things is relatively objective and easy. And we can prescribe action steps for governments to take those steps. And I'm assuming, on the software side of things, it's a little bit more gray and we don't know how to proceed that well.
It's my intuition corrector. Yeah, that's definitely case. I mean, you know, easy is a strong word on the hardware side. I've talked to many people in policy positions and many of them are like, this is literally impossible you are insane. Other people, though, have said, oh, no, this is like totally some of the government can do, especially using national defense, you know, arguments. Like, you know, we already have a chip ban where we don't export H100s to China and stuff like this. We already track this. Like, the US government totally has the affordances and the capability, you know, it has the capabilities and the incentives to do this.
This is totally within their wheelhouse. This is not some crazy new thing that has to be invented, but getting the government to do anything is hell. So this is a massive undertaking. That being said. So software is much more complicated. I don't think we can or should try to ban mathematics. That seems insane. I don't think this is a kind of level of coordination that humanity is capable of. You know, maybe we're like super smart enlightened aliens. We could maybe like all handshake on it. But like, I don't think humanity can really work that way. But there are things that can be done here. There are actual straight forward, very easy and clear things that can be done here, actually, which is again, doesn't say, doesn't fix everything, but it does by time.
So in particular, the reason I think there are such easy wins here is because things are so bad. Because things are so bad. No one's even trying to control these things. There's a lot of low hanging fruit. So one of the like very obvious things, which is not again, I'm not saying it's easy. But I think this would make a difference. Is I think there should be strict liability for not just model deployers, but also model developers. You develop an AI system and someone uses it to commit a crime. You should be charged as if you committed the crime yourself. You should be liable. And strict liability, let's take a moment to unpack that.
Strict liability is when you the individual, that falls back on you. Yes. Just want to define that term before we move on too far. Yes. That's correct. I think this would be the ideal case. Again, I have had a comment section. I'm sure is yelling at me right now. I have heard from plenty of people of policy people telling me how this is impossible. But I have also heard from very senior people that this is possible. Basically, the number one step is removing 230, section 230, protections from AI systems. This is already being discussed by senators in the Senate. So section 230 is the like, you know, internet neutrality thing.
We're like, if you host content, which is illegal, but you didn't know about it and you remove it. You're not liable for it. Your users were doing it and you had no hands in it. One of the obvious things is just do not apply section 230 to AI. Simply if you develop a model or a system and AI system, which causes actual criminal harm, you are criminally liable for it personally. This is currently not the case. I think this is the case. This is currently not the case. Currently, there's a, again, look, I love open servers. I love my nerd friends. But it's insane. Is that some nerd, somewhere in the university, can develop a voice cloning system.
You know, take 15 second audio of anyone, perfect on their voice, and use it, you know, scam their parents to, you know, call and bomb threats, you know, whatever, to totally ruin their lives. And the person who developed that model, we posted it to GitHub, has zero liability. It's not only that they don't have any legal liability. They don't even feel responsible. The people who do this feel no shame. It's not part of culture of open source culture to even feel very responsible for this kind of thing. It's like, oh, you developed a cool thing. You know, throw it out there. It's fine. You know, let the lawyers figure out how society thinks it's not your problem.
Like, I had an actual conversation. It was a real conversation I had with some tech people. And I was talking about how I think this should be strict liability. And then one of the people, sincerely, may the point, well, that, no, that's impossible. We can't do that. Like, how would they possibly, you ever determined who is at fault? This couldn't be done. And then a friend of mine was like, courts. That is what courts are for. How do you not know what courts are? So there's a huge disconnect between a lot of the tech sector and like, how civil society actually functions. Like the, the infrastructure of civil, like, actual functioning government and societies and courts and so on.
We have dealt with, you know, things that enable criminal activity in the past. And, you know, like, like, another great example is automobiles. So automobiles, there was a huge fight. I think it was in the seventies if I remember correctly. I might get the dates wrong. I'm, you know, bad with remembering exact numbers. There is a huge fight, legal battle, where automakers were arguing that they were not responsible for any deaths that occurred due to cars. And including, if their car is malfunctioned. And a lot of people push back at that. They said, well, no, if your car malfunctions, if the brakes, you know, don't work, if it catches fire or whatever, no, the maker of the car should be liable for that.
Not, you know, the driver, like, how, it's not the his fault that the car caught fire. And this was a massive legal battle. This was not, and now, nowadays, we do hold, you know, car companies liable. If you build a car and it explodes and it kills somebody, yeah, you, your responsibility, you build the car. You put the car out there, like, of course, you're responsible for this. And of course, there is a spectrum here. You know, there's a large spectrum. I'm not saying I know where on the spectrum is right thing. You know, should everyone who developed Linux be held personally responsible if someone uses Linux for something bad?
I think no. But it's a spectrum. It's not a yes or no. It's not everyone is hold responsible or no one ever. It's a spectrum. And this is how laws work. This is how courts work. And so the idea is that society have to find where on the spectrum are be comfortable. And I'm simply stating, I think we're way on the one side, and we should move more towards the middle where there is actual accountability. That same conversation I had with this person was very funny as a different person actually, but same conversation. When I talked about strict liability, someone called me out. And they're like, okay, Connor, well, you developed open source technology.
Do you want to be held accountable if someone you use as your language models, you developed a loopery eye to cause harm? And my answer is yes. Yes. I would like to live in a society where eyes technologists, if I cause harm to civil society, if I cause harm to other people, that I am held accountable to this. I think this is good. I think this is the kind of society I would like to live in. I can see a large number of the crypto people, which are 95% of this audience, recoiling at this. And I'm sorry. I want to give an anecdote from the crypto world, which I think is universally accepted as like we are in a legal battle with the United States Department of Treasury.
Because they deemed, I don't know how familiar you are with this. But this smart contract on the theory I'm called tornado cash, they deemed using that software uploaded to the Ethereum blockchain, using that as an American citizen is illegal. And so I'm personally actually suing Janet Yelena. And I think this is a very interesting question. I think this is a very interesting question. I think this is a very interesting question. I am personally actually suing Janet Yelena in the Department of Treasury based on that. And saying somebody, open source developer, they live in the Netherlands uploaded this piece of hardware to Ethereum.
And I was able to use that as a tool, a neutral tool. There happened, I was achieving some sort of financial privacy because Ethereum is transparent and open and everywhere can see my transactions. And I would like to have created some sort of private version of my wallet that no one used, that no one knew about. So I used tornado cash that end simultaneously. So did North Korea. And so my money was right next to North Korea. And so the Department of Treasury deemed this piece of software to be illegal to use by all Americans. And so a few of us in the crypto world banded together into sue the Department of Treasury saying, hey, you cannot make a neutral piece of software illegal just because some bad actors are doing bad things with it, including perhaps funding the development of nuclear weapons.
And so I think this is a very, very interesting question. I think this is a very interesting question. I think this is a very very interesting question. And so I can see where, but like when you tell me this about AI is like, how else do we solve this AI alignment problem other than giving strict liability to the people that create the models that ultimately destroy the world? To me, I'm like, well, I don't have a better solution. And so I don't know what else to do. And so I'm torn at this. I see the crypto code libertarian inside of me on one side of things. And then the, hey, let's not die from AI on the other side of things.
And I don't want to see the conflict here. Yeah. And I'm sure you have run into these conversations a number of times. Absolutely. And like, I think, so thank you for sharing that. Like, that's, that's a great illustration of, again, I'm coming back to the beginning of the podcast. It's not about good and bad. It's about finding the right thing. I think you, Suey and the US government is fantastic. Like, I think you're doing a civil duty here. I think you're, you're improving society for all of us by doing this. Whatever comes out of this case, like, someone has to do it. This is how society progresses.
This is the mechanism by which our society finds consensus on what is the correct dial setting. It's not that the government magically knows the correct dial setting. No, this is why in the USA, we do sue. And this is great. This is good. It's a huge pain. I know this must be so much pain for you to do. Oh, the shit. Like, such a, such a, and that's why I'm like thinking. Yeah, like, you know, that's why I'm thanking you and these guys, you know, I don't have a horse in this race. You know, I don't know whether this is good or bad. I don't have a horse in this race. But I think it's good that it's going to court.
I think this is good that someone is holding the position. This is good. And someone is holding this position. This is bad. And this should be thought out. This should be debated. This shouldn't be taken for granted that things are obviously good or obviously bad. Again, being in this conversation, things are good or bad. It's about decisions. It's about consequences. I don't know. Maybe the software being banned is net good. Maybe I don't know. Maybe it's not like the way you describe it. I don't know. Like I really don't know. And I would love for you know, people who don't understand this technology much better than me.
People like you. And you know, people, you know, the government who you hopefully have like a better understanding of international security something to battle it out. And like actually hopefully find truth. And our courts are not perfect. You know, I'm sure there's going to be a bunch of bullshit. And like whatever the verdict is, no one's going to be happy with it. You know, I understand. But this is how our society works. This is how we make progress. This is how we, as a democratic, you know, liberal, you know, judicial, you know, rule of law, rule of law country make progress. And I'm saying that this is painful, but it just has to be done.
It's a price of living in a complex society. It's a price of playing in a complex society with complex trade-offs, with complex technologies, which I think crypto is. I think crypto is an amazing technology that has incredibly complex, you know, benefits and downsides. You know, there is blast radius. You know, FTX was a huge blast radius. You know, it is massive blast radius, right? And that doesn't mean crypto is bad. No, and like, of course it doesn't. Like just because there's scammers doesn't mean it's bad. But it doesn't mean it's good. It's just a technology. It's neutral. The same way AI is.
It is not good or bad. It's about how do we as a society digest this technology? How do we get as much of the good stuff while preventing as much as the bad stuff? And this is simply not easy. This is all I'm saying. I'm not saying I have the perfect solution. It solves all the problems forever. Don't worry. Just do this. I'm saying this is a hard process. This is like doing science. Like in a sense, going to court is like doing science. You're doing like, you know, law making science or engineering. You're doing the epistemological labor that needs to get done to actually get to the right kind of civilizational software that we want to be running.
Because we don't know what the right software is. We don't know what the right laws are. We don't know what the right liability or the right insurance games. I don't know is, right? And we have to find out. Someone has to come up with them and they have to debate against people who come up with alternative things. And we have to try things out and some of them blow up and then people sue and then we try another stage and so on. So I think this is a natural part of how a functioning society should work. And sometimes this is a huge massive pain. I understand. But this is like, I've talked before about like, okay, humanity can't coordinate that well.
We can coordinate. But like, this is what coordination actually looked like. Coordination does not look like, oh, we're all happy friends and we hold hands and we all sing songs. That's not what I mean when I use the work coordination. I'm a coordination. I mean we sue each other. I mean we have debates. I mean we have we fight. We argue civilly, you know, no violence, you know, we're very civil about it. We have rules. We follow laws. We also follow laws that we don't agree with. Like there's a bunch of laws I don't agree with. But I am willing to make compromises on this. And I'm willing to compromise with the government and with my fellow citizens that I will like about, you know, if I go to a country where maybe I don't agree with all their laws, well, I'm there.
I'm not going to break them. And this is not just because of like fear of retribution. It's also because of contracts and just like, you know, making deals and like being fair and changing processes. I think violence is extremely bad for this reason. Because violence violates our contracts. Our social contract is this is not how we solve disputes. If you disagree with something you don't use violence. You have debates. You start a campaign. You start a, you know, you start you talk about a podcast you know, whatever. Like there we have mechanisms and we should use them. And they're not great.
Like in case you haven't seen Twitter lately, our coordination mechanisms aren't great. They're not very good. But they're something. And we can do better. We can coordinate. It is possible. It's not easy again. At the beginning of this conversation, when we were trying to unpack the different categories for how AI might come to destroy the world, you called this a deeply meta problem. You talked about how this, how we need to solve philosophical problems about what it means to be human. It's a problem that really goes for the heart of the human condition. And I've been talking to quite a large number of AI people in the last couple months or so.
And it seems, it's been very interesting to me every time the topic of AI alignment and ABI safety comes up, which is almost all the time. How different approaches the conversation can take across the board. And it seems to be that there are so many different problems that need to be solved in order to solve the AI safety problem in the first place. I'm wondering if you could just unpack. I'm pointing you in a direction here and seeing if I can just unleash you here. But it seems to be like the AI alignment problem goes very deep as to what it means to be human and what it means to solve problems in the first place.
And especially when we have our approaching this technology that I think if we as humans just like developed as a society over and over and over and over again, like we reroll the dice of the human experiment, you arrive at AGI almost all the time, like 99.99% of the time. So it seems to be like, hey, humanity we have arrived at what seems to be the largest problem that humanity has ever faced. Nuclear war, nuclear bombs big problem, not as big. Right? Like disease, big problem, not as big. This is like the big problem. And so it seems to be posing very big questions. And so as it comes to the end of this conversation, I'm wondering how to tie this off with just understanding the relationship between what it means to be human and how to solve AI alignment.
Just so what does this vector of conversation sound like to you? Sounds like the real question, you know, the ultimate question, our alignment is an important field. And sometimes its definitions get stretched to the point where like it starts encompassing these larger and larger meta problems. I think this is not a coincidence. I think really in a potential rephrasing of the alignment problem is the general question of how do you control a powerful system using a weaker system? This is something that is not unique to AI. This is a problem that exists in organizations in multicellular organisms.
Cancer is a form of misalignment. Some of your cells start having different values from the rest of your cells. And they start taking actions of maximizing actions which are harmful to the rest of your organism. Our organism has alignment mechanisms. It has stuff like apoptosis or like you sell suicide that when there's mechanisms that detects that a cell is going down a path which might be dangerous, the cell kills itself. And sometimes these mechanisms don't work and you get cancers. So this might be a bit of a torturous metaphor. But like I do think there's actually a deeper truth here in that alignment and power and coordination are in a sense all facets of the same coin.
They're all facets of the same problem of how do you organize chaos? How do you create stable systems equilibria? How do you resist entropy? How do you resist just the eternal cold noise of the universe? Because by default, that's all there is. By default, there's only cold nothingness. That's what the universe is. It's space. There's nothing else. Space and just random particles. But then things can go here and to complexity. They can go here into they can they can they can form, you know, whether by simple mechanisms such as gravity or chemical things to you know, create planets and stars. And then we have higher and higher level things.
We start you know, seeing complex molecules. Then we start seeing, you know, the first self replicators we see, you know, RNA molecules and, you know, protocells, you see, you know, early bacteria and archaea. And then they start to coordinate among each other. You know, a lot of theories is that the mitochondria and the cell is actually was one's a separate organism that got absorbed into our cells into eukaryotic cells. And so there's a coronation happening there in a in a very abstract sense. And then eventually these cells learn to organize it even larger scale to organize into multicellular organisms.
And then first very primitive, you know, probably like sponges or like, you know, like jellyfish or something, probably very simple organisms. But then stuff started to specialize. But to specialize, you have to coordinate better. You have to have better brain cell to live. It needs a lot of digestion cells to help it out. And a lot of blood cells to help it out. And a lot of like, like the amount of infrastructure that needs to be in place and coordinate it to support a human brain is ridiculous. There's a massive, huge system. And everyone has to play their part. You know, some muscle cell can't just be like, ah, you know, screw it.
I'm going to be a neuron now. You know, if that happens, we call this cancer. You know, we, this is a disease. This is McLeven has a lot of great work on this. I'm like bioelectricity and like, you know, cell identity and stuff like this. And then it goes further. And then we get to the the meta level to the, you know, we go from the genes to the memes, you know, as you know, Steven Haas, uh, uh, uh, uh, uh, uh, uh, dockens might say, um, where, uh, we go from purely genetic evolution or like multisolar evolution to mimetic evolution to culture evolution. There becomes now it's about cultures and tribes and civilizations and, uh, you know, companies and gods and religions and these kinds of things.
And these can also coordinate and mutate and, you know, war and, you know, change. So there's this, this is recurring motif of life, of these like higher order patterns of complexity and like, and coordination and stuff emerging. And the way for these things to emerge is through coordination. It is through coherence of smaller patterns, cohering into larger emergent patterns on higher levels of scale. And this is also where we humans are. We're nestled here somewhere in the hierarchy, you know, somewhere above multicellular, somewhere below, you know, pure mimetic, you know, religions or, you know, stories somewhere in between.
We're kind of like, you know, with one one foot we're in the realm of animals of like the earth. And one foot we're in the realm of the gods, we're in the realm of mimetics of information, of stories, of religions, we're kind of this interface between these two things. And this, so what is the next step from here? Like, where do we go? Like, what does it mean? Like, what does it mean for us to want something? Like, in a sense, like, you know, if we build AI, this is pushing us further into the realm of mimetics, of knowledge, of pure abstract software, of something that exists not as a brain, not as a specific piece of hardware, but as a purely, like as basic as spirit, you know, just like an immaterial, you know, code, it's the next step, you know, it goes up, it goes, you know, from the transition from biologic, purely biological, purely physical, physical, biological, you know, then half mimetic, half biological to purely mimetic, they can run on any way, where there's a CPU that can run its code, AI can copy itself, and it can exist and it's purely mimetic realm.
So, how do, what does this mean? Well, this means many things. It means, could be mean fantastic things. Culture allows us to do fantastic things. Our control over mimetics, our culture of technology, our science, it allows us to build this wonderful place. We're in like, look at this, you know, we've got internet and it's, you know, I can eat all the food I want, and it's great. My animal is feeling, my inner animal is feeling great about this whole like, civilization stuff. Seems pretty cool. But things can also go wrong. Last radius. As we dabble in the powerful, as we dabble in magic, you know, in like the, you know, technology, as we dabble in this, we, you know, last radius increases, power increases, and if we, and we need this is all just a very nice poetic reframing of where we start at this podcast.
It's about how do we deal with the upper level, you know? There's also a question of down level, like the question of cancer control or like, you know, medicine is the question of how do we align the levels below us? Or engineering or like physics? Are the questions of how do we align the levels below us with our values? Now we have the question, because we want some things, you know? There's many things I like, and there's many things I don't like. I don't like my friends to die. I don't like being in pain. I would like a beautiful universe with cosmopolitan art and adventure and all these nice things.
And this is possible. But it's not something that an animal can do. It's not something me or you as half animals could even do. It is something that God could do. It is something that a very, very powerful intelligence, a mimetic super creatures could do. It's the same way that the mimetic super creatures that we call civilization has built technology that you or me could never build by ourselves. And this is great. But what are our alignment mechanisms? You know, our cells have opoptosis. You know, they have a lot of stuff. And even that's not perfect. But now I'm saying, okay, medicine is great.
You know, physics is great. You know, we're developing a lower level alignment text. We also need to build the upper level alignment text. We need to build the upper level that we can understand mimetics and AI and optimization. And so unless this is magic, that's the cool thing in a sense. It is both magic and it's not magic. It's like information processing. It is things that can be understood. It is coordination. Humans could solve many coordination problems. You know, mean you can coordinate. You know, mean all the other people down the street. We can all live together peacefully. This is coordination.
It is controlling something, you know, working as part of something bigger than myself. And if we want a really good outcome, if we really want the world to go well, if we really want, you know, our far descendants, you know, whether they're human or not to, you know, live a great life. But we have to not blow this all up. We don't have to actually make sure that the next step we make is not just some random thing that, you know, someone cooked up on a server somewhere. But it's actually something we like. It's something we endorse. We don't want something to inherit the universe. And just, you know, blow it up or use it for nothingness.
We want the universe to be something that we like. And this is really the full problem of alignment. It's not just the question of, okay, how do we solve this narrow problem? How do we regulate this thing? It's how do we think about what does it mean for humans to want things? What do we want? You know, a lot of our preferences are culturally constructed. We want things, because their friends want it, or because they saw it on TV. Is that a real preference, or is that a fake preference? I mean, I don't know. Like this is something we should be asking. What does it mean to be good? Is that even a well-coherent concept?
If we can't agree in any of this, is there some way we can bargain? Is there some kind of, you know, harsanis, veil, type solution where, you know, we build some super-aligned type four system? It appears out what they all want, and makes some kind of like fair distribution. I don't know. What we really want is these type four system, type four aligned systems that are so aligned, that they figure it out for us. You know, they're like, I'm going to figure out all humans. I'm going to understand their psychology, their traumas, their life story. I'm going to deeply, deeply understand them, in ways that no human could understand them.
I will use all this, you know, incredible superhuman intelligence in order not to do something random, but to do what of these, you know, poor half animals actually would like, what would actually make them happy, what would actually be good for them. And I have no idea how to build such a thing. I mean, I have a few ideas, but I don't know. This system is not forbidden by physics. There's no reason that such a system cannot be constructed. But we are so far away from that. We're not even going to get to there. If we can even get to the point, where we can just have like systems that we can carefully use, we coordinate on, and we use them to, you know, make this a cool world.
You know, where people are happy, where people don't have to fear for the next day, where people aren't hungry, where people can just chill. That would be pretty nice. I, we can get there. It's technically possible. Obviously it's not free. Progress is not free. If you just take the shortest path on the, on the tech tree, you don't get to the final tech, you blow yourself up on the way. There is a final tech, somewhere deep, deep, deep down the tree. And we're very careful. But how we traverse this, if we, you know, improve our coordination tech, if we, you know, work carefully about this, and we, we, we, we, can get there.
And that's, you know, I think, what, like, Ali Ezra said, would mean is truly solving alignment. That final tech, there were, like, we have understood what are values. What does humanity mean? How do we bargain between values? How do we, you know, think about reality? How do we think about, you know, meaning? How do we think about all these things? And how, and not just that, but, you know, never mind the philosophy here. The code, how do we turn this? How do we build this? It's not, I think that I like to think about philosophy that I think is often lost in modern philosophy, is that good philosophy should make you stronger.
Philosophy should solve problems. The philosophy is ultimately going to solve problems. Why are you doing it? What, really, if you really do philosophy, if you really go to the end of philosophy, you should come out on the other side, with something that allows you to do these things, that allows you to build these systems, to create this great future, to make the universe good, to make it great. And I don't know what that means. I'm not claiming that I do. Oh, God, no. I barely have a few technical ideas about how we can get to type two systems. A little bit of an idea. But even that, it's going to be tough.
We just, all of us, you know, humanity, politics, everything. It's going to be heroic. It's going to be heroic task to get to that point. But hopefully, we just have to be heroic one more time. Connor, I really appreciate it all of that. That was a very cerebral and deep conversation. If bankless listeners are out there and they want to continue this conversation, where should we send them? I know that your company Conjecture has a fantastic blog. I can definitely put those links in the show notes. Is there any other place that people who want to continue this conversation might want to end up?
So our website is a bit, not that super up-to-date. But I thank you for your kind words anyways. I'm on Twitter at MP Collapse. I'm not super reactive, but I exist there. I sometimes hang around a loop three eyes, discord still, not super common. But, you know, if you ping me, there's a decent chance I'll respond at some point. I go on a lot of podcasts. I do a lot of, if you're interested in learning more about some of the regulation, things I'm talking about, there is a website that you've made called stop.ai. There's an askerisk, ask for the AI, is that we're not trying to stop all AI, just a specific type of AI.
There's a lot of content on there about like fleshing out some of these things. Yeah, and in general, you know, I'm around. I definitely think that's correct. A bankless nation, you know the deal, crypto is risky, ethos risky, AI alignment is risky and apparently we need a heroic effort to solve that problem. But nonetheless, you can lose what you put in. We are headed west. This is frontier. It's not for everyone, but we are glad you were with us on the bankless journey. Thanks a lot.