Debriefing the episode with Eliezer Yudkowsky. This one was so good, we had to share. The fate of humanity might depend on it.
The Debrief Episode goes out EVERY MONDAY for Bankless Citizens. Want the Debrief Episode? Get the Premium RSS feed by Subscribing to Bankless!
WATCH THE FULL EPISODE HERE: https://youtu.be/gA1sNLL6yg4
------ 🚀 SUBSCRIBE TO NEWSLETTER: https://newsletter.banklesshq.com/
----- Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.
Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here: https://www.bankless.com/disclosures
No summary for this episode yet.
Welcome to the debrief. This is our episode after the episode with Eliezer, Witkowski, David. I didn't realize that this was going to hit you so hard, man. It really did. I don't think I was ready for this. This wasn't your first time going down the AI alignment rabbit hole. No, certainly not. And I'd also read a lot of what Eliezer had said before, listened to previous podcasts, heard him make the case. But I think there's something much more visceral about this versus any other time I've read his writings or any other time I've seen him. And this is like, I feel like I was looking across someone who was like utterly defeated and he, he said he still had some hope left, but it didn't feel like that looking across to him on our zoom screen.
It was hope in the sense that like sometimes just people doubt themselves and that's an equivalent amount of hope for him. Yeah. This is a man who spent 20 years working on the AI alignment problem and education problem and is, um, as I understand, one of the foremost preeminent thinkers on the subject, um, who is basically like, it almost felt like to me that he was, um, throwing up his hands and saying, not throwing up his hands. He's basically saying, what more can I do? I'm just going to live out the rest of my days peacefully and, um, go die in the way that I think is best. You know what I mean?
It felt like a general on the battlefield who like knows that they've already lost and the army's about to get wiped out and saying to the troops, like, okay, go die as you see fit. You know what I mean? Like it, it had that kind of feeling. And, uh, toward the end, I don't know if, uh, Eliezer was getting emotional or like not. I was wondering about that. I mean, how can you not? It seemed like it, right? I mean, like, um, just, I think it's that it's also the, um, the sincerity through which he expressed these viewpoints. Like there's not a doubt in my mind that this man believes what he is saying.
Uh, and I combined that and I'm like, well, um, maybe this guy's like the vitalic of artificial intelligence or like AI alignment and he's given up, you know, like, um, what hope is there? And so like, I guess I was thinking we would enter this podcast and have like, well, here are all the ways that it could go really wrong. And I think we should be more concerned about these things and there needs to be more attention, but I thought there would be some silver lining lining right it's like you know like all the existential types of conversations about uh nuclear proliferation nuclear holocaust or about like global warming or you'll pick your poison biological weapons there's always like this but if we do xyz then like there could be a happy ending there was none of that here there was no happy ending and that's what hit me especially hard in this episode yeah and we in the agenda we're like all right what's the bull case for ai and like which is bankless language for just saying tell us the positives uh and then there's also like tell us and then also what the warnings like the the red flags that we need to look out for and i knew that this episode was going to be like oh there's a lot more warnings than there is blue sky yeah but yes i was not expecting uh like hey there's no blue sky it's only a pit of there's only the void there's that's the only thing um and i think perhaps like one of the reasons why you're reacting to this is just because like this is the guy who's on the furthest reaches of the frontier who's clearly smart about this who's clearly thought a lot about this and is is the guy to lead the charge against this and it seems like he gave up years ago yeah and he gave up after giving a big try like you know 2015 he had the attention of the world i feel like this is when Nick Bostrom's book came out, the book on super intelligence, I believe it's called.
And like this big conference and you had like the billionaires and sort of the tech elite, the Bill Gates and the Elon Musk's saying, yeah, this is a big issue. He thought that that was the moment. And then I feel like all of maybe the heroes or the helpers who are supposed to partner with him kind of disappointed him because it turned out all that they were interested in doing or all they ended up doing was getting wealthy off of new AI projects or getting some sort of social signal boost from this and not actually doing anything to help solve the problem. And so I feel like he's coming out of that too and he's just like, well, I guess this is how we die.
um and you know like we know the the peril of moloch traps and coordination failures and how intractable that they are this is the moloch the most molochy moloch that i've that there is basically yeah this is actually the like other things are like a moloch light or moloch's cousin no this is moloch as in like this is the last step there is no there's no there's no reality passed this MOLOC. It's like trying to stop the internet. How would you even go about doing that when there's such a tremendous economic upside for everybody to want to continue this project called the internet? It's like trying to stop electricity.
It's like, I mean, this ball is in motion. I don't know. I guess I just pray to God that this guy is wrong. Right. It wouldn't even be him being wrong. Well, that's what he said. He said the only chance is that I'm wrong or that he has there's some unforeseen way of solving this. It's like a logic problem, right? Like this is just this is just like this massive logic problem. And he's come to this like maze in this logic path where it's like you always end up there over and over and over again. My one question is, do you think he spent too much time in his own head thinking about this and has gone through like very dark paths as a result?
and it's kind of um no i mean i it makes and like i said it's it's it's like a logic problem right and so i think he's just gone through this logical path of deduction and he's come out of this conclusion and i think it's very normal for a lot of for people to not want to think it's like date very deep in our dna is like i don't want to think about death like i don't want to think about the end of humanity i'm going to think about literally anything else and so he's just the only guy who's like smart enough to articulate the position and committed enough to the actual logical process to get there and he just happens to be the one sober guy he was like hey all you other people who are trying to like make yourself naive in this cozy blanket of profits and revenue using ai uh like you guys are part of the problem um it's it has the feeling of like some of the early sentiments i've i've read of the physicists who created um the first atomic bombs right yeah like the oppenheimer project and like this feeling of like my god what have we unleashed and like out of that um them wondering uh how long humanity would actually last like that there is some sort of a you know even this is um band of physicists i believe grouping together who had equal concerns i i don't know the full history but of the doomsday clock right of like now we have to tell the world how dire this situation actually is of nuclear proliferation and how close we are to midnight on the doomsday clock.
I guess I've never really viewed the nuclear proliferation from that perspective, but that, that makes sense as in like, if you are a scientist who just saw a nuclear bomb go off for the first time, and then you kind of put the pieces together of like, Oh, soon everyone's going to have this power. Uh, and then all of a sudden, like, well, of course there's a doomsday clock because as soon as everyone has this power, the odds of somebody pressing the button drops to almost certainty over time. I think a large number of like a decent proportion of the physicists who like were involved in these projects, um, didn't think we would last, you know, another couple of decades.
Right. But you're not making the comparison between nuclear arms and AI because like the obvious, the, the, the difference here is that in the AI example, the nuclear bombs are sentient and have an agency to live. Yes. Um, At our expense. That's sort of the, yeah, I mean, we asked that question, and he's like, this is way worse. Doesn't he have the motivation to create like a doomsday clock type of social education apparatus because, like, what is the point? I mean, this is, I mean, he kept giving kind of this laundry detergent analogy, I think like for this idea that, that, um, creating an AI can be used, uh, you can create it in AI using garden variety ingredients.
Right. You don't have to have like, you know, enriched. The ability is it's, well, this is what, what Daniel Schmachtenberger talks about. We're just like the means, the means to destroy the world in many different ways is becoming easier and more accessible as technology progresses. and so this is like why any and every sentient civilization will always progress towards this inevitable inevitable outcome is because we will always make technology and we will always make ai but ai is not the only thing in this category it's just like the worst it's the worst one but there's also like the ability to make like a absolutely massively deadly virus in the comfort of your own home is soon to be in the hands of everyone because of bio bio engineering right Right.
But I always think on the other side, I know that's a huge threat, but I'm always like, but there are vaccines too. And that technology gets better. Right. This is why this is the worst one, because there is no, it's the assumption that the morality scale, morality ethics of AI and humans is completely just like divergent as in just like orthogonal is the word that Eliezer has used in other capacities. It's like AIs and AIs will start to create their own frameworks of morality and ethics and it will be insular to the ai species and it will not contain our morality and ethics it will be completely divergent from each other and so what they think is good or bad will be on a completely different plane of existence and their plane of existence won't intersect with ours and that's the crux that like every other technology does not have and why nuclear arms and uh uh generating a virus and what was the other one um nuclear generating a virus i mean nanobots could be another one right well that's it's really the morality conversation that's the separating factor here between the ai doomsday and all the other doomsdays yeah i that's why you asked why this hit me it's like yes i've dealt with the existential things you know like um obviously different coordination failures um before but the certainty of this from a like again i mean you go back to like i've only saw it once and I was half paying attention, but don't look up movie.
I don't even know if I quoted that movie scene correctly anyway, but like it's just, it felt like a scientist who had been spending decades trying to tell the world that this asteroid was approaching. And now is that kind of the end limit? Like, my God, these people aren't listening. We actually as a species don't have the ability to coordinate, to coordinate and solve this one and figure it out. and um yeah that was really like depressing like of the level of an asteroid is careening towards earth and we are doing nothing about it that's why it hit me so hard did it hit you like is it this but you know i i've done this before and i i'm not to say that you that you haven't because you said you've gone down this rabbit hole before i thought i was ready so this is this isn't new for you but like yeah I mean I remember like listening to Eliezer on Sam Harris's podcast way back when and I was like I was like painting my dad's house uh it was like my summer job and I was listening to it and I was like I was going through the existential crisis from 2018 right yeah and as I going through the existential crisis I was like oh this is bad like oh this is really bad but I need to you know get myself through like physical therapy school and like and so what I'm still going to do all the same things I am going to do tomorrow as a result of this information.
And so like, how is it going to impact your life? Like, you're still going to go pick up your kids, still going to go kiss your wife. Good night. You're still going to do the bankless podcast. Like what you're going to do about it. I guess, but like, I, I was more optimistic that we'd have a shot at like persisting past the next 100 years than I was coming out of this episode. Are you going to turn into like an AI doomer? And is that what Eliezer is? You know, is he an AI doomer? Is that somebody is that like how we should dismiss him? Like, is that a dismissal of his points? Or is that just like, I don't know, I guess maybe you're being more stoic about it than me right now, which is like, well, you know, if it's been a nice ride anyway, you know, I guess if, um, if Eliezer is, is right.
Um, I mean, you could drop everything and start, and we could turn the bankless podcast into the AI alignment problem podcast, and we could start to fight that fight if you wanted to. That is something that we could do. I guess, right? I guess. If you want to start to work towards solving this problem, because it's basically you go about living your life as is and just enjoy the fact that you're alive in the first place, or you turn into an AI doomer and you build your underground bunker for when the AI comes. There's no underground bunker. I think I would probably just enjoy life right now. Or you just turn your entire life into following Eliezer's and like start to join that coordination group, which I think I totally suggest that we do.
But I still kind of want to do the more normal things as well. I mean, I think that I mean, maybe in some way crypto. Right. I mean, we talk about solving coordination problems. I think we're nowhere near to like. Solving the coordination problem of artificial intelligence. In fact, this is part of the content we couldn't get to, bankless listeners, because it seemed so pointless. But we wanted to ask him questions like, crypto, now that we've created this programmable money system where the robots get bank accounts, and not only do they get bank accounts, they can actually build banks themselves.
Have we, as crypto, just empowered this artificial intelligence? We wanted to ask him questions like that. And like, but it just seemed so pointless by the time we got to it. It's like, of course his answer would be yes. But if you know, he would have to, he would have to be a different personality. He's like, he would have to put on the hat of like, Oh, you want me to be the AI bullish person? Let me take off my actual hat and put on my fake hat, which is my AI bull person. And then I'll be a fake person in order to act out. He was not fake at all in this conversation. That's one thing I'll say.
it. It's like, I expected him to be like, yeah, you know, you crypto people should be careful with what, what you're doing. Um, there are some good things and some ways you can raise money to fight this fight or solve coordination problems in other ways that could be advantageous. But, um, in other ways you're creating infrastructure to like, you know, increase the power and decrease the timeline through which an artificial general intelligence can come destroy us. I expected that but like by the time we got to it it's just pointless because I already knew what his answer was going to be it's like yes and it doesn't matter it's like that there's just a there's a nihilism it went past like an absurdism of like yeah we're screwed let's laugh about it like Rick and Morty style it got to like oh man this is heavy shit yeah like anyway that's how it hit me I don't know if it's gonna how it's gonna hit the listener some of you guys might be listening to this and be like I know much more about artificial gender intelligence.
I know about the counter arguments to someone like Eliezer. Is this even a technical artificial intelligence question? This is not about... It's a coordination failure. It's a coordination and morality and philosophical question. That's why it hit me hard. It's not about the details of AI. It's just like the concept of AI is just one of the pieces of the puzzle. Yeah. I mean, do you think that there's any possibility that he is completely... I know there's a possibility. Do you think it's... Here's my bull case that I think maybe, perhaps, Eliezer might also agree with for how we still exist.
Okay. We make the AI that he thinks that we're going to make, and the AI just does not give a fuck about us. Well, I tried to propose that. You ants can just have your earth, and you guys are making it marginally difficult to harvest your resources as resources elsewhere. So we're going to go elsewhere. If you guys become the hardest resources, no longer the hardest resources to cultivate, we'll come back. But right now, we'll just go grab Mars. So it just like blasts off into the Milky Way somewhere? It's just like distracting. Yeah, it just goes elsewhere. We're just like, see you later. And we get like a few more generations to live before they come back and then eat us.
Or do they, yeah, maybe they don't need us. Maybe they don't care. Maybe they have other things to do. I mean there are these possible outcomes that's maybe part of the follow up Q&A it still didn't totally make sense to me that the AIs would be like auto evil we want to mail everyone a bacteria that's going to destroy every single human being and rearrange their atoms maybe the default is ignore if you're not getting in my way if you're not going to shut me down, if you can't shut me down, then see you later. I'm going to blast off and like go explore the rest of the galaxy. I don't know.
So here's my question for the hopefully incoming Q and a session with Eliezer. So we have to train our AI models, right? We have to, we have to train them on data. What data do we have? The internet, where did all the data come from on the internet? It came from humans. So don't we actually imbue our culture and who we are as humans into AIs that way? And like, even though it's not technically part of the code as to how to learn values and morals, won't they just like absorb it just because that's where their data is coming from? Yeah. I mean, that, that is people's like sort of the argument of like, why can't it be a gentle parent to us?
Why does it have to be like, why can't it kind of be some sort of like a father figure for humanity and be like, It will literally have our DNA in it. Well, not literally. Well, no, but like it could have our memetics in it. It could. Yeah, right. I mean, we could ask that question again. I feel like we proposed that and he answered it. Maybe he wasn't in the headspace to kind of like answer it in more detail or maybe the question has to be constructed in a different way. Or maybe I just like pepper. I think there were times where I overwhelmed him with questions. I said this in the intro. and um just like it's just a style well he choose your own question your style is like here's a bunch of words it's it's all collectively a vibe respond to the vibe which is actually good podcasting but that's how i've learned to ask questions and many many people uh work with that style of things we're like you overload them with questions but they get the vibe and they already know what they want to say or they just choose which one to answer exactly yeah they will most guests just want like uh i think i've been on the guest seat more than you have but like you most guests just you just pick the answer that you want to give you already you know what you want to say anyways it doesn't matter what the question is yeah he is not like that he's very process oriented and logical and he's like you sent me three uh queries you've made you and i need to return three direct questions and like it's too many queries at the same time it's like Don't DDoS me, bro.
Exactly, yeah. But yeah, that's kind of a style thing aside. But yeah, as far as the substance, I don't know. I feel like this list of people he mentioned, Paul, Cristiano, Aja, Kultra, Kelsey Piper, Robin Hanson. I've heard of Robin Hanson. I don't know very much. Robin Hanson, he wrote this book, Elephant in the Brain, which is one of my all-time favorite books. Yeah, it was a book at East Denver, 2018. Do you think that this could be like just somebody listening to this is like, oh, cute, the crypto guys are like interviewing an AI person. They're all scared. Isn't that cute? It's their first time.
And they're like really good answers for why we won't all be destroyed by artificial general intelligence. No, because crypto, sorry. Thanks. Crypto, like we're pretty, we're futurists in crypto. Like we don't know, it's not like we know nothing about AI. I thought we knew a little bit. Like we knew more than. I think a lot of people will listen to this podcast and be like, that is complete BS. I'm talking about not crypto people. I'm talking about normies. They'll listen to this and be like, what is he talking about? I'm not afraid of Siri. See you later. What a crackpot. But crypto people, we're totally into this.
We understand. I was trying to think about that while making the agenda for the podcast. like okay we've never done intentional AI content on the podcast but I'm not going to assume that the average bankless listener doesn't know about the alignment problem I'm going to guess that at least 50% of bankless listeners already knew about the alignment problem going into this podcast yeah evil AI coming to kill us and we can't teach it morality and it just gets super intelligent and then we didn't even use the paperclip analogy the idea that you construct some general intelligence to create a paperclip factory.
And what it ends up doing is, as a byproduct, turn every atom in the reachable universe into a paperclip, including all of the rest of humanity. And this is like an analogy, and I think a device actually created by Eliezer, which is like another fun fact. This guy has been thinking about this stuff for a while. Yeah, this guy, he's inside of a lot of conversations that relates to AI, for sure. It's heavy, man. It's heavy. are you good I just like are you processing I'm good I'm fine yeah I I'm yeah I'm generally like about these things yeah you you seem like I need to kind of check on you tomorrow morning dude I was like I mean like I was like um getting worked up in that episode a little bit like a little bit like yeah um wow shit like yeah it's that that was the prognosis for humanity it's fatal what Yeah, I guess I haven't thought about this stuff in a while.
Don't let me drag you down, dude. Keep your vibes up, okay? Don't let me drag the rest of the nation down. Hey, we can turn this into an AI alignment podcast if you want. I'm not smart enough to do that. I don't. Actually, no, it goes back to, I don't think it's an AI thing. It's just like a philosophy and awareness thing. Yeah. We are really good at educating. We can just include in the intro, welcome to bankless money in finance also remember to talk about the AI a problem and get everyone on board with the AI alignment problem and now into the episode we got 20 years left at best to 220 years yeah hell of my 20 I got heavy and some things to do I guess it's heavy all right well maybe he's wrong though can't can we say that okay here's one thing I'll say that's always like I've always kind of shelves like yeah there's It's the long tail of maybe the black swan works in our favor this time.
So Vitalik has given to Eliezer's institute in the past. And so we reached out and just, as we do, we ask kind of our close contacts of like, hey, are there any, you're a big brain here. We're having this big brain on. Can you tell it? Would you ask them if you were us? And Vitalik, I don't think he'd mind us saying. And he's like, well, just stay away from topics like, you know, like the centralization problems of artificial intelligence because Eliezer is well past that. He's like, he doesn't care about centralization. He wouldn't have let us go there. Yeah, he's like, we're all going to die.
He would have immediately said, who cares about centralization? Exactly, exactly. So, okay, so we didn't go there and we weren't silly enough. But like the talk's comment was Eliezer's probability of doom is probably like a .9, 90% probability of doom. Actually, after this episode, I think it's like a 99.8. It's like 98 plus, yeah. And then he said, but my probability of doom is probably a .1, so 10% odds. I like those odds. He said .1? That's what he said. So I like those odds a lot better. And Vitalik is also someone who's very smart, and I want to know why that delta exists. and now he hasn't spent his whole life on artificial intelligence obviously and AI safety so you know like maybe there's a kind of a delta it's like I wouldn't trust Eliezer's opinion on all things crypto of course but I don't know maybe there's some hope there that there are like I guess we need to get other opinions is what I'm saying before we kind of and I feel like that's one thing I'd like to do I don't want to turn this into an AI podcast but like I want to hear somebody before we do I need a second opinion I need some hope David I need somebody to come on so I think Vitalik's like the actually the best person to do this because like once again this is not an AI issue this is a philosophy thing and who else other than AI and then Vitalik well he's pretty optimistic you know in general but um that's a question i didn't we didn't have time for with eliezer i wanted to ask like were you always so pessimistic or what i i was worried that that would almost seem so too prying but i guess that retrospectively could have been a good question um i think he was fairly open i was just like i was a little i'd say worried about his mental state but it was just very like um yeah dude it was very down i was just curious about like okay since you've so uh convictedly come to this conclusion like what do you do with your day we said he's on sabbatical that was the other thing on sabbatical he's doing time i might come back into this i have some more at value dad that's why i have time for podcasts look maybe he's just burnt out with it maybe uh i don't know man that's all i got uh i i need to let this episode percolate sleep on this one and uh i apologies to everyone listening that if we accidentally give you an existential crisis We try to keep things light and upbeat and optimistic and bullish.
Ultimately we are bullish on humanity. We're the most bullish people ever. And that podcast is like, is one thing like, damn you, there's no way to be bullish about that. Right. How do we turn this? How do we flip this one? How do we spin this narrative? Not even us. There's another subject matter, which I can't remember. Maybe it was Eliezer on Sam Harris, but he talked about how knowledge is discovered, like what knowledge is and how knowledge kind of exists even without a form factor to hold it in. Does that make sense? Yeah. As in like, um, uh, and, and so like, there's a lot of knowledge out there that humans don't know.
Uh, there's a big gap between where we are versus where we will be when AI comes. Um, 10, 20 years is a lot of time. Um, yeah, let's remember because there's a lot we don't know. Eliezer is not an all knowing being either. And like he is, um, yeah, ultimately he's one smart person who's taking a close look at this and come back to sparing their, you know, other people who I assume come back more optimistic. So, um, yeah, I guess there's that. Yeah. Well, uh, that's it. Uh, I accidentally gave myself an existential crisis. We'll try to record a podcast, but, uh, I'll bounce back tomorrow for state of the nation.
We'll get some recording done. We'll be back on crypto topics and I'll just forget this ever happened. Huh? Okay. I'll get Logan to make a, make a PO app. I accidentally gave myself an accidental crisis while doing it. It's a one of one PO. Wow. We'll get that. All right. Well, um, check in on me tomorrow and, uh, let's talk about it then. Uh, banks nation. Hope you enjoyed the debrief. I guess we just keep on doing this. Cause why not? Oh, cause why not?