Main Topic: The role of memory in learning and the importance of understanding constituent material.
Key Points:
Participants:
Takeaways:
Key Quotes:
A few weeks ago, I sat beside Andy Matuschak to record how he reads a textbook.
Even though my own job is to learn things, I was shocked with how much more intense, painstaking, and effective his learning process was.
So I asked if we could record a conversation about how he learns and a bunch of other topics:
* How he identifies and interrogates his confusion (much harder than it seems, and requires an extremely effortful and slow pace)
* Why memorization is essential to understanding and decision-making
* How come some people (like Tyler Cowen) can integrate so much information without an explicit note taking or spaced repetition system.
* How LLMs and video games will change education
* How independent researchers and writers can make money
* The balance of freedom and discipline in education
* Why we produce fewer von Neumann-like prodigies nowadays
* How multi-trillion dollar companies like Apple (where he was previously responsible for bedrock iOS features) manage to coordinate millions of different considerations (from the cost of different components to the needs of users, etc) into new products designed by 10s of 1000s of people.
Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.
To see Andy’s process in action, check out the video where we record him studying a quantum physics textbook, talking aloud about his thought process, and using his memory system prototype to internalize the material.
You can check out his website and personal notes, and follow him on Twitter.
Cometeer
Visit cometeer.com/lunar for $20 off your first order on the best coffee of your life!
If you want to sponsor an episode, contact me at [email protected].
Timestamps
(00:00:52) - Skillful reading
(00:02:30) - Do people care about understanding?
(00:06:52) - Structuring effective self-teaching
(00:16:37) - Memory and forgetting
(00:33:10) - Andy’s memory practice
(00:40:07) - Intellectual stamina
(00:44:27) - New media for learning (video, games, streaming)
(00:58:51) - Schools are designed for the median student
(01:05:12) - Is learning inherently miserable?
(01:11:57) - How Andy would structure his kids’ education
(01:30:00) - The usefulness of hypertext
(01:41:22) - How computer tools enable iteration
(01:50:44) - Monetizing public work
(02:08:36) - Spaced repetition
(02:10:16) - Andy’s personal website and notes
(02:12:44) - Working at Apple
(02:19:25) - Spaced repetition 2
Dwarkesh Patel
We under-appreciate the role that memory has in our lives. If what you're trying to do is to understand something pretty difficult, your ability to understand that thing is still absolutely going to be bound on your memory of the constituent material.
Andy Matuschak
For the median student, the education system mostly wants to make the student do things they don't want to do. It's not about helping them achieve their goals more easily or more effectively, for the most part. It's about achieving goals that aren't theirs.
Dwarkesh Patel
The histories in educational psychology that I'm most aligned with are like the most robotic, authoritarian kind of histories, and also the ones that are most like unschooling and Montessori-esque. Do LLMs make memorization more or less valuable?
Andy Matuschak
LLMs depend on our ability to externalize things and to make them legible.
Dwarkesh Patel
Basically, everyone in the educational space are focused on really the bottom quartile, not even medium. Today, I have the pleasure of speaking with Andy Matuschak, who is a researcher, engineer, and designer working on tools for thought. In addition to this podcast on Andy's YouTube channel, we did an interesting collaboration, which I encourage you all to check out, where I just watched Andy try to learn some new material. It was just an intro chapter of quantum mechanics. And honestly, I was expecting to see some cool techniques or be impressed, but I was way more surprised than I expected to be by the deliberateness, the effortfulness, of the practice.
What was really… I mean, it was like 15 minutes a page in this textbook. Any small thing that Andy thought, like, "I don't fully understand this. The author is trying to say something here. He's trying to draw an analogy or relationship. I'm not sure I totally comprehend the relationship between this classical mechanics equation and the quantum mechanics equation the author thinks is analogous." Just really delving deep into that. I thought that was really interesting, that this is a way to approach new material. So in this conversation, I'm looking forward to talking with Andy about not only that experience, but a whole bunch of his other research and the other tools he's built.
Let me ask you this. That experience made me think, this is somebody who actually cares about understanding the material, going through it this deliberately. Do you think people in general care about actually integrating and understanding the material they're consuming in books and textbooks? Don't you think they'd make more of an effort to actually assimilate that information if they cared to get it?
Andy Matuschak
Yeah. I mean, I think the statement is just a little too general to comment on. I think it's certainly the case that most students don't actually want to do this because they're learning stuff that they don't actually care about learning. Or even if they do care about learning it, often there isn't a clear connection between whatever reading or activity they're doing in the moment and the thing that originally inspired them for the subject, what they actually want to do. So there's always something tenuous going on.
I think on the other hand, it's amazing to look at, say, subreddits and to look at the level of nerdery and fascination that will be brought to bear on gardening equipment or knots, for instance. People are competing to tie some very obscure 18th century knot, and they're flipping through almanacs from the period. So when people are interested and it connects to something that's truly meaningful for them, I think they really do want to absorb. And we see that in their behavior.
There's a second thing that I think is relevant. To explain this, I will reference Mortimer Adler and Van Doren's *How to Read a Book*, which is a great guide on serious reading. They consider the case of people who often have books on their bedside table, sometimes very difficult or demanding books. These are kind of aspirational, like, "Oh, I wish I could read *King Lear*. I want to be the kind of person who reads *King Lear*." You put it on your bedside table and people will read it before bed and they'll find that they fall asleep while they're reading it. They're not really absorbing or understanding this book. It's not just an issue of memory. It's that they simply are not apprehending the words on the page.
The authors of *How to Read a Book* make the case that the issue with these people who are falling asleep reading *King Lear* is, in many cases, not that they don't want to stay awake and really deal with that text. In many cases, it's that they actually don't know how. They butt their heads up against this very difficult wall of material. It's almost like a rock climber who's not very experienced going up against a wall that has only these really subtle notches. To an experienced rock climber, those subtle notches are like a ladder. They can get right in there and start making some progress and seeing what's up with this wall. But if you're an inexperienced rock climber, it just looks like a solid wall.
So the claim—maybe this is an optimistic claim, you can take me to task—is that there is such a thing as being a more skillful reader. And being a more skillful reader will actually, in practice, in many cases when the reading is aligned with your actual interests, produce a more serious, more understanding-forward kind of reading.
Dwarkesh Patel
Right. So there are two models of why people might fail to retain the material they're consuming. One is they got it at some point, but they forgot it. And the other is they never understood it in the first place, and they just never noticed that they never understood it. What I found really interesting was you going paragraph by paragraph, sentence by sentence, "Have I got this?" And by the way, this is material that I had tried to go through the week before. And there were things when you dwelled on something, I'm like, actually, I don't understand that either. And I didn't notice I didn't understand that. How are you able to notice your confusion while you were going through?
Andy Matuschak
This is, again, a habit. It's a skill that can be built. Adler and Van Doren suggest that the first and most important rule of skillful reading, active reading, is asking questions and trying to answer them. If you just dwell on that—dwell on what kinds of questions should I be asking, how should I go about asking them, how should I go about answering them when the author isn't present—then you'll get very far. They also say, conversely, and this isn't meant as a criticism, an undemanding reader asks no questions and gets no answers. I certainly have read many, many books that way, particularly before I developed this habit. I often found myself falling into that second category where the issue was not that I failed to remember things, but rather that my eyes just kind of skidded across paragraphs without even realizing. You're halfway through a chapter and you're thinking, "What is this chapter about?"
Dwarkesh Patel
Okay, so a broader question. Now that we have all these online resources, some of which you've helped develop, Khan Academy and elsewhere, it seems that the value of conscientiousness as a trait has dramatically increased. If you can motivate yourself to learn these things, the world is out there for you to absorb. What are the sort of design or UI or even content modifications that can be made to give you a conscientiousness boost? Where in the past you have a professor, you have peers, you have in-person deadlines that motivate you. Is there something equivalent to a pen and paper, how that boosts your mathematical IQ for conscientiousness?
Andy Matuschak
Right. So one enduring result in educational psychology is that when you're doing a lot of cognition, metacognition is difficult. What I mean by that is when you're thinking really hard about the stuff on the page, it's very difficult for you to plan, regulate yourself, figure out what the best next action is, reflect and evaluate whether you're understanding things—all the stuff we're talking about, about asking questions. That all gets harder as the material gets harder and as it gets less familiar.
So one common thread in learning science has been to outsource metacognition. Some of the ways we outsource that are actually very familiar. They're things like somebody gives you a syllabus and tells you what to read when, and you reference that. That is a user interface, that is a design practice. If you're a self-motivated student, one thing you can do—and that I've done—is just go appropriate a syllabus from some graduate level course that corresponds to the text that you're reading and say, "Well, that might be a good guide as to what's most important, how to approach this."
There are also lots of things that one can build directly into the interfaces. Just as one example, in Quantum Country, which was a textbook that Michael Nielsen and I developed to explore some ideas around augmented reading experiences, we embedded a bunch of review questions every 1,500 words or so in this text on quantum computation. Our primary intention in doing this was to help people remember what they read. We had this theory that part of what makes it hard to learn a complex subject is that there are all these new definitions and notation and terms being thrown at you at once, and you're being asked to combine these things which are still unfamiliar. You're constantly having to retrieve these elements and struggling to do it. Either it's taking a while or your success rate is low.
But it had this other metacognitive benefit that was really important. You read 1,500 words, and now you're being asked these questions. That is an opportunity for you to notice that you did not, in fact, absorb what was in that thing. Not that you don't remember, but that there's a word in the question that is apparently important that you simply didn't even notice. So not only does that give you feedback—it tells you, "Oh, maybe you need to go reread that specific section"—but it may also change your behavior towards future sections. In interviews, readers told us that after they reached the first set of questions or a particularly difficult set of questions, they found themselves slowing down and reading more attentively, or realizing that their reading practices were ineffective in general, kind of in the way you were mentioning towards the start of the conversation.
There's been a bunch of research on the key phrase here: "adjunct questions"—questions that go along with a text. That's kind of what I was just talking about. They have all kinds of effects. The adjunct questions have the kind of effects on forward material I was just describing, and they also have the effect of making you reflect on what you've just learned. In addition to the questions being asked, you might find yourself pondering, "Well, I'm being asked about this, but why does this matter?"
Dwarkesh Patel
On the point of adopting the syllabus from somebody else, one problem you might have as a self-learner is you have some goal—this is the reason I'm learning this—and then you start thinking, "Well, do I really need this chapter? Do I really need this content?" At that point, you're doing the metacognition that you were trying to use a syllabus to avoid. If you are trying to self-learn and there is a resource that is a close approximation of the syllabus you want, should you just say, "Hey, I don't know why I need this chapter. I'm just going to go through it"? Or should you use your own judgment there?
Andy Matuschak
This is a pretty classic issue for learning in general. You have this problem where you have to bootstrap yourself in a domain. You have to outsource the question of what is necessary to know. You might know that you really want to build a model that can generate images given descriptions, like Midjourney, but you don't even know what you need to study to do that. So you pick up some textbooks on machine learning. You're kind of outsourcing the answer to this question to the author: what is necessary to know to build things? Maybe you can find a book that's actually labeled "What you need to know to make an image generating model." But even then, you're outsourcing your answer to the author. You can take that answer as a start and treat it as tentative, and revise it iteratively. As you become more skilled, you can lean less on it. And you probably should.
I think a very common mistake that people make is to feel like they need to do the thing the right way, that is exhaustive and completionist. They fail because they find themselves bored or unmotivated because the material doesn't actually seem to relate to what they want to know, but they're just going on faith that if they follow what the author says, everything will be good. They find themselves having trouble for that reason, and then they just stop. This is bad. They would be better off just skipping around according to their interest and continuing.
One other thing I'll say about this is that the role these syllabi play is as a scaffold. This is a term from learning science. If you want to get higher up a building, you may not be able to climb it yourself, but you can build some scaffolding around it. And then suddenly you can reach the top shelf or the top of that building. Where the metaphor breaks down is that although scaffolding is ubiquitous in education—we give you simpler versions of questions first, that's a kind of scaffolding; we partially work the answer first, that's a kind of scaffolding; we give you worked examples first, where we might ask you to predict the next step of the worked example, that's also a kind of scaffolding—where the metaphor breaks down is that once you become more capable, we try to remove the scaffolding. It's called fading. The idea is that once you have solved a lot of calculus problems, you don't need half of it worked out and you're just filling in one of the blanks anymore. In fact, doing that would not be as effective a learning experience.
So the application with the syllabi might be something like: if I'm studying something in computer science, which is a domain that I know really well, I don't need those syllabi. Not in the same way for most subjects. I think that's mostly just because the amount of cognitive demand placed on me by the subject is much lower than it is for other subjects. So much of it is familiar already that I can deploy my own planning more effectively as I go. But it's also the case that because I know so many things about the subject, I can do a better job from the get-go of making a plan. Making a plan requires modeling a path or predicting a path, saying, "Well, I guess I'd need to see how this connects to that." If your destination and your starting point are very far away, then you can't necessarily see all the things in between or how to draw those lines. But if those things are only a couple hops away, you can maybe infer pretty accurately.
Dwarkesh Patel
I guess this maybe implies that if you do want to learn about a subject, it might just be helpful to do an intro to X subject course or textbook, not necessarily because it is instrumentally valuable to whatever problem you're interested in, but because it'll give you the context by which to proceed on the actual learning.
Andy Matuschak
That's true. It's also the case that you don't even know all the stuff there is. And this is another key problem. There's a fundamental tension in unschooling, for instance. Just let the kids pursue what they're interested in. That's cool. There's a lot of good things about that. But also, say that a kid's true passion turns out to be ocean geology or something, and they're in a landlocked country and there's just no one around them that talks about ocean geology. Then they're missing out on some great opportunity. But if the school had a program where they are bringing in guest speakers, and then there's a special lecture on ocean geology from this person and it lights up the kid's world, even if they wouldn't have chosen that lecture, that's a good thing.
Dwarkesh Patel
Unschooling is actually an interesting subject to talk to you about. We'll get back to that. But before that, I want to ask you about this excerpt from a Paul Graham blog post. The title of the post says, "Reading and experience train your model of the world. And even if you forget the experience or what you read, its effect on your model of the world persists. Your mind is like a compiled program. You've lost the source of it. It works, but you don't know why." So it's a compiled program. You don't need the source code. Is it okay that we're forgetting so much of what we're reading?
Andy Matuschak
Yeah. I mean, what he's saying is true to some extent. Whether or not that extent is sufficient is going to depend a great deal on the situation and on what you need. If your aspiration actually depends on having a deep, detailed understanding of the material, then the kind of imprint on your worldview or on your automatic responses made by the book may not be sufficient. On the other hand, if what you want is to absorb a lot of different ways of looking at the world, knowing the details of these isn't necessarily important. Maybe you just want to know that Confucius emphasizes community and society as a moral patient, in contrast to the individualism of a bunch of humanist philosophers, and that's the level you feel you need to make decisions in that domain. I think that's fine.
Very practically speaking, it's funny that he uses the word "compile" because one of the prominent theories of cognition—how we come to know and learn things—is this theory called ACT-R by John Anderson. A key part of it is a process that he calls "knowledge compilation." This is the process by which we take individual facts and turn them into higher-level patterns that we can generalize and apply in more contexts. I think that's what Paul is gesturing at. You read a book, it contains a story, a case study. By reading it, you learn to generalize to some extent, and you apply it in other contexts when it seems relevant.
The reason I bring up Anderson's theory is that he has a bunch of specific claims about what's necessary for knowledge compilation to happen, and what you'll be able to do as a consequence of certain degrees of knowledge compilation. I think he'd probably respond to this by saying something like: actually, in order to effectively compile things that you've learned into schemas that will match future scenarios effectively, you need to be exposed repeatedly to those things. You need to use them. You need to do a variety of things that will basically show your brain that it is relevant to apply these things in combination. Simply reading probably won't do that. But if you read and you have a lot of conversations and you're in a context where it's demanding and drawing on what you read, then you may naturally do that kind of compilation step.
Dwarkesh Patel
I've actually been thinking about this in preparation for talking with you. I look back on some of my old conversations. I've had the pleasure to talk to a lot of interesting people across a lot of different fields. At the time I interviewed them and had done all the prep, I actually had a lot more context than I can remember now. Sometimes I'll listen back to a conversation and I won't even remember the content in the conversation. I remember thinking after the conversation, "I knew so much more about this field and it was compressed into this one hour interview." I had to prep other things that might come up. And afterwards, I'm like, "I don't even remember the things that were in this one hour." But then the other part of me thinks, "Well, I'm getting better at doing the podcast." That might imply that I've picked up something. But it is a shame that I didn't have some sort of rigorous practice throughout the time of retaining the material that I was keeping.
Andy Matuschak
Well, yeah. I mean, I expect the main way in which you're getting better is actually not really about any of the details of those materials. I think it's about your practices as an interviewer, the way that you generate questions. You probably have a bunch of patterns, whether you know it or not. You read a thing that a person has written in hopes of generating good questions about it. And even though you don't have this habit for textbooks yet, of constantly demanding things of the textbook, you have, I think, started to develop this for essays or blog posts that interesting people you're interviewing have read. To point to this Anderson theory, in the course of repeatedly doing that, you've made parts of it automatic, so that you don't need to do it consciously. You can focus more on the material. You can probably take on more difficult material or actually understand material at a higher level than you could have before because less of yourself is engaged in this question of "How do I make the questions from the material?"
Dwarkesh Patel
I certainly hope so. Otherwise, there's a question to be asked of what I've been doing all these years. So, having interviewed some of these people who are infobores and have consumed and continuously consume a lot of content—this is something you also noticed and pointed out in your notes—Tyler Cowen, for example, I don't think he has any sort of note-taking practice. He just devours information. What is your theory of how these people are integrating things?
Andy Matuschak
Tyler's a good example. I think he's actually a little easier than some others we might discuss. So let's talk about Tyler for a second. One of the other things that's so interesting about Tyler is his writing obligations. This is a man who's blogged every day since, I don't know, 2007 or something, and has a Bloomberg column, I think weekly, something like 1,500 words, and also has published something like a book a year for a decade or more, and occasionally publishes some academic articles, plus a bunch of other collateral. That is notes. I think it's also important to note that the way Tyler writes these blog posts and the way Tyler does these columns and even the books is very different from the way that many other book authors work. Tyler's blog posts often have this real first-draft mentality to them. He's just thinking out loud.
Dwarkesh Patel
Right, right.
Andy Matuschak
And he's got decades of practice thinking out loud and writing down a decent take the first time. So he gets something pretty good the first time, much of the time. And that works for him. So that kind of is a note, right? Doing the thing—your initial thoughts on the subject—is kind of what you would write.
Dwarkesh Patel
One of my former guests, Scott Young, was comparing Bryan Caplan to Tyler Cowen's books. He said, "You read a Bryan Caplan book and it's like a chess game. The opponent is—if you try to move a pawn up on this case for education, I've got this rook that I can move here. With Tyler, it's more like he's shooting the shit out of the subject, basically."
Andy Matuschak
Pellets from a blunderbuss. Yeah, right, right.
Dwarkesh Patel
On a separate question, do LLMs make memorization more or less valuable? There's a case you can make for both. Is it more important to have more Anki cards in your deck? Maybe this is a good time to talk about what memorization is or what it's for.
Andy Matuschak
We could use that word to refer to the practice of learning more trivia, for instance. A thing that I and some people I know have done is gone through this book, *Cell Biology by the Numbers*, which says all of these things like, "How big exactly is a nucleotide? How much volume does it take up?" It's kind of helpful occasionally to know that it's about a nanoliter and that that can help you model things. You can just commit all of those things to memory. That's one kind of memorization. And we could talk about how LLMs affect that.
But I just want to make the case that so much of what you do and experience day to day is memory bound, or is memory influenced in important ways. Just for instance, your ability to understand a difficult argument, even in the course of a text, is memory bound. Some of that's working memory, but your ability to understand an argument that has many steps in it—more steps than you can keep in your working memory—depends on your ability to think of some of those steps in terms of stuff that you already know, so that you can reduce it or abstract it.
Likewise, in creative work, there's a bunch of studies trying to catalog case studies of how people have flashes of insight. One of the things that's a pretty consistent source for insight is noticing a surprising connection or a surprising contradiction. It probably feels pretty familiar. You're reading through the newspaper and you see, "People have finally figured out how to do X." And you're like, "Wait a minute, that means if I combine it with this other thing, we'd be able to do Y." That's only possible if the other thing is in your memory. If you have to think to look up the other thing, then the newspaper wouldn't seem so salient to you.
Likewise, in just really boring ways, early on in my time at Khan Academy, I learned in a very thorough way using memory systems a whole lot of details about the education market. This let me be in high-level executive conversations where we're trying to figure out strategy stuff, and somebody would propose a particular direction or "What about this? What about that?" I could say, "Well, the total budget spending for instructional materials is this, and that market is growing by this percent per year, and 10% of students in the US are in this place," and so on and so forth. So basically on the fly, I can evaluate ideas in a way that others can't. So anyway, this and other things are kind of just part of my rant about people. I think in general, we underappreciate the role that memory has in our lives.
So just to come back to the question: explicit memorization, or explicit making sure that you can recall the thing reliably, we can test it against these things. For the case of the creative instinct, noticing the contradiction, noticing the connection, I imagine that we will have future notebooks that will do some of this noticing with us and that will decrease our need to be able to rely on our own sense of salience. But I guess I don't know how much. I'm skeptical. My own experience coming up with weird ideas that feel very new is that it feels very personal. It feels very inchoate. I often haven't been able to describe textually the constituents of the thing very clearly. There's just a feeling that something in this general direction is connected with something in that general direction, or there's a tension. So that makes me a little hesitant. LLMs depend on our ability to externalize things and to make them legible.
Back to the learning point about the role of memory: if what you're trying to do is to understand something pretty difficult, your ability to understand that thing is still absolutely going to be bound on your memory of the constituent material.
Dwarkesh Patel
Do you think there's value in—there's pedagogical value in forgetting? I guess some anecdotal or unrelated evidence is in neural networks, sometimes you can improve performance by pruning some of the weights. Obviously, we forget things, right?
So clearly, we don't remember everything. When we sleep, we lose a lot of our memories. Is it possible that by forgetting the details and only getting the gist, that actually helps us better generalize the insights we're getting from text and things like that? What do you think of that way of thinking?
Andy Matuschak
Yeah, it could be. So memory is very connected to attention, and we can't attend to everything, right? So one of the roles of memory is to help guide us to the things that are important. Maybe I happen to know the magnitude and energy of an electron volt—that's something I can draw on because of the memory system stuff. But I also don't want that to be front and center in my mind all the time. I don't want it to be hyper-salient the way that some very important design principle is to me. So yeah, there's some role there.
There's also some theories that the reason we have forgetting is that our environment—our ancestral environment—was very traumatic, and so our episodic memory in particular, we would like to maybe not be all that faithful. I actually don't know the status of those theories.
Dwarkesh Patel
Probably why we forget dreams as well, right? Like dreams are pretty traumatic if you thought of them at the same level as a real life experience.
Andy Matuschak
Yeah. So, I mean, another weird thing about memory is that as far as we can determine, memories aren't lost exactly, at least not completely. There's a series of interesting experiments that people have used to demonstrate that decades later, things are still there. If you can cue them right, people can bring things back, even things that they feel are lost. And of course, you can also cue people in ways that are hallucinatory, so we need to be careful about that.
But I guess the reason why I bring that up is that it flies in the face of this view that there's a limit. One of the things that I think is kind of weird about this memory system stuff, or memory championships, is people ask, "Oh, if you do these things, will you start to forget other normal human stuff?" And what's weird is, no. I've been doing this memory system stuff for years, and I just know more stuff now. This is aligned with the experimental literature, which seems to suggest that there are probably upper bounds, but we're not close to them. Some of these memory champions have memorized truly absurd orders of magnitude—maybe two orders of magnitude more things than I have practiced. Certainly people who are multilingual have really, really absurd numbers of things memorized. So there isn't like a resource management argument.
Dwarkesh Patel
Huh. Well, if there isn't, why do we forget so many things? Is there some reason the brain just... some of the things we're coming across, maybe they're just not—we were trained in the ancestral environment to find certain things salient that just don't map onto books?
Andy Matuschak
No, it's a good question. We're getting to a part of the cognitive science space that I'm less familiar with, and also that I suspect we simply know less about. But let me just riff a little bit.
One of the things that we sort of know is this idea of spreading activation. So when you go to try to look something up, or when you try to deal with a particular situation, there's something almost kind of like DNS exchanges, or routing on a network. We start from some point that's a stimulus, and just speaking very informally, we kind of expand outwards from there. There are effectively weights on those connections. By tuning those weights effectively, we route the packets on the network. Memory is encoded in these weights, at least partially.
So if you didn't forget things, then you might just have this weird cacophony on the network. In particular, what's salient, what to do next, which response seems most appropriate to this question—you might answer those kinds of things very ineffectively because all this stuff is coming up for you that is much less relevant.
One of the theories about how well we remember stuff and in what circumstances is actually called predictive utility theory. It suggests that the probability of retrieval of a particular item in a given situation actually corresponds with a model of to what extent the brain predicts it will be useful.
Dwarkesh Patel
Right. And then the prediction doesn't necessarily map onto...
Andy Matuschak
Doesn't necessarily, exactly. So when you repeatedly access something, when you practice retrieving it, the prediction of the utility of the thing goes up. And when you do it in a variety of situations, it goes up across a broader distribution.
Dwarkesh Patel
Okay, so this is interesting. When did you start your memory practice? Presumably it was after Apple?
Andy Matuschak
Yeah.
Dwarkesh Patel
Okay, so let me ask you this. So at Apple, you were in charge of a bunch of important flagship features on iOS and I'm guessing other things. Presumably, you didn't have some sort of practice, but since you were encountering these things day to day, that natural frequency and way in which problems came up—did you have a worse understanding of those problems and the things, now knowing what you do and knowing that having the practices you do, you're able to comprehend now? I don't know if that question made sense.
Andy Matuschak
No, that's a great question. So, I mean, here's a fun thing. I was much better at what I was doing then than I am at what I'm doing now. That's pretty funny. So it was just totally different.
Let's talk about this a little bit. This feels very, very juicy for me. Most of what I was doing was engineering, some of it very difficult engineering, but mostly engineering on things that were fairly well understood. So I wasn't trying to decide what should be done usually. Sometimes I was from a technical perspective, but certainly rarely from a product perspective was that a relevant question for me. I was kind of a somewhat design-minded engineer, and I did a bunch of engineering and design-ish things on tasks which were set out for me.
At that point, I had been programming for a really long time—I don't know, 13 years maybe by the time I joined Apple, longer—and programming in Apple's ecosystem for probably two-thirds of that time, three-quarters of that time. So everything was just really familiar. It was mostly flow all the time, every day. I was just in it. I knew the stuff that I needed to know. I was very well practiced, and the space didn't change that much. Most engineers at Apple, most of the time, are not pushing the frontier of what is known, trying to discover. They're doing very difficult technical work, mostly applying things that they already know and understand quite well to problems which are usually moderately well and pretty well understood.
Memory was essential to me doing that job well, but I had already built most of it by the time I got there. I already built just tons of stuff for the Apple platform. I learned a ton of stuff about the internals of those systems. But because I already had such a rich understanding both of Apple's platforms and of computer science and engineering in general, I had this really rich network for stuff to slot into. So learning stuff is easier when you have other stuff to connect it to. It's a nice principle. Metacognitive load on me was lighter because others were figuring out what we should be doing.
By contrast, now I'm doing research. I'm trying to discover things that are not known. I'm trying to make things that didn't exist. The hard questions that I answer are mostly, "What should be done?" or "What should I do?" And that question is not just a technical one of how should I implement this feature that needs to get built, but what intervention on a reader should be taken? That requires synthesizing lots of different unfamiliar literatures.
Dwarkesh Patel
There's two different threads I want to go on. Maybe I'll just mention the other one. This is also related to the thing we were talking about a few minutes ago with LLMs. And there's this thing that I'm sure you've talked about yourself as well—the Swanson linking, where this guy was just somebody who read the medical literature. He was just familiar with a lot of esoteric results. Different things would come up and he would be able to figure out what different things are connected. For example, I think you noticed in one case that headaches are linked to some other symptom, and that other symptom is linked to magnesium deficiency. And so apparently a whole bunch of people's headaches were solved once they were given a magnesium supplement. You notice that connection. Again, this is the kind of combinatorial thing that you wouldn't notice otherwise.
But on this subject itself. So listen, there's this natural way in which you're able to get up to speed in all the things that are happening at Apple. And is it possible, and maybe advantageous, to do similar kinds of things in other fields? For example, instead of doing an explicit spaced repetition system when you're trying to absorb material from books, you just read a cluster of books and hopefully things will just come up that are relevant again and again. Or is there value in having explicit practice of setting up cards and so on?
Andy Matuschak
Yeah, right. So again, the answer is going to be "it depends." I think that the maybe the most familiar example of what you're talking about is immersion learning a new language. Immersion learning is a great thing. It's going to be more interesting and more effective than doing spaced repetition practice. It's going to be integrative, it's going to be social—there's a bunch of stuff about social learning that's relevant.
A problem, though, is that say you decide you want to learn Swahili today, and you go down to the local Swahili community center and you're like, "Cool, I'm going to immerse myself." Good luck. You can't even get started. So through this lens, explicit practice is a way to bootstrap yourself.
Likewise, a great way to become a better musician. All of the best pianists at sight reading that I knew in university played with churches. They were so good at sight reading because they had to show up every Sunday and play a different thing—a new hymn every Sunday. So this is immersion also. Over time, they're learning all these cadences and these things that are really common. But you can't show up and be the church pianist every Sunday in the first place if you don't already have some decent foundation. So this is a bootstrapping argument: one role for explicit practice of this kind is to get yourself into a position where you can more naturalistically reinforce.
Dwarkesh Patel
I see. Okay, got it.
Andy Matuschak
But there are still going to be instances where the naturalistic reinforcement isn't going to work. So for example, the linking that you brought up—one issue for doctors is rare diagnoses. If it's only going to be once every couple of years that you see a patient that's going to present with these symptoms, that's not going to be frequent enough to naturally reinforce your memory of that. You're going to need some out-of-band mechanism. And unfortunately, I think for many kind of creative leaps and creative insights, that may be closer to the regime that we're in.
Dwarkesh Patel
Yeah, that makes a lot of sense. Where in many fields, you just have the things you're regularly doing is the thing you need to reinforce. It makes a lot of sense that if you're a researcher, the long tail of events that might come up is the thing you—it might happen once every few months, but the regularity is not a thing that matters. It's sort of the effect on your work.
Here's a question I actually have. So when we were doing the practice, or when we were doing the quantum mechanics textbook, it was like three hours, and afterwards I was just exhausted. And I was actually surprised that you went the entire three hours without interruption. So afterwards I was packing up and you were like, "Hey, I'm about to actually go to my piano lesson." I was so confused at how you had the stamina to keep going. Is the stamina just inherent in you, or is that something you did to develop?
Andy Matuschak
So one of the things that I think is funny about stamina is, first off, there's some kind of weird grass-is-always-greener situation where I often feel struck by other people's stamina and feel like I have very little of it. I struggle with energy. I've actually written extensively about all my struggles with energy and ways of managing energy. I spent a lot of time thinking about it and managing my energy levels, structuring my day around it. So I think there is something where one often feels lower stamina than one actually is because one misapprehends others' stamina.
Okay, so in that particular situation, how do I explain why three hours of studying? First off, social. If I were alone and studying that book for three hours, and I weren't effectively trying to perform for you, it wouldn't have been nearly as energizing for me. I definitely would have taken breaks. I still would have been able to go for three hours, I think.
And part of the reason for that is that it's simply way less hard than things I normally do in some sense. Learning quantum mechanics should be much harder, and it kind of is cognitively demanding in a lot of ways. It's much more cognitively demanding in a direct way than what I actually do day to day. But it's much less demanding on what William James calls "the energies of men"—something like a life force that permits you to act according to your will. Maybe it's gumption, maybe it's willpower, maybe some people call it spoons. I don't know, these aren't all the same thing exactly.
But sitting and staring at a page and deciding what you should do next on a research project is incredibly draining on that resource. The not knowing—sitting and not knowing—is the hardest thing that I do in my work. So there's something wonderful about being presented with, "Oh, great, somebody else is going to tell me what to do. This is great."
Dwarkesh Patel
So although it might be less demanding than your usual work, it is definitely more demanding than the way in which I or most people approach textbooks or other material. In the sense that I would just like read through, and then once I get to the exercise, I'm like, "Let's see what I didn't understand." Whereas just the attention and the intensity to go through sentence by sentence and constantly be paying attention seems like way more exhausting than...
Andy Matuschak
Yeah, I mean, so this is sort of true. It's definitely the case that I will occasionally do some of this before-bed reading where I think, "Oh, let me just do a little bit more," and it's basically useless.
But I want to make the case that there is a kind of pocket that you can fall into. Maybe you call it flow, I don't know. Where the demandingness that you're bringing to bear is matched to your ability. The book is not overwhelming. You feel like you can make your way through it. And this is actually more engaging.
I occasionally will find myself reading as an undemanding reader and finding my attention kind of slipping because I feel like I'm just not that attached to the text emotionally. I'm kind of reading dutifully, trying to get through it. That produces sometimes an adversarial aspect where the text is in my way, or it's kind of something to be accomplished. Often then I will find that I need to bring more gumption to bear to kind of power through and make myself sit there and keep flipping the pages. But if I actually just open my curiosity and open my attention and really start engaging with the book...
Dwarkesh Patel
There are sorts of ideas that people have come up with for different pedagogical tools, which are mediums that give closer connection to the reader. One is you have some sort of fiction account where a concept is introduced and reinforced, or you have a video game with characters you care about. And as far as I know, there isn't something that has really taken off using these sorts of new mediums. What do you think that is? Is it just an inherent limitation of everything but text and lectures, or people just haven't given it the right content and design?
Andy Matuschak
Yeah, I'm fascinated by this question. Let's see, I can say a few things about it.
One is that I would argue that one medium has taken off in an absolutely enormous way, and that's video. People love video. People will watch Grant Sanderson spend an hour going through some explanation of an esoteric math problem—people who would never crack a Springer graduate textbook in mathematics. The issue is that they will, in general, not walk away from that interaction with much understanding. But they're much more engaged. So that's cool. That's suggestive. And it suggests the question: is there a version of that that actually produces detailed understanding?
Maybe one approach to producing that might be a game. My favorite example of this is *The Witness* by Jonathan Blow. Have you played *The Witness*?
Dwarkesh Patel
Okay.
Andy Matuschak
I think *The Witness* is an absolutely extraordinary work of art. It's a game that has no text—at least no text that's relevant to the game elements. In kind of classic *Myst* style, you wake up on an island and figure out what's going on. The game proceeds to explain to you, without using words, but just by shaping your environment, a series of extremely complex mechanics of a system that exists in this world. So you learn a bunch of stuff. It gets to the point where it feels like you're in conversation with the game's designers. It's like, "Ah, he's asking me to do this here." No one's asking you, right? There's no text, but you can feel that you are being asked, and you perform some interaction in the environment, and you feel that you have answered, and the game responds in kind. This is very, very interesting. It's a medium of action.
Some people have tried to make educational games, games which are explicitly about arithmetic or something. Jonathan Blow's game is not about that. The mechanics that you learn are about the environment. I don't think anybody has yet really succeeded in doing this about explicit subjects. There are things like *Kerbal Space Program*—maybe people learn some things about project management or orbital mechanics from that. Zachtronics has a bunch of games that are sort of about assembly language, roughly speaking. Maybe you can learn some things about that.
The issue seems to be that games are ultimately an aesthetic form. The purpose of the game is to have an experience that feels a particular way. So they're serving a different purpose than Grant's videos or a text. Grant's videos are also serving a different purpose from the text. The text you might pick up because you're like, "I want to be able to build a robot," so you pick up a textbook on robotics. Is there something that you could pick up that's sort of like a game—an active environment—that you'd use in a similar situation to "I want to learn to build a robot"? Maybe, kind of. We don't quite have those yet.
We have some things that are kind of like that. I don't know if you've seen *From Nand to Tetris*.
Dwarkesh Patel
Oh, yeah, yeah, yeah.
Andy Matuschak
This is a very interesting project that's kind of along these lines. What characterizes games is doing—it's active. So when I was asking all those questions of the book, that was active learning, active reading. *Nand to Tetris* is naturally active. It's a course in which you start with basically nothing—you start with memory—and you build a virtual computer, you build Tetris, you build a processor. The whole thing's active the whole time. You're making the computer grow. This is doing a similar job to the question-asking that I was doing, except that you don't have to regulate all of that yourself. The regulation, the choice of what activity to do, is in the structure of the material.
I think there is waiting to be created some kind of mass medium that is like that, but that can be applied in many, many circumstances. We have the non-mass medium version of it already, and it's apprenticeship. If you want to be a good yoga teacher, you go hang out in yoga studios. If you want to be a good surfer, you go to the beach when the other surfers are there, and you participate peripherally, and you talk to them, and you learn about their tactics. They might give you some feedback eventually, and you'll start to participate less and less peripherally over time, and eventually you'll be part of the community. This isn't a mass medium. We can't print a billion copies of it like we can with a book.
Dwarkesh Patel
What is the experience of watching George Hotz on a stream, a quote of "tiny grad"? How does that compare to just being in an office with him? Because even if you're in an office with him, there would be constraints on his time and how much engagement there would be. So why isn't video a scalable way to increase the friendship?
Andy Matuschak
I'm incredibly excited about streaming actually as a medium for this. We're kind of gesturing at a particular kind of learning that needs to happen. It's often called tacit knowledge. One of the things that you have to learn to do as an engineer is to learn to deal with a hundred thousand different weird situations where something is not behaving the right way. Eventually you learn a kind of pattern recognition, ways of dealing with this. Much of this is not described in any book. It's not explicitly taught. You just kind of learn it by doing it over a long period of time.
So by watching George do it, I think that people do absorb stuff. They can absorb some of that knowledge. That's part of how apprentices absorb that knowledge. There are a few things that are missing. You're not getting feedback. There's a whole lot of chat there, a whole lot of stuff that probably isn't all that meaningful. But it's also true for apprentices.
I'm pretty excited about streaming videos. I have complained loudly that there aren't more designer streamers. One of the things that I think is really interesting is that we have some disciplines like programming where there are a million books and courses about how to learn to program. They don't give you everything you need—there's this tacit knowledge stuff that you need to develop. But if you work through these courses, if you go through the MIT OpenCourseWare for computer science, you'll be able to build some stuff, and you'll be able to lift yourself up.
This is not true in all domains. In particular, design, but lots of other domains that are sort of like that—musical composition, architecture, something like this. It's normally done in studio classes. Lots and lots of hands-on feedback. The feedback is highly contingent, highly contextual. We just haven't figured out how to communicate this. So it's good to see lots of programmer streamers, but I really want to see streamers in these other domains.
Dwarkesh Patel
Right. On the point about the more programming books—ironically, the reason why there are more resources on programming is that it's just so legible, but it already makes it easier to understand in the first place. So you just have this reinforcement. But *Nand to Tetris* is like a video game analog to learning maybe not just programming but how things in the internals of a computer work. But programming has an element where it already feels like a video game. I have a friend who has a sort of intense manic energy, and he used to be addicted to video games when he was a teenager, and now he just stays up all night in these coding binges. It's just the same part of the brain.
Are you optimistic about things like video games and fiction being able to work in fields that are not already kind of like a video game, like programming?
Andy Matuschak
I think what makes programming feel like a video game is this sense of instantaneousness, this sense of direct contact with the environment. You're learning about a conceptual world, but that world is right underneath your hands. As you manipulate it, you're constantly getting this feedback—the red squiggly, you're pressing Command+R regularly and seeing it fail. And that feels great.
There's this feeling that's very common for programmers, and it's laden with doom. The feeling is: it's 9 PM and you've been working on a thing all day and it's almost working. It's almost working. And you know, if you just debug this one thing, then your project will be done and you'll be able to go to sleep. So you're like, "Well, I'll just stay up and debug this one last thing." And then you start debugging it and you get it and you solve it. And that feels great. And then immediately you run into one more thing. "Oh, it's almost running all the way through. It's almost going end to end." And you're like, "Well, I'll just stay up a little bit longer." And before you know it, it's 2 AM and you keep going because it feels so good. You feel the sense of forward progress. You're not just staring at a wall.
For the programming problems where you are at a brick wall, it doesn't feel like this. It feels bad.
Can every field be transformed into something where you can feel the sense of forward progress, where you can get this rapid feedback cycle? I think that's really hard. I think some fields, it's not clear to me that they can be transformed in that way. I think we can get a lot closer in most cases than we're at right now.
So what's hard about designing a user interface is that often there's this feeling of exploring a combinatorial search space. Programming often feels like a search problem too. You have a sense that there's some right way to solve the problem, maybe some set of right ways, and you're kind of looking for it. You have some heuristics that guide you: "Oh, this might be a dynamic programming problem," or "This might be something that is solved well by separating concerns."
Design often feels less like that. You have those heuristics too, you have those patterns too. Often it just feels like, "Nope, I just need to try 300 things." There's this characteristic where you'll look at a designer's Figma file. What you do in Figma is you press Command+D. This is the core action of Figma—to duplicate. So you have an artboard, you tried something, and you're like, "That didn't work." So you select it and you press Command+D. And what you end up with—you look at design Twitter and it's just all these screenshots of people's Figmas with a million artboards. They're just trying stuff.
You don't have this feeling—or at least I don't, and I think many designers don't—of progress. It's like you're just kind of exploring parts of the search space, and you're learning that parts of the search space don't work. Eventually you stumble on one that does, but you don't have this feeling of getting closer often. Often there will be weeks that go by without feeling like you're getting closer because what you're doing is just narrowing the search space.
Dwarkesh Patel
Interesting. Although there are people who are obsessed with design in there. And what is the sort of loop that keeps them obsessed with a process that doesn't feel intrinsically forward-feeding?
Andy Matuschak
So to some extent, I think they are skillful. The people that I know who are like this—it's a combination of they're skillful often, and the nature of the problems that they're solving are highly tractable. A kind of example of a thing that designers will often rabbit hole into is designing a poster. It actually used to be kind of a cliche that at Facebook, there were all these posters up on the wall of the office—very, very elaborate, beautifully designed posters for a talk that someone was coming to give at Facebook. And it's like, why did somebody put all this effort into it?
Like, well, it feels really good because a poster is this really constrained, finite, ephemeral thing. You can start it and within a few hours, yeah, there's a search space, but you can find a decent part of it pretty rapidly. Once you're there, there's this beautiful and very enticing feeling of turning the crank and making it better, polishing it, trying this or that. But when you're trying this or that, all of the options are kind of okay, and you're trying them out of curiosity, like maybe it can be even better. That's very different from the kind of design where you just simply don't know how to do it.
I think it's part of why those designers loved making those posters. It's a snack, it's a treat. It's also something they get to control, whereas ordinarily they don't.
Dwarkesh Patel
Just don't tell the manager how many software engineering hours were used up in the poster designing at Facebook.
Andy Matuschak
No software engineering. It's only designers.
Dwarkesh Patel
Got it. But for the software engineers, code golf is the equivalent, right? What is code golf? In golf, you try to get the lowest score, so code golf, you try to solve the problem as minimally as possible. Like, "Ah, I don't need this. I can combine this. I can do it in three lines. If I use Haskell, I can do it in one line." That's a kind of thing programmers do. But just endless refactoring is another thing that's kind of like this. You have the thing working, but it could be more beautiful.
So it seems like the tools and ideas you're developing are especially geared towards very intelligent and very motivated students. If they would be different, how would the tools you would develop for a median student in the education system look, both in motivation and in other ways?
Andy Matuschak
Yeah, they'd be super different. I kind of got out of the educational space in part because I don't like the framing of this problem. For the median student, the education system mostly wants to make the student do things they don't want to do. It's not about helping them achieve their goals more easily or effectively. For the most part, it's about achieving goals that aren't theirs. Obviously, that's not always true, but for the median student, I think it kind of is.
When I was at Khan Academy, I was thinking about this problem. One of the angles I found really interesting was focusing on inquiry learning and transforming the learning experience into something that actually is related to their goals. We're asking questions that are authentically interesting, that they authentically want to answer, and that they can participate in in a way that feels natural.
We did a lot of experiments with dynamic media representations of things. The idea being that you've probably seen those plastic blocks or things that kids can play with to get an idea of numbers and number systems. Kids will play with these things unprompted because they're fun. It's a pleasure to handle them, a pleasure to manipulate them. When you have them in hand, it's very natural to suggest, "Can you make a pattern like this? Why can't you seem to make patterns like that? Why is that?"
For instance, Cuisenaire rods are a set of 10 rods that have unit length one to ten, and they're all different colors. You can take the rod that represents eight and put two of the rods that represent four up next to it and show that you can divide it into two rods effectively. But then if you take seven, there is no other pair of rods of the same color you can put next to it, so you get these different patterns. Things naturally suggest themselves by experimenting with these materials and having conversations with people around them.
One of the things we were interested in was, are there things that are like that for more advanced topics? Can we create something that's kind of like those rods, but about a more advanced topic in math or about debates in history? One of our tactics was to lean heavily on social interaction. People like talking about stuff with people if it's a real conversation. For the same reason I had to use less willpower to study that quantum mechanics text because you were there with me, a student who's engaged in a real activity with a peer will need less willpower as well. They'll also learn from their peer if you structure things right. So social learning becomes interesting.
But at a high level, I've mostly abandoned this question to others. Basically everyone in the educational space—this isn't totally true, but 90-plus percent—are focused on the bottom quartile, not even the median. There's a good reason for this. Many people in education are motivated by arguments of equity and opportunity. They want everybody to have the opportunities they had. They're motivated by the injustices they see in differing access and support. They're motivated by the very real disadvantages that accrue to the bottom quartile performing students.
It's also true that the marginal impact you'll have on that student's life will probably be much greater than the marginal impact on an 80th percentile performing student. The argument goes that that student will be fine, which is probably true, but there's a big marginal difference between fine and supercharged.
Dwarkesh Patel
Yeah, I mean, that's true.
Andy Matuschak
I say all this to say I understand why the vast majority of people in education are focused on what they're focused on. I think it is good, and I'm glad they're doing it. I've mostly decided to let them do that and focus elsewhere.
Dwarkesh Patel
No, I see tremendous value in focusing on the realistically cool new shit that's coming out. Where's that coming from, and what's the way to increase that? But it's interesting to know that the same tools might not just work across the spectrum.
Andy Matuschak
I can also say part of the trouble here is that the cool shit is very likely to come from students who are performing at the 20th percentile in school because they're disaffected and bored and none of this stuff matters to them. So part of the trouble here is that by opting out of helping these people learn, there are all kinds of interesting inventions that could probably occur that aren't occurring. I don't quite know how to contend with that. I'm trying to bite off a piece of a problem that feels maybe tractable.
Dwarkesh Patel
Once all the tools are built, at the end of your career with the learning process, is it supposed to feel fun? Or does it have to feel fun? Is there an element where even when all the tools are there, there's just a level of David Goggins—this is going to be miserable, but I've decided to learn this in this way and I just have to go through it? Where does misery come from?
Andy Matuschak
Hmm. Where does it come from?
Dwarkesh Patel
I'm asking this honestly, not really rhetorically. Let me try to answer my own question.
Andy Matuschak
Let me say first off, I am broadly speaking very opposed to what I understand to be David Goggins-esque attitudes towards almost anything. In this particular instance, I think something like: if I ask why it is miserable to learn a particular subject, the answers that come to mind are things like, first off, I don't care about this subject. I think that's not what we're talking about. You're asking about a world in which these great tools exist and someone is using one of these tools to try to do something they really care about.
Another reason why it could be miserable is that you have some idea that you're not going fast enough, or you're failing, or you're struggling. The misery comes from resisting that. It comes from feeling like you're doing poorly and you shouldn't be doing poorly, like it's bad that you're doing poorly. Maybe you're feeling fearful, like others are going to judge you, or you don't have enough time. I think that's basically an emotional problem that needs to get healed rather than a practical problem with learning.
In the case of something like organic chemistry, where you truly do just need to learn 200 names, that can be done very cheaply using modern memory systems. Organic chemistry students suffer through this and they don't need to. But even with modern memory systems, you're probably going to spend a total of, I don't know, 100 minutes across some weeks studying all of these formulae. That still is unpleasant. Can that be resolved? I think the answer is yes.
I was thinking about this in the context of the cell biology by the numbers book I was telling you about, where there are all these things like the volume of the nucleotide is a nanoliter. Studying the flashcard "what's the volume of a nucleotide" is not terribly pleasant. It's not suffering exactly, it's fine. I'll do it while waiting in line. But I think there is a better version of that, which is solving an interesting Fermi problem that involves that term. Something like, "If I have a vial of the COVID vaccine, how many copies of the COVID RNA are likely to actually be in it if the vial is a milliliter large?" That's kind of a fun little question. I can enjoy sitting and noodling on that. In doing so, I will need to retrieve the volume of the nucleotide to help me make that approximation. So I think there are moves like that you can use to paper over any remaining stuff that feels necessarily unpleasant or rote.
Dwarkesh Patel
I'm actually surprised to hear you say that, because one way I read your stuff is that this is actually a way of endorsing the traditional way of thinking about education, but using new tools to get the traditional outcomes. To give you an example, you go back to a headmaster from the 1900s and say, "Is it important to have the taxonomy of a subject memorized?" He says, "Of course it is. That's how you're going to spend a year memorizing taxonomy." And you would say memorization is actually important so that you have a dictionary by which to proceed on the subject. So in those ways, you have new systems for doing that same kind of thing.
The reason I was expecting you to say that you have to be disciplined if you decide to learn something is that in the case of the three hours of intense learning followed by an intense piano session, you were just really tired at the end. And you were like, "But no, this isn't something I have to do this evening." So yes, I'm actually kind of surprised to hear you say that.
Andy Matuschak
Yeah. No, I really enjoy this tension. I'm probably overstating my reaction to the Goggins reference, a bit of an overcorrection. But this really is how I feel, and I feel this tension all the time. The histories in educational psychology that I'm most aligned with are the most robotic, authoritarian kind of histories, and also the ones that are most unschooling and Montessori-esque. I really have a ton of sympathy for elements of both directions. There's a kind of weird synthesis in my head that I can't fully externalize.
I guess part of what I'm saying is aspirational. It certainly is the case that I do in practice use willpower to make things happen. Just as an example of something totally contrary to everything I was saying, I use a tool called Beeminder, which charges me if I don't do certain things. This sounds kind of military, certainly more authoritarian than the freewheeling butterflies gesture I was making a moment ago. I use it to make sure I do my memory practice. Shouldn't my memory practice be so joyful? It's at the center of my research. It should be the most interesting, exciting part of my day, but often it's not. So I use this to do it anyway. There's some tension here.
I think I do want to say the reason I'm willing to endorse this headmaster's view about the taxonomy has to do with the price. I did a bunch of memorization in high school and it was very inefficient and very uncertain. It was emotionally difficult because I wouldn't even feel confident that I had learned the stuff. I didn't know what it was to learn something reliably, to be confident that I'd be able to recall it. It was hugely time consuming because I didn't have techniques or tools. Now, part of why I respond so favorably to "yeah, just learn the taxonomy" is that for me, it's just trivial. Yeah, sure, throw it in the deck. It'll consume a total of 15 minutes over the next few weeks, and then I'll know it. It just doesn't cost anything. Other things in learning do still have real costs, and those are maybe more difficult to negotiate.
Dwarkesh Patel
Actually, this is a good place to ask you about unschooling and your attitude towards it. I think somebody on Twitter had this question: your kids as they're growing up, how are you structuring their education?
Andy Matuschak
To be clear, I don't have kids.
Dwarkesh Patel
Hypothetical kids.
Andy Matuschak
Right, so you're going to hear the foolish response of a person talking about what one would do hypothetically. This is very difficult. The school of course has many purposes other than instructional. It has a social purpose, a societal purpose, a behavioral purpose, and a pragmatic purpose of basically babysitting. Those things can be unbundled. I think it's pretty interesting to consider that. If I actually did have a kid, I would probably consider that project pretty thoroughly. It's pretty likely that some kind of homeschooling situation would occur. It probably wouldn't be me being the teacher, but it would probably be people I would hire. I have some resources. I'm not wealthy, but I have some resources. That is maybe a difference.
During the pandemic, I was struck by Brian Tobol. He started a company which is now defunct, called Schoolhouse. The idea was that he noticed people were getting together in pods during the pandemic. In particular, they got together in pods with their classmates from school, maybe five or six kids. Some of these pods started hiring elementary school teachers who were not working because of the pandemic. The teachers would come to the backyard of one of these people's houses, and the five or six kids would get together with the teacher and do stuff all day. Buying one teacher's time split five or six ways was actually really tractable. Say you want to pay the person $50 an hour. That seems reasonable for a teacher. It actually costs substantially less than a private school. Schoolhouse cost something like a fifth of the cost of an elementary school.
Once you get to older grades, you might need specialists. It's actually not clear if you do. My friend Alec Resnick is working on a very interesting school called Powder House in Somerville, Massachusetts, that does something like the model I just described. You have adults who are in more of a coaching role. They aren't necessarily domain specialists, but they'll connect people with domain specialists. So anyway, I would explore something like that model. I'm sorry, this is a little vague.
Dwarkesh Patel
Let me ask about specifics. You have a 12-year-old child. At this point, you've taught them arithmetic and reading and everything. Do you proceed and have them learn biology, chemistry? Or do you just say, "What are you interested in? Are you interested in Roman history? Let's learn about the aqueducts." Or is there an actual curriculum that proceeds until they get to college?
Andy Matuschak
Yeah, this is really challenging. One of the heroes of the reform school movement is the philosopher John Dewey. He has a lovely book called *Experience in Education*, written near the end of his time, looking back on all his efforts to reform schooling in an unschooling-ish direction. He was never as extreme as that, but broadly looking for freedom on the child's part. He makes this wonderful argument that because a 12-year-old doesn't have a fully developed prefrontal cortex, certainly not a fully developed sense of self, to let them do whatever their whim commands them to do in any given moment is actually not freedom. Rather, it is chaining them to whatever that impulse is. It makes them the subject of these tides of impulse. I think that's a pretty compelling argument. It doesn't authorize tyranny, but it also suggests you've got to be a little bit skeptical about the plans of 12-year-olds.
Dwarkesh Patel
How skeptical?
Andy Matuschak
I don't know. I think I would probably have stronger opinions on that if I had a 12-year-old. But my instinct as a foolish non-parent would be something like a kind of mix. I would be interested in exposing the 12-year-old to lots of topics and possibilities. I would be valuable in expressing the consequences of any particular actions. If they just want to compose music all day, we could talk about what that means, what kind of life that looks like. I would try to be non-coercive as much as possible. To some extent, the child should be allowed to feel the consequences of their choices.
This is complicated by the fact that, again, I'm not wealthy, but any child of mine would have chances. If they made some weird choice about a career path when they're 13 and so they didn't get into Harvard or whatever, that would be okay. They could be 24 and finally figure it out then, or 32 and finally figure it out then. It would probably turn out fine. So this doesn't seem like reliable guidance. You should notice I'm feeling very confused about it.
Dwarkesh Patel
No worries. Okay, so one question I have is historically, and maybe even to the modern day, it seems like improving education has been a very intractable problem. You did reference this earlier when we were talking about gearing towards the median student versus whatever percentile you're working with. Do you feel like there's been progress even in the percentile you're gearing your stuff towards? If not, what is the explanation for the relative stasis? We have so many new tools with IT. What explains the broader stagnation here?
Andy Matuschak
The fun answer to your question is that there's been a ton of progress. Actually, things have been pretty good. In 1900, 6% of teenagers graduated high school in the US. That doesn't mean 94% didn't have an education we would regard as a high school education, but it kind of means that. It roughly means that. Many of these people were homeschooled. It's also the case that a high school education meant something lesser to them. A substantial fraction of high schoolers now study AP courses and complete them in high school. That's at the high end. On the low end, illiteracy was a very live situation 100 years ago in the US and is emphatically not now.
It is the case that something like 10 to 15% of adults, depending on which polls you use, might struggle to perform simple kinds of number manipulation or reading or writing tasks. But our bar has basically moved. It used to be, "Can you read at all?" And these tasks are maybe a little artificial, maybe not relevant to their day-to-day, which is why they're experiencing this. The fraction of the population who graduates at 17 knowing a particular amount of stuff has basically moved up monotonically. This is mostly about the bottom portion of the population. It used to be the majority were effectively uneducated past age 10, other than informally and in their trade. The story of the 20th century has been in part one of mass education. Part of why we have a service economy and IT economy is that basically all of our population is educated at a particular level.
If you look at national tests of fourth, eighth, and twelfth grade math and language proficiency, you'll see really slow movement in the 75th percentile and practically none at all in recent decades. But you'll see absolutely enormous movement in the bottom quartile. In some sense, the story of the last 20, 30 years has been closing what's often called the achievement gap, where certain groups from underfunded schools or unsupportive households were just not having anything like the educational attainment of their peers. That story has changed.
Dwarkesh Patel
Every other part of the distribution has been moved upwards. Has the ceiling been raised significantly?
Andy Matuschak
Well, it depends on what we mean by the ceiling. You can go back hundreds of years and the most learned people around are just incredible. Look at how many books Thomas Jefferson read. There's a story where Kennedy had a bunch of Nobel laureates at the White House in 1963, and he said, "This is the greatest collection of genius and insight and wisdom that has been collected into this room ever since the time that Thomas Jefferson dined alone." I think it's very hard to raise the ceiling. The ceiling has aristocratic tutors. The ceiling has whatever family dynamics and heritable propensities produce tremendous intellectual greatness. Early 20th century schools produced von Neumann. It's certainly not at all clear that they are now producing more von Neumanns. In fact, von Neumann's production seemed to have probably very little to do with any kind of mass schooling we would recognize. As far as the very top, I think that's difficult. We're talking about an institution created for the masses.
Dwarkesh Patel
There have always been people using resources outside of those kinds of systems. The mass system doesn't seem to help those people. That doesn't seem surprising. By the way, on the von Neumann thing, the mass system doesn't help him. What is the production function for von Neumann?
Andy Matuschak
Lots of people have studied this. I actually am not a student of von Neumann's history. I know that many of his peers, the 20th century greats, got something like aristocratic tutoring or came from small Eastern European incredible schools. There are stories about these things. I actually don't know them, I'm sorry.
Dwarkesh Patel
You actually heard about that one high school?
Andy Matuschak
Yes, yes. Okay, interesting. Are we getting worse at the von Neumann production, or is it just static?
Dwarkesh Patel
Well, maybe. Here's a theory that seems kind of plausible. If someone was going to have aristocratic tutors in the late 19th century, would they now go to a fancy private school, and would that experience actually be less good for them? I don't know. I think it's probably more likely that they'd go to the fancy private school and also still have fancy tutors, and then go to a very exclusive university where they're going to get a bunch of highly hands-on interaction with professors. Although the reason that might not be the case is the opportunity cost for people who might become teachers or aristocratic tutors is much higher now. The kind of person who would be your tutor can now directly be making lots of money in Silicon Valley or on Wall Street.
Andy Matuschak
That's interesting. Okay, so that would be an argument that maybe it's not so much about the 20th century that we've gotten worse about this, but more over history. Maybe Aristotle was a tutor to Alexander the Great, and now Aristotle would be a full professor and take that job. That might be so. I think it may be the case that some tutors have been priced out of the market. But it's not clear to me that the most expensive tutors actually would be the best.
There is a bunch of empirical research on tutoring. One of the questions they ask is what kind of experience level the tutors need to have. It's interesting how far you get in tutoring efficacy when the tutor doesn't necessarily know anything. Just having another warm body there actually contributes a very large effect. Things get better as you get an expert, and I also have a healthy skepticism of these studies. I think part of the role of having Aristotle as a tutor is communicating a worldview. It's not something that would show up on a test that these studies would be measuring. Having an extremely inspiring individual might actually be the important component. And inspiring is going to be highly correlated with expensive.
Dwarkesh Patel
I think not necessarily. I don't know. That feels complicated. Especially today, the material is available. What the tutor is bringing is the inspiration and the motivation. Not exclusively, but one of the large parts of their role.
Andy Matuschak
That's right. They're not really responsible for instruction. I'll say also, I know lots of people who have postdoc tutors right now. These people as graduate students are very pleased often to have a $60 an hour tutoring commission. That's a little sad, but the pool of available postdocs to hire as tutors is very large now compared to how it would have been 100 years ago.
Dwarkesh Patel
The pool being bigger doesn't mean the top 1% are getting more, though. So I think that's undecided. There is a question of, have teachers gotten better at their jobs over the last 50 years, say?
And there are some ways in which maybe they have. There have been a bunch of projects of trying to disseminate certain research results, ways of instruction that are more effective than other ways. Like it's good to interleave stuff, for instance, rather than doing blocked units where it's like, okay, we're going to talk about the Civil War and then we're going to talk about women's suffrage. It's better that those are somewhat far apart, but it's better to kind of weave these things into each other. Not just in history, but in general. So that kind of dissemination has been happening more systematically in the last few decades. I'm unaware of any studies trying to establish anything about the efficacy of teachers now versus long ago.
Andy Matuschak
Well, I'm sure you've seen the claim that one of the consequences of the very unfair circumstance of the mid-20th century was that one of the very few occupations an intelligent woman could pursue was teaching. And now that other options are available—which is obviously hugely, hugely good—there's other competition for those same very intelligent women.
Dwarkesh Patel
Oh, that's interesting. I haven't heard that claim.
Andy Matuschak
Yeah, I'd have to think about it. It's not clear to me how much intelligence matters, if you want to think of that as some kind of separable quantity. Or whatever trade is relevant. You just had a population that was hostage to either housework or teaching. If that were true, and there are a bunch of people who are now astrophysicists, it's not clear to me that they would have been the good teachers. Being a good teacher is often about empathy and effective communication and care. It's very personal, very intimate. You need to understand the subject, but to teach a 15-year-old, you actually don't need to understand it at a postgraduate level necessarily.
There's a bunch of studies on the impact of domain knowledge on teaching efficacy. I've read some in math. One thing that comes up is if you aren't very familiar or comfortable in math, you will struggle to do inquiry-oriented classes—classes that are more about creative ways of thinking with math or open-ended problems, as opposed to "here's how to do this algorithm." To conduct those kinds of classes, you have to be able to think on your feet. You pose a difficult question to which there may not be just one appropriate answer, and your students will throw all kinds of stuff at you. You have to be able to take that stuff, integrate it, show how one student's answer relates to another's, and show how those conceptions can be built upon to produce some useful understanding. This kind of improvisation requires mathematical familiarity and ability, but I don't think it requires anything like an extraordinary ability.
Dwarkesh Patel
Yeah, but more than the extraordinary have been pulled out of teaching as a consequence.
Andy Matuschak
I'm just wondering what the correlation is. If effective teaching is mostly about empathy, then maybe it's anti-correlated. The people who are going to be good particle physicists might not make good teachers anyway.
Dwarkesh Patel
Interesting. Maybe. Why hasn't hypertext changed how people write more? So often, I write a blog post and I wonder how much different it is with the knowledge that I can add footnotes and link to things. I'm actually a fan of how Wikipedia organizes content. It's genuinely surprising how often the best explanation of a subject is just this resource that is trying to explain every single subject. There's this practice where you don't need to do exposition in every single topic. You can just hide it behind links. So why hasn't hypertext changed writing more? Online writing, at least.
Andy Matuschak
This is a really good question. I think the reason Wikipedia works as well as it does is that encyclopedia entries are already forced to stand on their own. That was true before hypertext existed. In fact, encyclopedias were already hypertext-ish before there was hypertext. There are other interesting kinds of hypertext that existed pre-computers. There's a very interesting book called the *Syntopicon* from Adler. If you want to understand what classical authors had to say about a topic like a father's responsibility to a daughter, you can look that up in the *Syntopicon* and you will get references across Rousseau, through the Bible, and so on. Those are kind of hyperlinks. They're printed on dead trees, but you're expected to get the books down and look up the appropriate pages. The *Syntopicon* wasn't that successful. I think it's in part because those concepts, unlike the Wikipedia entries, don't quite stand on their own so cleanly. You need sinews, you need linkages.
While Wikipedia is an astounding resource, I find it rarely to be the best available introduction or explanation of a topic. I find it often to be a good jumping-off point. It'll help me know the right thing to ask about. It's good as a reference. Hypertext is a very effective navigational aid. It can help you get to a spot you're looking for very quickly because it's about automating flipping through pages. For a reference, it's very effective. If you have a table of chemical compounds and their properties, hypertext is going to let you navigate that book very effectively. Dictionaries have been revolutionized by hypertext. Navigating around thesauruses by clicking on links to say, "Oh, shade it a little bit more like that," is a much better thesaurus.
I'm making the case that certain kinds of texts are more amenable to hypertext because they are more amenable to having the reader dropped in the middle of them. Encyclopedias are like that, dictionaries are like that. Most text is not like that. Most concepts are not like that. Most ideas are embedded in something holistic or richer. They require a narrative arc. They're difficult to excerpt. Not everything, but things that are not so raw and atomically informational.
There were all these dreams of hypertext novels. Some people wrote them. One of the problems a hypertext novel has can be seen in a choose-your-own-adventure book that existed before digital hypertext. The author is forced to write something like a lowest common denominator story. The page that is the destination of a hyperlink has to work as the endpoint of all of its references. So it can't establish any coherent or consistent arc unless there's a sameness to all of the references. The more sameness there is to the references, the less useful hypertext is. A lot of people have been disappointed by this conclusion, I among them.
I do find hypertext very useful in my own notes, not really for reading. I don't think it makes for a very good reading experience for others.
Dwarkesh Patel
You have a separate web page where you have your working notes for the audience. It's a very cool UI format to explore your thoughts.
Andy Matuschak
Thanks. It does an interesting thing for me as a writer. It lets me build stuff up over time. Today I was reading a very old cognitive psychology paper on the topic of adjunct questions, which we discussed earlier—the effects of asking questions while you read, not on remembering the information covered in the questions, but on the general effect it has on stuff that isn't touched by the questions. I have some notes on the design decisions of the mnemonic medium, this Quantum Country thing I was talking about earlier, interleaving the questions into the text. Those notes are partial. They evolve over time. What was the impact of doing this? My notes about that have come from interviews with readers. They expand when I read a paper like this that's relevant to them. When I go to design the next system and I'm thinking about the role of questions in text, I'll have a place to look. The role of hypertext in this is roughly as a navigational aid. It's possible to do this without hypertext—you just end up with what Luhmann had, a giant dresser made of card files rather than drawers for clothing.
This goes back to the original conversation about why people like Tyler [Cowen] are able to integrate so much information without an explicit note-taking system. I just remembered another person who comes to mind immediately when I think about a person like this: Bernhard Barth. Again, you have an example of somebody who is extremely prolific—a daily finance newsletter with tremendously detailed and insightful content. It's like a daily note-taking practice in some sense. Nothing quite accumulates for either of them, at least not in the same way.
Dwarkesh Patel
One thing I found interesting about Matt Levine's newsletter is that when he's talking about a topic repeatedly, like the recent bank collapse, he will have to explain some concept like interest rate risk over and over again for days. Every day he has to explain it, but every day he explains it anew. Every day the explanation is colored a little bit by that day. This is an argument against the kind of note-taking that I do. It's an argument for ephemerality, for recreating the thing every day because it will change and become inflected by what you're thinking about now and your experiences.
Andy Matuschak
I think it's pretty interesting. I find myself these days doing a mix. I have a journal that's about today, and I'll do a bunch of writing, and often I'm recapitulating stuff I've written before. And I have these other things that are trying to be more durable, a useful reference that can stand outside of time. The combination feels useful. I don't yet have a clear model of when one is better than the other.
Dwarkesh Patel
An interesting way to tie in what you just said with hypertext: Burns' newsletter doesn't give that much context. Often you'll find yourself lost about what the concept being talked about is if you're not familiar with the topic. I asked him if he'd considered doing narrations of his blog posts, like Scott Alexander has somebody who has a podcast where they narrate his blog posts. He said he didn't think it would work out as well for his because he heavily relies on the old blogosphere's norms around hypertext, where you can add jokes and sarcasm based on links. For example, he had to write up about SPF and his collapse. He had a bunch of links: "You want to learn more about margin calls, read this. You want to learn more about this." And he goes, "And if you want to learn more about the psychology of utilitarian bets, read this." And it was just a link to the Amazon page of *Crime and Punishment*. That kind of stuff is harder to do.
Andy Matuschak
Yeah, you're right. He's leaning more on his past explanations, which is interesting because he can't update them. That format of writing a newsletter and then linking to past newsletters, or as you say, the former blogosphere thing to do—you have a series of six words and each word is a link to a previous post on the topic. I've written stuff like that. It's kind of funny. It's approximating the durable note thing I was writing about, but without the ability to revise it over time. Maybe for many topics, you don't need that ability. I wonder now what fraction of my notes are basically in the state they were when I did my first major revision of them. It's probably at least a third. It might be more than half.
Dwarkesh Patel
What person-driven notes have you published?
Andy Matuschak
I don't know. By word count, by note. My journal notes are not published, and there's one of those every day, so there are a lot of them. If we're looking by note, we're excluding all of those. I also have a note about all of the people in my life. Those are, for the most part, not public unless they're public individuals. There are a lot of notes that are not public, but they're mostly not durable. They wouldn't be all that meaningful to others. The journals might be, but they're also intimate.
Dwarkesh Patel
Are they written in a way that would be intelligible if you were to give them to somebody else?
Andy Matuschak
It depends. Usually my journals are complete sentences, complete paragraphs. Sometimes bullets, sometimes kind of veering and breaking and changing to new subjects suddenly. They tend to be filled with links to the things I'm talking about, in part because I'm trying to accumulate context in those things.
Dwarkesh Patel
Why not just shorthand?
Andy Matuschak
Partially because past me is another person. It's a cliché, but I am routinely looking at journal entries from a year ago. You could view that as a failure of this note-writing system. Maybe in some ideal sense, I shouldn't be looking at these journal entries because if something's important and it's going to be something I refer to a year later, it should be in some durable, evergreen note. I don't know. You don't always want to do that. It feels like prepping. Maybe there's an amount of prepping that's good. We live in California, and maybe everybody should have an earthquake kit. Maybe that's good, but maybe you don't need to hoard 300 cans of beans. There's an amount of prepping that feels reasonable, and there's an amount that feels dutiful and unpleasant.
Dwarkesh Patel
As a researcher in these sorts of Silicon Valley circles, what is your opinion on the startup advice of "do things fast, fail fast, get to users immediately with an MVP"? As somebody who is making products but is also in a different mode than a typical startup, how do you think about advice like that?
Andy Matuschak
I have complicated feelings about this. I need different advice on different days. Different people need different advice on different days. When I was getting into this kind of work, that advice led me to not think all that deeply about the ideas I was exploring. An idea would come up, and I'd think, "Oh, I can try that." Then I would try it, learn something, and repeat. There wasn't a sense of building a theory of what the problem is and what it would mean to solve it. Instead, it was just a theory of action, as opposed to a theory of change. If you imagine you're at some current position and eventually you want to get to some goal state, a theory of action is you look around and say, "What can I do? What can I build? What do I see as possible?" A theory of change is to look at the endpoint and try to work backwards. The metaphor is imperfect because in research, you usually don't exactly know what the endpoint is, and you certainly don't know how to work backwards. Following that advice historically often led me to try things that were straightforward.
I think the most powerful design work has ideas in it. What makes a nonlinear text editor—the text editors we all know and love—so powerful is the observation that writing is a nonlinear process, but writing with a pen linearizes it. Many, many other observations like that, and on the nature of what it means to have a thinking environment, is how we got that particular interface. The way we got powerful programming environments, insofar as we have them, is by people thinking very hard about what it means to specify a system and coming up with new primitives that express those ideas. The most powerful interfaces are often the expression of new ideas or new primitives that capture new ways of doing, new kinds of objects that can be manipulated.
In Photoshop, you can manipulate a photo by means of a construct called a layer. This is a very strange idea. It has some precedent in dark rooms where you could have sheets of film—not the negatives, but sheets of gels you could put over the lights to affect the exposure. But in Photoshop, they're non-destructive and continuously manipulatable. The layer is a new primitive introduced into the activity of photo editing, and it utterly changed what you could do in photo editing.
What I'm saying in a very long-winded and confused way is that I think it's difficult to have ideas by means of building an MVP very rapidly. Now, if you have an idea that you think is interesting, it is good to test it rapidly. Part of why I'm confused in my response is that it's good advice once you have something worth testing. But for me, adopting that mindset—and I've lived in it for so long that it's very ingrained in me—makes me not sit in stillness and in confusion and in contemplation with the ideas long enough for them to be good.
Very concretely: Michael Nielsen and I made this Quantum Country thing. When I was trying to think about what to do next, the most obvious idea was, "What if we just try that with lots of other things?" That idea occurred to me, and the pandemic had just struck, so I was feeling a little timid creatively or emotionally. I wanted something that felt kind of safe. I knew I could do that. "Okay, I can build a platform that generalizes this thing we did for this textbook." So I did. I did it relatively quickly, in a few months. That wasn't the right thing to do. It wasn't really the right question to be asking. The idea wasn't that strong. It wasn't the right way to test it. Building this highly general version of it, I would have been better off building more one-offs rather than a self-serve thing that anyone could use.
This comes down to the difference in aim. I'm not trying to build some kind of scalable thing for the world at this moment. I'm trying to build the idea. The prototype is an expression of the idea. Once it arrives at a good place, then maybe there can be some scalable solution. But it's not necessarily at that place yet. Until it is, there's a lot of thinking and sketching that goes along with the building and prototyping. Part of my confusion is that often I still need to hear this advice. Often I will tie myself in knots in theory land, and what I really need is to have a friend sit me down and say, "Is there a piece of this that you can carve off and build next week?" So you're hearing a lot of tension.
Dwarkesh Patel
Interesting. What was the consequence of shipping Orbit out? Did it feel right at this scale?
Andy Matuschak
I learned some things. It was fine. It taught me a lot about where that particular format succeeds and fails in other venues. It was just not a very effective way to find those things out. I built this very general MVP. It has very few features and is very simple, but it was highly general. It's a deployed thing that has infrastructure, accounts, all this stuff you do when you're building a real thing. That's very different from working with one author and seeing if I can make it work with one other book that's very different from Quantum Country. To form a specific question or theory about, "Well, it worked for this text. What's the next kind of text that would be good to test with?" And then to do that, I could have done it much more rapidly.
Dwarkesh Patel
Why do you think this idea of "tools for thought" has nerd-sniped so many people in Silicon Valley?
Andy Matuschak
It contains this message for technologists that they can potentially be very powerful, and that's always tantalizing. I think it also feels very actionable for people in a way that's actually super misleading. I meet tons of people who tell me they're interested in tools for thought, and 95-plus percent of them are engineers. The problem is that building an interesting tool for thought is basically entirely a design problem. Their design ideas are usually not very good or troubled in a variety of ways. Yet they can make a thing. They can make a thing that solves a problem maybe for them in their lives, and that feels very tantalizing or encouraging. It feels like something they can get their hands around.
I think we in Silicon Valley are very, very interested in thought. We are a thinky people. People are very interested and engaged with anything that can potentially expand our capacity there. That too is tantalizing. "What if I could think better?" It's also tantalizing because it's meta. There are all these cliches about people tinkering with their dot files endlessly or tinkering with their blog website, which has two posts on it, but they have to rewrite it because they want to do something else. And now the new one will have three posts on it before they rewrite it again. Tools for thought also scratch that itch. It's work about the work.
This sounds very cynical. I don't mean for it to be. I'm just trying to earnestly answer the question. Here's a more optimistic and generous response: Many of us got into computing because computers portray a sense of personal empowerment and possibility. We maybe remember growing up, locked in our bedrooms at midnight, fooling around. We have this very powerful tool at our disposal, and it's opening up these worlds for us. For many people here, that was a formative part of their personal development. Anybody pointing to that and saying, "We can do more stuff like that," is going to be pretty compelling.
Dwarkesh Patel
Okay. This was an interesting question from Matt Clancy on Twitter: "What are the characteristics of a good crowdfunded research project?"
Andy Matuschak
One of the unfortunate things I've learned in my crowdfunding experience is that there are some dynamics that seem hard to change. One of them is churn rate. Like any subscription revenue business model, you lose subscribers every month. In my case, it's about 2%. It's not that large, but it does mean I need a certain number of new subscribers all the time. One thing I've learned is that the churn rate is surprisingly insensitive to anything I do. I've experimented with a variety of things, and it really hasn't meaningfully changed the churn rate. What does change things is getting more people into the top of the funnel. In other words, marketing. There are some things that have maybe affected the fraction of those people in the top of the funnel who convert. I really hate this way of thinking about it.
In summary, the thing I've discovered that's kind of sad is that I end up having to think about this a little bit. In particular, I realize that this project only even slightly works—the one I'm doing for crowdfunding—because it's understandable to others, interesting to others, and already in a place where there are some results that look kind of promising. People are like, "Oh, more like that." But it's very easy to imagine other projects that are not broadly applicable. If I were doing marine geology stuff, I probably wouldn't have a big crowd of internet people excited. So that's one property: this work is very general. It applies to many, many people. It applies to people who have disposable income. If I were doing a research project on writing practices of disadvantaged artists, I think my audience might not have as much disposable income.
I have already made some progress. I think that's probably important, unfortunately. It's probably very difficult to use crowdfunding in the very early days of a research project. I've already chosen a research agenda or direction, and I can kind of express it. This says that crowdfunding applies probably after the first few stages of research have been completed. At some point, I'm going to be using this crowdfunding to figure out the next thing, and I won't be able to explain it to anybody. There certainly are seedlings like that, but you have to have something in flight. I probably need to be able to say something about my progress with some kind of regularity.
For instance, my wife is working on a study of biological markers of age in association with delirium and traumatic brain injury. To do this, she is signing up patients who show up to the hospital with traumatic brain injury. Once they agree to participate, she takes various blood samples. Recruiting enough patients to get the significance she requires will take two years. She can report a little bit of intermediate stuff, but certainly not a monthly update. It would be a weird Patreon post.
Dwarkesh Patel
Yeah, I mean, I can't quite report monthly updates either, but I think there's a cadence that's necessary. Why bother with it at all? I'm sure there are many wealthy individuals who would be happy to single-handedly fund your research. Is there a reason you chose crowdfunding?
Andy Matuschak
Those wealthy individuals are very welcome to reach out and offer to do so. I've been fortunate to have many high net worth individuals as sponsors. But each of them is providing, on my Patreon, a sponsorship of $100 a month. That's what I get from these people. I'm certainly not getting wild offers for more.
Dwarkesh Patel
I think you're using the wrong tool, given the distribution of your audience's wealth.
Andy Matuschak
Maybe. There are a couple of ways to interpret your question. One question is, "Why crowdfund when I could appeal to high net worth individuals?" Another version is, "Why crowdfund at all?" As opposed to raising grants or talking to philanthropies. Are you mostly focused on the first of those?
Dwarkesh Patel
Yes. If I'm going to be honest, it's because it has worked. The history of the crowdfunding of this project is, like many things in my life, the result of goading from Michael Nielsen.
Andy Matuschak
Early on, when we were working on this Quantum Country project, he suggested we set this up. And I kind of hemmed and hawed. I said, yeah, it's going to be a distraction. We don't really need this right now. Let's deal with it later when we have something to show. And he said, no, no, no. Let's just get it started. It's going to be a long time to get enough subscribers and so on. And it turned out he was right.
The process of crowdfunding a project takes maybe a couple of years, at least in my experience, to build up a subscriber base, and starting earlier was better. If that hadn't worked or if we hadn't started early, I think I probably would have just reached out and asked for individual help. And I probably will, if it fails on me.
I'll also say, when there have been specific projects that I've wanted to do that require hiring people, I have reached out to high net worth friends and they've helped, but in the low five figure, four figure kind of range. And that's great. I'm very grateful. So the answer may be a mix.
One of the big limitations to the crowdfunding thing is it seems pretty clear to me it can't sustain a team or institution. It can barely sustain me. I earn somewhere between a grad student and a junior faculty member. That's kind of OK, I guess, for a variety of reasons that are pretty particular to my circumstance. But it certainly wouldn't be OK for everybody. Even for me, it doesn't allow me to support others.
It's even more striking because in terms of the success of what a public intellectual is, you're like a public researcher. Some of your research is public or you publish it in a public facing way. Even in a context where you're pretty well known amongst the audience that would be happy to fund this kind of thing, if the LeBron James of independent public research is between a grad student and a junior faculty member, it's not a great sign in general.
I think it's worth considering that I'm also maybe not very good at this. I'm not that successful as a researcher. I object to the LeBron James characterization. It's true that I'm maybe the most successful crowdfunded researcher in tech stuff, and that's kind of weird. But the last couple of years, I've figured some stuff out, but I wouldn't say I've had any spectacular hit publications.
One thing that is true is that when I have big publications, I get a lot of new subscribers. So I think there is some kind of market force that could be higher if I were having a more spectacular success with my research.
I think it's also true that I pretty systematically avoid marketing it. That's a kind of self-protection thing. I am really worried about the corrosive influence of audience and marketing on honest inquiry. It is very, very easy to distort my work. It's almost a default to try to make it be something that people would be more likely to like, rather than the thing that I actually want to investigate, or to do the boring simple version rather than the interesting deep version so that I can publish more stuff more often.
One thing I've chosen not to do that has definitely cost me financially is to publish what academics would call minimum viable papers. It's very common to take any new marginal insight that is above a particular bar and publish that as a little thing. I just haven't done that. I've written little informal letters to my patrons saying, hey, I figured this thing out this month. If I were an academic, I probably would have published that as a paper. If I were a marketing oriented, crowdfunded researcher, I probably would have done some glossy thing and promoted it. But actually, I just don't think it's that big a deal. I'd rather get on to the next thing.
Really, what I'm worried about is marketing. It makes it so hard to be honest with oneself, at least in my experience. Not only to be honest about what I think is interesting and important, but even to be honest about the results. Every paper is in some sense a little marketing piece trying to make the case that it's significant, that its results are really exciting or really important. That is really corrosive to discovery.
You need a really strong emotional connection to the work in order to do good work. Part of that emotional connection comes from a sense of excitement of maybe being hot on the tail of something really good. But there's a temptation to portray what you found in the best possible light, to downplay its limitations, to take up space, and to totalize. All of this is death for discovery.
Dwarkesh Patel
It is interesting to hear that from somebody who, inadvertently and without intentionally trying to do so, has done a good job of spreading your material. I've known about you for a long time. I do wonder if there's an element of, if you get to a certain level of quality, trying to market your stuff not only doesn't help, but probably hurts you. If you try to think of somebody like Gwern trying to post YouTube shorts of his blog posts, it would just be like, what are you doing, man? It's just so good that he doesn't need to promote it.
Andy Matuschak
Gwern is an interesting example because there's a simpler failure mode. I still routinely run into people who tell me, oh, I've really liked your work for a while. I didn't know you had a Patreon. That's a simple failure of a certain kind of marketing on my part. I think Gwern actually has this even worse. I adore Gwern. I have learned so much from him. You can go to his Patreon page and he actually makes public his revenue. He makes on Patreon a tiny fraction of what I do. I think this is inappropriate. Gwern is a much more impactful researcher than I am, and he has a much bigger audience than I do.
The fact that they aren't converting into patrons is mostly a matter of the way that he talks about it and the way that he presents it. It's not that he needs to market more people to his webpage. I expect he has plenty of traffic and a large audience, much larger than mine. I think it's more just that there are a bunch of variables about the way that you talk about this membership offering, and none of us really want to think about them. I've ended up at a slightly more effective part of the space, but I'm pretty sure that there's much more effective ways to do whatever it is I'm doing.
Dwarkesh Patel
This is a really interesting problem. I have a Substack where if people choose, they can help contribute to the podcast. While it's broad enough revenue to help pay for certain episodes and traveling, in comparison to now that I'm going to be doing ads, it's a small fraction. Some people might say it's unfortunate that you have to do ads. Maybe listeners will just be finding out for the first time that there was an option on Substack. But also you don't want to be in the position where you're asking listeners for money every episode.
Andy Matuschak
I hate asking people for money. I think this is a common issue for creative people. I hate it. I really hate it. I probably need to get over this.
I do want to make one point. I had much more success with my Patreon when I recast it from "please support me" to something that feels more like an offering: "become a member." When you become a member, these things will happen. The things are not terribly substantial necessarily, but there's a difference between a tip jar and a membership in people's minds. Becoming a member means something. If you could offer something small that feels membership-ish, you might get very different results.
Gwern has the kind of tip jar vibe. I these days have kind of a member vibe. My instinct is that if he were to move to a "become a member of Gwern's lab" kind of thing, he would have better results.
Dwarkesh Patel
He has a thing on Patreon where it's like if you donate five bucks or eight bucks, he'll read an entire book and review it.
Andy Matuschak
This is crazy. I don't know if anybody's ever taken him up on this.
Dwarkesh Patel
That's like valuing his time at a dollar an hour.
Andy Matuschak
Yeah. I don't quite understand this. I think it's also the case that you'd probably have an easier time asking for subscriptions if you had a larger audience first. You can build the audience for free and then have some bonus offering that's behind a wall, maybe. I feel very conflicted about this, actually. Maybe you can help me think about it.
I have all these patron essays. It's where most of my writing is these days because I'm waiting until I can collect enough things for the next big public piece. I have a couple of big public pieces in various stages of flight. I'm writing a lot for patrons, and I think much of my audience out there doesn't even know that's there. One challenge of member only content is even making clear that it's there to others. Often people will try to achieve this by tweeting about or sending newsletters about this subscriber only content. I just can't bring myself to do it. It feels terrible to say, here's a link, but you can't view it. I can't do it. I don't know how you think about this or if you think about subscriber only material for Lunar Society.
Dwarkesh Patel
I was actually just about to mention this to you. I'm a patron and I got a chance to read all your patron only essays. They're great. I was actually thinking while I was reading them, it's really unfortunate that a person might not know they exist or if they're not familiar enough with your work to go ahead and sign up. It's just behind the Patreon. So it's a shame that the way to fund public work is to make some of that work less public.
Andy Matuschak
I think there are better ways to do this. I think there are design solutions. For instance, if it were the case that my work was mostly all in one place rather than in these separate places, and the subset of the work that's public was visually and structurally adjacent to the subset that's private, it would be clear that there's this additional stuff available. Perhaps you can see the first bit of it. Substack has this kind of thing to get some sense of what it is that you'd be seeing. I've invested zero effort into figuring out an appropriate presentation of this stuff.
Dwarkesh Patel
Another thing to consider is that a big part of the impact of your writing work is how many people actually consume it. The expected value on that is dominated by the probability it goes viral. It just can't go viral if it's behind a paywall. For example, I think your recent post based on your experience in industry at Apple about the possibilities of a Vision Pro and in what ways it's living up to and not, that would have just...
Andy Matuschak
Oh, thanks. I did make it public. I put it on Twitter and it was on the front page of Hacker News.
Dwarkesh Patel
Oh, right, okay.
Andy Matuschak
I think you're right. Usually I don't want this stuff to go viral. It's in medias res. I think the primary value that most of it has for people is opening up a window into a particular, very unusual kind of creative work that they don't normally get to see the behind the scenes of. Most of it is context laden. It's not really freestanding, and I don't really want to write it as if it could be freestanding. I've occasionally had the experience of one of these kinds of things getting widely distributed and then getting all these comments of people being angrily confused about what I'm even talking about. That's kind of discouraging. All of this to say, when I want to write something for broad public consumption, I write something for broad public consumption.
Dwarkesh Patel
Okay, I've got some questions from Twitter. Bring it on.
This is actually another question from Matt Clancy. Are there other examples of beneficial knowledge work practices that perhaps mostly work because they are a form of spaced repetition practice, but the participants don't realize it?
Andy Matuschak
Yeah, I think this is embedded in our working world. For a researcher, when you need to write papers regularly and you're writing those background sections and repeatedly explaining the history of a particular line of research and citing the appropriate sources, that is a kind of spaced repetition. When you have students and you're mentoring them in conversation about, in this kind of situation, you really need to remember to do X, that is a kind of spaced repetition. All of this stuff is kind of accidental.
Doctors have grand rounds where, even when they're not seeing patients regularly, they're still exposed to other patients. There's often a structure where while the patient is being presented, you're supposed to be trying to think, what would I ask? What would my differential be? Before you hear it, there's covert retrieval happening.
I think it's everywhere in our world. It's spaced and it's repeated. The thing that differentiates the kind of formal practice that I've been exploring is mostly that it focuses on material that you wouldn't otherwise normally have repeated, either because you're too early with it to have a consistent practice or because it's not firmly tethered enough in anything in your life.
Dwarkesh Patel
This is a question from Ian Bonagas. What is the optimal amount of effort that should go into a personal website? I think you might have noticed the amount of CSS that exists on dwarkesh.org, which is very beautiful.
Andy Matuschak
I don't like it. This is what everybody says about their website, right? It's three years old. That means I want to redesign it, but I will not allow myself because it feels like a distraction. What's the right amount of effort?
There's no general answer to that question. Of course, that's going to be my answer. But what can I say about it? What's the job of the website? What's it trying to do? I think many, especially engineers, do themselves a disservice by fretting over their websites unnecessarily, building vast technical infrastructure when really what they want is a place to post markdown files. They're better off just getting a Ghost installation and going.
The main thing to think about is what is it that you want to put out in the world? What is the ideal form of that thing? Try to find some way of organizing and expressing that. We have these common patterns, like a blog or a portfolio. Often people end up forcing themselves into these patterns. People end up using blogging software to make something that's kind of durable. Very interesting personal websites often come from people who are thinking about the shape of the thing that they want to put out into the world and making something that speaks to it. Often once you understand the shape, making the thing is not that effortful.
My website was not an enormous project for me. It probably should have been a slightly larger one given that my income depends on people coming through it.
Dwarkesh Patel
The working notes with the...
Andy Matuschak
That was a weekend, really. I feel kind of bad about it because it's made its way into tons of commercial projects now and people are like, this is the way to present network notes. I actually think it's not very good in a variety of ways. I spent a couple days on it.
Dwarkesh Patel
Wow. I thought this is where the question was alluding to, that you must have spent months on this. Nope.
Andy Matuschak
It is a little bit of the thing about the mechanic hitting one thing and kind of knowing the thing. I have design intuitions that led me in a particular direction, but there's lots of things I don't like about it. I just haven't allowed myself to spend any more time on it because I just don't think it's important enough.
Dwarkesh Patel
I have a question about your time at Apple before I ask the final Twitter question. Everybody has an iPhone, and from the outside there just must be so many different trade-offs and constraints when a thing like this is being designed. What is the supply of certain components and the cost? What do different consumers want? What features is the R&D team ready to put forward? At your time at Apple, you were responsible for a lot of these cornerstone design features. How is all that information integrated where a guy is like, all right, taking all these constraints into account, this is the design? How does that happen?
Andy Matuschak
One thing that's very interesting is that it's very compartmentalized. Basically none of what you just said was relevant to me. It was all pre-specified. At Apple, you have a little domain that's your own, and the boundaries of that domain are determined by everybody else's little domain. There's a person who's responsible for thermals. Actually, there's a team that's responsible for thermals. They figure out, what is our thermal budget? How much can we have the CPU on during what kinds of working situations? I basically can't argue with that. Those are just my constraints.
Dwarkesh Patel
But aren't those constraints informed by...
Andy Matuschak
Oh, it's iterative. We'll run into stuff where there's a thing we really want to do, but we can't pull it off because it drains power too much. Hey Siri is an interesting example. To be able to activate a voice command at any time without interacting with the device is great. People prototype that by having a thing listening in the background and watching for it. But that requires having the main CPU on all the time processing audio buffers. You simply can't do that; it drains the battery. That attempt led to eventually having this dedicated co-processor that runs at a lower power. It's very limited and restricted, and it can be on when the main CPU is not on, and it can listen for that sound.
Dwarkesh Patel
Is there a person whose job it is to take all things into account and be like, I decided given the memos from everybody that Thermals, you guys need to work on this?
Andy Matuschak
Not exactly. It's a little more push and pull. A given team, some of their priorities will be internally determined. The Thermals team has its hobby horses and knows what it thinks is important. Some of them will be externally determined. There is an executive team that makes ultimate decisions about the main priorities for next year's devices. They'll say, next year, we're going to do this Face ID thing to unlock the phone, and we're not going to have a home button. As soon as you want to not have the home button and you want to have the screen go edge to edge, this has all of these impacts top to bottom on the device. That decision creates lots of necessary work for lots of teams.
Some stuff is handled at a more local level. For instance, more locally to the iOS team, rather than at a top level executive team, the director of iOS apps might decide, we have this problem that because the apps were built at the same time as the system frameworks, we end up building our apps using this weird Frankenstein, partially internal framework, partially the public one that our developers use. The internal one is always a little bit different and not always maintained reliably, so we have all these problems about the skew between the two. A big priority for us is going to be to rewrite all the pieces of our apps to only use the public bits so that they could be distributed on the App Store. That's a more local decision.
Dwarkesh Patel
What I find really interesting about this is that it's possible for a two trillion dollar company to integrate all this information, to have a cohesive hierarchy where so many different products and trade-offs are being made. Does that make you think that over time, these very well-functioning tech firms will get bigger and bigger, that they can actually handle the cost of having this much overhead?
Andy Matuschak
Let me first respond to this observation about the enormity of the company, and then maybe we'll talk about the other firms. I think the reason Apple is able to do this is because of the way that they delegate. While there is a very strong command and control structure and for important decisions they really are made by a small group of people at the top, the individual leaders in the various areas at all levels of the hierarchy have an enormous amount of latitude. That's the only way that any of this can work. Individual people are given very, very strong responsibility and authority within domains to make decisions. That's how you can have all of these disparate products.
Craig Federighi is head of software at Apple. What does that mean? How can you be head of software? How many platforms do they have? iOS, iPadOS, watchOS, visionOS, macOS. Also, there's an operating system running in a bunch of the cables, on little chips in the cables. All of that is under Craig. What does that mean? In practice, it means there is a set of software concerns that he's actually super concerned with and thinking about day to day. When I was at Apple, I had Craig Federighi in my office talking about gesture recognizer heuristics with me because that was something that was hyper salient to him. At the same time, he was basically completely ignoring 95% of software related decisions. He just fully delegated those things to others.
There's a really interesting Harvard Business Review piece from a few years back about Apple's management structure. They have a couple of different concentric rings of kinds of responsibility for any given leader. I don't remember exactly the breakdown, but call it 5% of things that you're responsible for that you have your hands on at all times and you are directly manipulating and controlling. Then there's a ring outside of that that's a little bit bigger. Those are the things that you're keeping an eye on. They are salient to you. You're getting reports on them. You're checking in on them. You're thinking about them. You're coming up with ideas and sending them down the chain, but you're not directly controlling them. Then there's a bunch of stuff that you figured out how to delegate and you want to hear if there are problems. They talk about how that structure has evolved over time. It's been eight years since I've been at Apple, so I'm sure it's practically unrecognizable to me.
Dwarkesh Patel
This is a question from Basil Halperin on Twitter. Is the lack of spaced repetition adoption a market failure, or is the lack of adoption efficient?
Andy Matuschak
I think it's probably mostly efficient. In places where spaced repetition as it stands, without substantial novel cultural knowledge that's difficult to transmit and isolate, is valuable, we see a lot of spaced repetition usage. Among medical students who are highly motivated, have lots of reason to study, and the material is shaped in a way that's highly amenable to spaced repetition usage, there's tons of it. The med student Anki subreddit is bigger than the Anki subreddit.
Likewise, among language learners, spaced repetition in various forms is extremely common. Duolingo has spaced repetition integrated into it. Spaced repetition is kind of naturally present in the process of immersion learning. Modern spaced repetition tools between the Leitner box and Wozniak's SuperMemo were both originally motivated by language learning. In language learning, there's a substantial market for spaced repetition.
It could probably be used in a variety of more creative ways. Russell Simmons has pointed out to me that studying individual vocabulary words on flashcards often misses integrative opportunities. What you really want is to study lots of sentences or something like that, or possibly to build up towards that. Duolingo does something kind of like that. People in spaced repetition for language learning subreddits mostly don't. Some of them do. It's kind of complicated. There are edges of the market where you need early adopters to try things that have rough edges. The early adopters sometimes get cut and they bleed a little bit. That's why people aren't rushing into it.
As to why spaced repetition isn't widely used to learn quantum physics, I think it's basically correctly priced. I can use spaced repetition to learn quantum physics a bit faster. It doesn't make it a fait accompli. It's not like learning anatomy where basically if you study the deck, you'll be done. You need some more stuff, and I'm working on some of that stuff. Also, you need an incredible amount of very unusual knowledge that's largely tacit at the moment in order to use it in that way. That's part of what motivated recording that other video, to show some of that in action. The fact that the market isn't acting on this thing that it kind of can't really act on seems pretty appropriate.
Dwarkesh Patel
I think that's a good place to tie off that other collaboration and this project. This is really interesting. Thank you so much. This is many hours of just insights and lots of food for thought.
Andy Matuschak
Wonderful. Thank you.
Dwarkesh Patel
Yeah, things are coming on.
Outro
Hey everybody, I hope you enjoyed that episode. As always, the most helpful thing you can do is to share the podcast. Send it to people you think might enjoy it, put it on Twitter, your group chats, etc. It just splits the world. Appreciate your listening. I'll see you next time. Cheers.