--:-- Host (Sonal): What can today's startups learn from Amazon's so-called, quote, "invention machine" and the culture of growth and innovation there that led to several successful and diverse lines of products? As it scaled, how did Amazon define and adapt its culture and practices to promote innovation even as it got more complex and large?
--:-- In this episode from February 2021, early Amazon execs Colin Bryar and Bill Carr go beyond the well-known artifacts of Amazon innovation – things like the memo or the press release before the product has even been built – and share the leadership principles, decision-making practices, and operational processes that helped Amazon continue to innovate, invent new products, and learn from its mistakes, all while it scaled.
--:-- It's all based on their book, *Working Backwards: Insights, Stories, and Secrets from Inside Amazon*, drawing upon the 27 years of combined experience they had of being in the room where it happened at Amazon.
--:-- Hi, everyone. Welcome to the a16z podcast. I'm Sonal, and today I have another one of our special exclusive first looks at a new book episode. It is both a very timely and evergreen topic because a new book coming out this week is titled *Working Backwards: Insights, Stories, and Secrets from Inside Amazon*.
--:-- In it, authors Colin Bryar and Bill Carr, who between them have a combined 27 years of experience in the company – where Bill was vice president of digital media, founded and led Amazon Music, Amazon Video, Amazon Studios for a decade, and where Colin started out in the software group, was a technical vice president, and then notably was one of Jeff Bezos' earliest shadows, a legendary program there. Fun fact: the first shadow before that, I believe, was Andy Jassy, president and CEO of Amazon Web Services, and now to be CEO of all of Amazon.
--:-- The book actually shares the origin story of AWS, among other businesses there, which we touch on briefly. Though, as a reminder, none of the following is investment advice. Be sure to see a16z.com/disclosures for more information.
--:-- But in any case, our focus today is really on what is the Amazon way and can other companies really adopt certain best practices too? In fact, as fast-growing companies establish and find their way, how do they define and scale their culture, processes, and more?
--:-- We actually spend most of the episode digging in detail into two operational practices in particular: the infamous memos instead of PowerPoints, and working backwards from a press release and FAQs, given the presence of those two topics in tech folklore and lots of misunderstandings as well. So I actually probe for the origin stories, the specific details of how they do and don't work, and other nuances so organizations of all kinds can take what they want or need.
--:-- Finally, we also debate within this episode the trade-offs of lean and minimum versus maximum viable products, whether the emphasis is on the wrong letter there, whether product managers have a role in companies organized like this, and more topics throughout.
--:-- But we start by very briefly discussing the foundational principles.
--:-- Host (Sonal): The first question I want to start with, especially since I hate the "why'd you write this book" question – Colin, Bill, honestly, there's a tendency for folks telling these stories, these kinds of narratives, to do a sort of hindsight is 20/20, not accounting for attrition data or the failure cases as well. So part of me is skeptical that startups can do what Amazon did. What's most noticeable is that Amazon had not just one or two product lines, but literally entirely different yet successful lines of businesses all under one roof. So what drives that? And can startups really relate to the Amazon story then?
--:-- Guest (Colin Bryar): Well, one thing is that the businesses you mentioned, they are substantially different. They require different expertise. They're different customer sets. AWS is B2B. There's streaming video. There's the e-commerce business. But they all have one thing in common, and that's what we talk about in the book. We call it the invention machine, which was the process and principles that Amazon used to develop these businesses. The ones that we talk about started off as ideas on a whiteboard or emails, which many people were skeptical we should even do. Some of them grew into household names, but they all started off very small and with just one or two people.
--:-- Guest (Bill Carr): I would first start by going back in time a little bit and place you where Jeff Bezos was where all entrepreneurs start out. At the beginning, Jeff worked out of a simple office building with a handful of employees, and he would be hands-on for everything. The very first customer support emails, Jeff wrote or co-wrote. He could review the work and think about all the policies. He could direct the team and set the tone and the pace.
--:-- Well, that works just fine when you're an early stage company and there's fewer than 20 of you and you can all do a stand-up each morning. But guess what? That completely breaks once you start to grow like a weed and you've found product market fit. Suddenly you look around and realize you've got a team of 130, 200, 500. You realize that there's all kinds of decisions and meetings happening around you. You can't be part of every decision.
--:-- So to me, what's most remarkable and notable is that Jeff sought to figure out ways to actually inject his lens of thinking into all those meetings and then sought to create a bunch of processes that would reinforce the way he would think about the work or do the work itself.
--:-- Host (Sonal): This idea of operationalizing the invention machine, as you guys are describing it. Some of those principles and processes at a high level to quickly summarize to set context for our listeners. They range from customer obsession, ownership, invent and simplify, are right a lot for leaders, learn and be curious, hire and develop the best, insist on the highest standards, think big, bias for action, frugality, earn trust, dive deep, have backbone, disagree and commit, deliver results – which I love and we don't have to unpack all of those because that will take all day and it's the whole reason your book exists.
--:-- I do want to ask about one before we go into some of the other practices, which is around leading. The one that really intrigued me was number four: "Are right a lot." You basically write that leaders are right a lot. They have strong judgment and good instincts. They seek diverse perspectives and work to disconfirm their beliefs. I love this because I always think to myself, yeah, damn it, leaders should be right all the time. Their instincts should be damn good. Tell me more about that one because that one made me chuckle a bit.
--:-- Guest (Bill Carr): Yeah, this is – and just to be clear, of course, we did not write that. That is Amazon's words that Jeff and the S-team, his direct reports, painstakingly reviewed to come up with that exact language to define that principle, as with all 14 others. They sweated over the details of every word, every sentence.
--:-- This principle, "are right a lot," in some ways it's very straightforward. As you move up in early stage CEO, as they grow and progress, they need to be more in the mode of delegating important work and auditing work. But their most important job, frankly, is to make decisions. Many decisions will come to you, whether it's presented with a spreadsheet, a document, PowerPoint – there's all kinds of data that will be presented to you with those decisions. But there are very, very few problems where the data gives you the answer. At the end of the day, you're going to have to use your judgment.
--:-- "Are right a lot" refers to, number one, when those leaders are confronted with those decisions, that more often than not, they pick the right door. But the second part of the definition refers to, more importantly, how those leaders make decisions. A lot of people think that leadership is about their very strong opinion, arguing their opinion and winning that argument. What great leaders do actually is they can stake out a position, but they are willing to update – I mean, change their mind on what is the right answer based on new information.
--:-- Even Steve Jobs talked about this at Apple, where some product that they launched ended up being a mistake. There was one of his reports who had been telling him all along, "We shouldn't do this, we shouldn't do this, we shouldn't do this." And then after it launched and it failed, he came back to that person and said, "Why didn't you talk me out of this?" And the guy said, "Wait a minute, I tried to talk you out of it. What are you talking about?" And he said, "Well, you didn't do a good enough job because you didn't present the evidence to me in a persuasive enough way to make me realize why your point of view was so important."
--:-- So it's thinking about it and framing it that way – about bringing forward the right data and the right information. And then it's also the job of the leader to solicit that to make the best decision.
--:-- Host (Sonal): Well, that is a perfect segue to a question I'm dying to ask you guys about. So many companies, their culture is like the mission statement and the values that you shove in a drawer. People spend so much time talking about the words and the precision of what they want those principles or values to be, but not actually how to operationalize it. In that vein, because your book really does outline how to operationalize that through the processes and practices that you guys share – one of the ones that comes up all the time in Silicon Valley folklore is the infamous "no PowerPoints, write a memo."
--:-- So let's tackle this one first. Because what you just shared, Bill, about "convince me, share the evidence in a persuasive way" – the point is to be effective and be heard, you have to do that well. So what is your best advice about how leaders and people in the group can share that information?
--:-- Guest (Colin Bryar): Amazon started experimenting with writing narratives in 2004. It was a result of weekly meetings – four hours every week with the S-team, Jeff's direct reports – where two to three teams would come in and give either updates on their business, a decision that needed to be made, or investigating new areas to go into. The business was growing fast and we realized that we're not making the right types of decisions. Some of the meetings would go over. We never really accomplished what we wanted to.
--:-- I was Jeff's TA at the time. We had been looking at other ways to conduct meetings and we're big fans of Edward Tufte, who's Professor Emeritus at Yale. He came to Amazon to speak a couple of times. After one particularly painful meeting, it was later on in the week, toward the end of the day, Jeff said, "Let's stop doing PowerPoints at these S-team meetings. It's the wrong tool for what we're trying to do. Let's switch over to narratives," which are really just now a six-page memo.
--:-- One thing that is a little bit misunderstood is these ideas don't come out fully formed. They started with four-page memos. It ended up that six pages was about the right length for an hour meeting. But we did it because narratives convey about 10 times as much information. The pixel density is about seven to nine times the pixel density. People read faster than people talk. You can have multi-causal arguments in a narrative much better than a hierarchical PowerPoint. We realized we just needed a better way to analyze complex situations and make better decisions. So we just experimented with this. And the first ones were not good.
--:-- Host (Sonal): Just to quote you guys, because this actually made me literally laugh out loud. You guys write: "The first few narratives were laughably poor when evaluated by today's standards. Some teams ignored the length limit, blah, blah, blah." I was like, yes, people are not – I mean, we may be wired to be good storytellers, but writing is actually a hard skill. I do worry that it becomes a bit performative for the best writers, not just the best presenters. One of my colleagues when I used to work at Xerox PARC used to call this "pissing on paper" – this idea of how dogs piss around their territory. There's also a real-time component where people who are performing in real time, like leaving comments in the middle of the meeting.
--:-- So a common misconception is: well, now it's just the best writers instead of the best presenters that kind of win out in narratives.
--:-- Guest (Bill Carr): We haven't found that to be the case. It's really the best thinkers who write the best narratives. Some of the best narratives I've ever read are by people whose first language is not English.
--:-- Guest (Colin Bryar): In fact, the best narratives, many of them are written by software development engineers who may not have even focused on their writing skills. A good narrative is a very data-based and fact-based document. Writing good narratives is way harder than making a good PowerPoint. I think honestly, a lot of companies don't do this just because it's hard.
--:-- Guest (Bill Carr): When Colin and I have brought this to other companies, what tends to happen mostly is that the author will vomit out about 25 pages of just sort of raw data at you. Getting that person to then shape it and narrow it down to six pages of well-thought-out narrative is really hard.
--:-- Guest (Colin Bryar): Yeah. And by the way, if you spend 10 hours a day reading detailed narratives, I got to tell you, it can be mentally exhausting. But when people muse like, "How does Amazon do it? How's it possible that they can effectively manage such diverse businesses?" – one of my number one arguments is that they use narratives to conduct meetings, not PowerPoint.
--:-- A former colleague of mine, Derek Anderson, who is now the chief financial officer at Snap, I think he made the observation once that Amazon has a narrative information multiplier. It's a strategic advantage that the company has over other companies because their executives are like seven to eight times better informed about what's happening in their company. They're able to give super granular specific feedback to those teams. It allowed the S-team to stay connected at a much deeper level. As Amazon started to move into more and more businesses, the spanning control of the S-team didn't grow by 100x. It's still a relatively small team. And they are just as involved when it was a small company as they are now when it's a large company.
--:-- Guest (Bill Carr): The other thing I'd add about narratives is it does remove a lot of bias from the process. You can have a charismatic speaker with a so-so or even a bad idea that convinces your company to do something that you should not do.
--:-- Guest (Colin Bryar): The converse is also true. If you have a shy engineer with a great idea but it's a boring presentation, it's one way in which Amazon removes bias to make better decisions. The idea floats to the top rather than how good of a talker the person is.
--:-- Guest (Bill Carr): We could go on about this forever. But the other part is, it actually is a great way to get your team more engaged from top to bottom, especially in a COVID time. The way that these meetings work is you share the document – whether it's with G Suite or with Word – then everyone can use the comment feature. It doesn't matter whether the person making the comment is a C-level person or a fresh out of college individual contributor. All those comments get seen and heard. Then when you get to the discussion phase, all those people actually can understand why we're making the decisions we're making.
--:-- When you do a PowerPoint, you have to wait to get to the punchline. While you're waiting, you're not sure what we're actually doing in this meeting. With a narrative, you just take all that in. So then you can have high quality discussion versus that interruption and disjointed conversation you have with a PowerPoint. And if you miss the meeting, you can just read that document. There are so many ways in which the narrative method is superior to PowerPoint that, as you can tell, we could never go back.
--:-- Host (Sonal): Okay, so how does one do meetings then based on these memos? One of the things you guys said is that sometimes it's shocking because the first 20 minutes of a meeting would be silent. That made me chuckle because one of the co-founders of Roam, the note-taking app, was sharing that sometimes they have entire meetings that are silent because they're just sharing notes with each other in Roam.
--:-- Guest (Colin Bryar): Amazon meetings with narratives, they are strange to the uninitiated. There's the chit chat before a meeting – when they were in person, or online people are starting to come in. But then there's silence for about 20 minutes. During those 20 minutes, people are just focused on reading. There's no sound. They're entering questions. Sometimes the presenting team can answer the quick questions and write in the comments. So for that 20 minutes, there's a huge transfer of information that you can't see. Then the rest of the 40 minutes is really just high quality Q&A and discussion on the questions that have already been entered or that come up. The six-page narrative is a forcing function to where you can cover that amount of information in a one hour meeting.
--:-- Host (Sonal): Tell me a little bit more about what needed to change in the specifics of meetings in terms of running them. Because the premise of this conversation is not everyone starts out as Amazon and that startups can do these things. I'm trying to really tease apart: are we just replicating the meeting dynamic in the memo? And then the same thing happens anyway. People would begin reading in the meeting. But then how would the decisions happen? What happens next?
--:-- Guest (Bill Carr): There are a variety of different kinds of documents you'd review in a meeting. It could be an annual operating plan. It could be a monthly business review. It could be a PR/FAQ about a new product. So the conversation – literally what you do once people stop reading – is you just go page by page. Alternatively, if it's a smaller group, let's say there's just 10 or 15, you could literally just go around the room and say, "Okay, Sonal, you're first. What questions and comments do you have on the document?" So you would get the feedback, questions, and comments from all participants.
--:-- The document itself would present some sort of specific proposal. It's asking for some budget amount. It's asking, "Are we going to launch this new product?" At the end of the meeting, you are tying it up and wrapping it up and saying either, "No, we agree that what you've written in this document, that plan works, approved, go ahead," or you debate and discuss it.
--:-- One of the things you might do at the end of the document is make a clear section after you've presented all the facts to say, "What are the decisions we need to make today?" And then list those out. Or "What are the important parts of this plan where we need your feedback and input? We're not sure whether we should go down path A or path B." A good document will clarify what are the decisions we're going to make.
--:-- Guest (Colin Bryar): The one thing I would add is that a lot of first timers to narratives, right after people read, they say, "Let me walk you through the narrative." And you stop that right away. You've just had that 20 minutes of high fidelity bandwidth. Why dumb it down with a two minute verbal walkthrough of the document? You want to get to that feedback loop as quickly as possible, just talking about the document.
--:-- Host (Sonal): We didn't actually really say what goes in the memo. You observed that the memos can vary in form and format by function and purpose. But can you at least describe what goes in the memo specifically? Even the ingredients would help.
--:-- Guest (Bill Carr): Amazon has different types of memos for different purposes. If it's some monthly or quarterly or annual review, there's the typical: what were the key wins? What did we do wrong? What can we do better? What are the key initiatives coming up for the next year?
--:-- You can have appendices too. The appendices are supplemental data that everyone's not required to read in the narrative meeting. But if you do need to go jump to it to answer a question, you can. It would include tables with a summary of our financial results from the prior year, a table with a summary of our plan for the next year.
--:-- They also work very well with design, which you may not think about at first. If you're going over mock-ups – either UI mock-ups for an app or physical prototypes of some hardware or process that you're going to build – having a narrative actually helps set the stage to say, "Before we take a look at anything, here's what we're trying to accomplish with the user experience. Here are our goals. Here are the challenges that we're trying to solve."
--:-- I've been at mock-up meetings where everyone thinks they're a UI expert. But if you don't have that beforehand and they comment on three or four things where we should move this over here... But you don't do that if everyone's on the same page with reading a short narrative beforehand.
--:-- Guest (Colin Bryar): Another thing that can be in a narrative that's really powerful, especially for something that's going on on an iterative basis, as if you're refining an idea over time, are tenets. Those are the design criteria that we are not going to compromise on or that we're going to fall back on when we have to make tough decisions. That's front and center, usually up in the beginning of the document. "Before we go over Amazon's pricing policy, I want to remind you, here are the four tenets that we're following to make the following decisions." To get to those tenets, it was difficult. It took several meetings just to say, "These are the correct set."
--:-- That's a good caching mechanism because if you're a manager at a company that uses narratives, you're going to be context switching and reviewing multiple ones every day. Sometimes we only meet with the team once a quarter and you want to be able to very quickly get up to speed.
--:-- Guest (Bill Carr): There's another important technique where you can actually add at the end of the document a fact section for frequently asked questions. So you're actually anticipating the kinds of questions that the audience are going to ask you. A talented senior leadership team is going to ask you hard probing questions about things like, "Well, you say in your plan that you're going to go do X, but it seems to me that there's an important hurdle here. You're going to need this important partnership or you have this important dependency. What gives you confidence that you're going to actually solve that problem?" And you would answer it. Not only does that help reflect whether the author and the team have good mastery of the issues, but it also helps speed up the discussion and the decision making.
--:-- A lot of my work leading Prime Video was making very expensive multi-year agreements with motion picture studios to acquire their films and TV shows for the service. These were big numbers, a lot of money involved. If we're going to go spend that much money, we're going to go write up a deal memo that describes, "Okay, we think we should acquire these films and TV shows. Here's what the package works. Here are the detailed metrics that show why we think this is a good investment" – or why not in some cases. Sometimes we would review it and say, "We've looked at this deal and we're not going to do it and why."
--:-- Again, just because it's a Word document doesn't mean you can't put in a chart, a graph, or in the appendix, put an Excel table. All those things can still be in there. But then the narrative needs to clearly state with a clear beginning, middle, and end: "Here's what we're proposing to do or not proposing to do and why."
--:-- Host (Sonal): I think the thing I found most compelling is how much this mimics how writers think, obviously. One of the things that I found when I first came to a non-media company and was working at one previously was, "Oh my God, I have to make so many freaking PowerPoints." And I hate it because it's a muscle that's not ideal for storytelling as much as people think it is. It really isn't.
--:-- But the next thing – what I love about what you're saying there is that the FAQs is actually the equivalent of doing the inoculation technique, which is what really good op-ed editors will build in. That's not just because you're trying to inoculate the counterparty to the arguments they're going to make. What you're really saying there – and I want to pull this thread – is you now can have a better, deeper discussion because you've laid to rest all the common things that you can literally get out of the way in a memo in 20 minutes.
--:-- Guest (Bill Carr): In a good document, that all comes out.
--:-- Host (Sonal): I want to actually go back to the tenets for a quick second and then pick up on a thread that you brought up. Colin, you described it as a caching mechanism for the leaders where they have to do a lot of context switching. So they get to revisit that cache when they have to get back into that context. But I was thinking about it from the point of view of the group in the room, not just the leader. How it really sets the shared context for the room to have the most collaborative mindset possible while disagreeing within that framework.
--:-- In your book, you write that tenets give the reader an anchor point from which to evaluate the rest. But here's the best part: if the tenant itself is in dispute, it's easier to address that directly rather than take on all the logical steps that derive from that position. Sometimes you guys spent meetings after meetings just debating to get the tenets right, which I think is really great because it allows you to actually separate the thing that the tenet is and then what flows from that.
--:-- Guest (Colin Bryar): Yes. I spent 15 years at Amazon and I've since gone on. One of the things that shocked me was that certain topics get recycled for debate constantly, right?
--:-- Host (Sonal): Yes, exactly.
--:-- Guest (Colin Bryar): I was like, "Whoa, we don't do that. We didn't do that back at Amazon." Why is that? One is the use of a tenet – all that debate and discussion can land on a fundamental issue about what this company should or should not do. If you don't come to a common agreement on it, then you will constantly waste so much time re-litigating these issues.
--:-- The second reason is that what the narrative process forces you to do is by actually putting down on a piece of paper not only a tenet, but then the specific plan. Then you set the date for the meeting and everyone reads it and then you decide. I realized that a lot of those things get re-litigated. So it was one of the ways which Amazon was very effective as a management company.
--:-- Host (Sonal): On that note, does it then serve an archiving function for new hires and onboarding? That you scale that tacit knowledge that's been made explicit? How does that work?
--:-- Guest (Colin Bryar): Great point. The other thing I also saw is that whenever you hired someone new in, within a matter of a week, they would want to go re-litigate. They too would trip across, "Well, why don't we do blank?" or "Why don't we do..." And it's like, "Oh my gosh." And you can just say, "Look, here's the tenet. Here's the document. Take a look at this and then come back and talk to me." Then when you do decide to re-litigate, it's an actual intent versus an accidental every new hire repeating, recycling the conversation.
--:-- Host (Sonal): Okay. So the other big Silicon Valley folklore when people talk about best practices from Amazon is this idea of working backwards from the press release. Now, I know that you guys talk about working backwards well beyond – it's a reason, it's the title of your book, obviously. But I really want to probe specifically into the mechanisms of what that means. Because like, how does that happen? Is it just, "Oh, I write a press release for this thing I want to do. It's a product, it's an idea, it's a service." I really would love to hear what, where, how, and why. And also I'd love to hear the origin stories if you have any specifics there as well.
--:-- Guest (Bill Carr): At its heart, the working backwards process is really starting from the customer perspective. Everything you do works backwards from that. The PR/FAQ – the press release and FAQ – that is the tool that Amazon uses to achieve the perspective of starting from the customer.
--:-- It's different than how a lot of companies develop ideas and products. A lot of companies use a skills-forward approach: "What are we good at? What are our core competencies? What are our competitors doing? How do we nudge into this adjacent market? How much market share can we get?" When I was in business school and taught to think about how you expand and grow, as Colin already described, you create a SWOT analysis – strengths, weaknesses, opportunities, threats. But there's no C. There was no customer.
--:-- Amazon has put in deliberate mechanisms to make sure that the customer is front and center from the very first iteration of an idea. So if anyone raises their hand and says, "I have a great idea," the first thing that the manager or the person in the group will say is, "That's great. Why don't you go write a working backwards document?" What that is, it's two things: a press release and then a frequently asked questions document. The press release has to clearly explain to the customer what it is you're building, what's the problem you're trying to solve for the customer, and why does it make their life better?
--:-- Guest (Bill Carr): It can be something very small or it can be moving into a brand new industry. It's a fractal process, which is great. Another thing is these PR/FAQ documents are an iterative process. First of all, most of the ideas that go through it don't make it out on the other side. And second, it takes several iterations and feedback to refine before the project gets greenlit.
--:-- In fact, the origin story of how we got to the PR/FAQ — at least a part of it — both Colin and I were present for this. In 2004, I landed in a new role working as one of the founding members of the digital media team at Amazon. For perspective, at the end of 2003, 77% of all of Amazon's worldwide revenue was media products. All physical media products: books, CDs, DVDs, VHS tapes. And the writing was on the wall that this business wasn't going to last. There were already a couple million iPods sold. Millions of people were using Napster for file sharing. It was pretty clear that now that we have the internet, it's just a matter of time before people consume their media digitally. But we didn't know how.
--:-- So what I did was pull out my bag of tricks from business school. I marched into meetings with Jeff with projections for how big the ebook business would be over the next 10 years, our projected market share, what the pro forma P&L looked like, the kind of deals we'd make with publishers, the competitive landscape. I was so proud of all this work. And he looks up at me and says, "Bill, where are the mock-ups?" And I didn't have any mock-ups at that point. I was thinking, "Oh, here's the projection. I'll get started and we'll launch a better ebook store." By "where are the mock-ups," what he meant was, "This is all super interesting — or not really, actually — but what's more interesting is what is going to be the customer experience?"
--:-- As we started to debate different ideas, whether in digital music or ebooks, the discussion was about why we would bother building a me-too versus what Apple already had with the iPod and iTunes. What's in it for the customer to have just another knockoff? What can we build that actually creates real value for customers? Something new we should invent on their behalf. We tried mock-ups for a little while, but frankly, there were so many questions and so many details we hadn't thought out. Jeff one day said, "Okay, I've got a better idea. Why don't everyone in this room write up the idea for what they think we should go build in digital? We'll write those things up, come back in a few days, read them, and go from there." This was before we'd done narratives. We hadn't figured out this PR/FAQ concept yet. But once we did that, everything changed.
--:-- Suddenly we were reading one document describing a puck that would sit on your countertop. You would talk to the puck and could order groceries from it. I described a document about what we might do in digital music. Another described an early version of what Kindle might become. But some of these documents were eight pages long. We needed to get this more pithy and clear. Jeff said, "I know, let's write the press release for each one of these ideas instead." He said we should write the press release because normally that comes last. Normally, the engineering group and the product group come up with the idea, and at the very end, when it's time to sell it, they chuck it over the wall to the marketing team and say, "Figure out how to go sell this thing." But what if by the time that thing got to the marketing team, they said, "Well, the thing you built — for that to really work, we actually need that to have a price point of $99. But you've built it in such a way that it's costing us to manufacture each one." So we wouldn't be able to sell too many. In other words, if you had known up front that you needed to hit a price point of less than $100 to make this product go, you would have designed the whole thing very differently and understood the constraints.
--:-- There might be vaporware concepts. There might be concepts you don't know how you're going to solve. There might be business model problems. But then in the FAQ, that defines what are the hard problems we're going to have to go tackle to make this exciting product a reality. We switched to that method and it was halting progress. Had we not really taken that approach, we would not have ended up with what turned out to be a breakthrough product: the Kindle.
--:-- Host (Sonal): That's fantastic. A couple of questions to just quickly probe on a few nuances. First of all, I know what you're really saying is flip the perspective from inward out to outward in, which I think is really interesting because that's something I constantly think about with content — oriented in the value to the reader. But what would you say on the flip side to the crowd that often says that part of the problem with "working backwards" is that you're not really inventing what they don't know they want? How does one write a press release for the startups out there thinking, "We're creating a new category. This is not something that exists"? Tell me more about how you might address that crowd with this PR/FAQ approach.
--:-- Guest (Bill Carr): I would submit that, in fact, that's what this process is actually designed for. In 2004, when we started on digital media, the way ebooks were — you could only read them on your PC. There were no offline readers or tablets or devices in those days. They were priced way too high, like the same price as the hardcover book. Ebooks had existed for four or five years before the Kindle launched in 2007, but it was a tiny, tiny business. It was tiny because no one had imagined what new thing needed to be built to make ebooks work.
--:-- We defined what would be the ideal reading device. Everything from it would be always connected to the internet — which, by the way, devices weren't that way back in 2007. That you wouldn't have to create some separate account or link your account to some mobile carrier. That when you got it out of the box, it already knew who you were. So if you had bought a bunch of ebooks online, they were magically already uploaded onto your device. All of those kinds of things would be described in the FAQ section to say, "Yes, here's hard problem one, and here's how we're going to solve this problem. Or we don't know how we're going to solve this problem yet, but here's our path for how we're going to go work on that."
--:-- Guest (Colin Bryar): I would just say there are two very real use cases where Amazon used the working backwards process to create something from scratch. With AWS, cloud computing didn't exist. As we were working through the Kindle issues — you talk about context switching — an hour later, we'd go to another conference room, and we'd be with Andy Jassy, who's the CEO of Web Services, and we'd be reviewing Word documents about what would eventually become cloud computing. It took us about two years to come up with what we were actually trying to create. The first two were centered around storage and compute eventually.
--:-- Where the press release really helped crystallize everyone's thinking is we came up with saying that we want people in a college dorm room to have the same access as an Amazon developer to world-class infrastructure. That was a powerful metaphor about what we were creating and what this infrastructure really needed to be. Starting from the customer experience, we had many, many small teams throughout the company just screaming at Jeff and the infrastructure team: "I built my service. It's taken me too long to deploy it and get it out and ready for customers." That's where it started off as provisioning, but it morphed through this working backwards process to compute. And then in terms of storage, there were a whole bunch of different types of storage. It narrowed down to Simple Storage Service as the very first thing we would build out from that.
--:-- Host (Sonal): I love that example because it really emphasizes that when you start with the PR/FAQ, you're essentially starting with the differentiation because you're really thinking in terms of the value. There are a million options for people to pick from, so that's when you go from provision to compute because you're thinking about the customer being this kid in the dorm room having access to...
--:-- We often talk about the power of this movement. Mark Andreessen in his original 2011 "Why Software Is Eating the World" op-ed points to the power of AWS being huge in bringing that about. We even have this op-ed about why every company is a fintech company. We call it the "AWS moment" in fintech. It's quite amazing the impact AWS has had on the industry. There's no question about that. But that precise point of starting with the PR/FAQ to take what could have just been "here's some storage and how to provision the services you need" to "here is how to create an entirely new business" — that's a whole different game.
--:-- The other thing that's really important to note is where the PR/FAQ process is misunderstood or in conflict with the startup community today — sort of a lean and agile approach. I want to hear about this. Love it. Have some fights, Bill.
--:-- Guest (Bill Carr): All right. So let me tell you what the problem is with the lean and agile approach. I love Eric Ries, for the record, but he himself is the first to say — and he's actually said it on this podcast — that sometimes people get a little culty and follow the letters of the rule instead of the principles of it. I actually personally am a big believer, having worked at Xerox PARC, in the maximum viable product sometimes instead of the minimum viable product.
--:-- Host (Sonal): Yeah, it's really actually the V part. The problem is that people focus on the M, the minimum, and they don't focus on the viable. So what is the definition of viable?
--:-- Guest (Bill Carr): What would you say it is?
--:-- Host (Sonal): To me, the definition is: if I go build this thing, I have created a — insert size of business here — $100 million, billion dollar business. I have enabled, if I'm right, something really big. But instead I see: "Okay, I've got a sprint. I've got a couple weeks. What can I get done in a couple weeks?" Or "What can I launch quickly to test and learn?" That creates completely different constraints. If your whole dev team thinks of their whole roadmap and breaks it into these little chunks of "how do I iterate quickly, test and learn," then you're going to launch a lot of small things where the actual size of the viable business on the other side might be sized in the $1 million or $2 million range. The actual potential good outcome is very, very small.
--:-- Now, there are plenty of places where this approach is totally applicable — search, where you're testing and learning with changes to the algorithm or a new AI model. But if you're starting from scratch and trying to create a new business, the problem with the MVP approach where the viable part isn't really thought out is that people haven't thought through what this could really become and why it might not work. In many cases, if they'd instead spent more time up front in the planning process, they could have thought much bigger about what the customer really needs and created significant value, rather than these little incremental changes that don't really move the needle at all. People don't do press releases for incremental releases. They do press releases for big advancements.
--:-- Guest (Colin Bryar): That's actually an excellent point. If you parachute into a new company and you look at their product roadmap and there's not one thing on there you'd write a press release about, then you've got a problem.
--:-- Guest (Bill Carr): To me, viable means you read the press release of what you're trying to build and you want to buy or use the product. If it's not something that you want to buy or use, it's not viable.
--:-- Host (Sonal): I would also push back on the definition of viable. Colin, when you say that obviously the customer is going to want to use it — but to me, viable is not just that it's something a customer would use, because I think there's a lot of dumb things customers would use, quite frankly. I heard it more as the enabling conditions to really make something bigger, and also to address a deeper, more underlying opportunity sometimes instead of the surface opportunity. Not to get all jingo-ish on this, but I think of Clayton Christensen's jobs-to-be-done framework. It's about what is the underlying job to be done that is being fulfilled for this person. So that's how I heard the viable: what does it take to make this happen, whether minimum or maximum? And not only thinking about the product, but all the enabling conditions to get to addressing the deeper opportunity.
--:-- Guest (Colin Bryar): Another way I've seen the MVP process misused — because it can be useful — is that the process itself becomes the goal. Because "I'm supposed to launch every two weeks" or "I have an MVP." So this MVP card goes to the front of the line and gets whatever resources it wants because it's an MVP. Don't let the process itself become the end goal. And don't let the MVP get in the way of understanding your customers, because some people actually forget the customer problems they're trying to solve — which is the whole point of doing an MVP in the first place.
--:-- Guest (Bill Carr): The whole point is to be able to get that feedback quickly. The whole reason that idea came about was that you don't want to have that very long delay in the feedback loop. But if you've watered down the feature set so much just to conform with the process, it's a shallow, pathetic version of what you're actually trying to test. You learn nothing because you never really gave it your best shot.
--:-- Host (Sonal): What's really interesting is you actually outlined the total addressable market, or TAM, in your book. I found that to be very interesting, especially in connection to the discussion about how to think about the biggest possible market for this product working backwards.
--:-- Guest (Colin Bryar): We spilled a fair amount of ink on the TAM part in the book because people with less experience sort of get this part wrong. You need to not only think about how many people have this problem, but how big is the need? For how many consumers is this problem really big enough that they're willing to spend money to do something about it? How much would they be willing to spend? And how many of these consumers actually have the characteristics or capabilities to make use of this product? A lot of people don't really step through that clearly.
--:-- The thing is that you want to ask all of the hard questions up front. As we talked about, this is an iterative process. In the review meetings, if there are a couple of questions you don't know the answer to, you just append those to the document and say, "Great, let's come back in two weeks, however long it's going to take."
--:-- Guest (Bill Carr): The FAQ can basically be broken up into two different components. One is an external FAQ explaining how does this product work, how much does it cost, why should I use this service versus what's out there on the market, or why do I need to change my behavior. And then the internal FAQs: what are the things we need to go organize and solve in order to make this product or feature a reality? And then, like you touched on, how big is this opportunity? What is the total addressable market? Then you can size up the opportunity, and that'll tell you whether it's worth doing or how much to invest in it. One of the things you ask when you go through the working backwards process is: is this big enough to be worth doing? It may be a good idea, but it just may not have the impact and scope on your customers or the organization to make a difference, to be worth devoting resources to.
--:-- When we were figuring out in digital media and AWS what we were going to go build, if you parachuted into either one of those teams and said, "Hey guys, how's it going?" we said, "Oh my God, we're so frustrated because we just want to go build this thing and get it out there. But Jeff is insisting that we go through this PR/FAQ process and figure this all out in advance." I was among the people who found this frustrating. It was only in hindsight that I realized how smart Jeff was to slow us down. We co-opted this Marine Scout sniper saying: "Slow is smooth and smooth is fast." Or "Measure twice, cut once." Sometimes Jeff would refer to himself as the Chief Slowdown Officer when teams were sort of itchy to go pull the trigger and build and launch something.
--:-- What I came to appreciate in retrospect is that to build, to actually create value, you really have to take the time and think big. That's what the PR/FAQ process does. If everyone can go write PR/FAQs and you've got 30 or 40 different ones to review, the best ideas are going to bubble to the top pretty clearly based on this process. Whereas the MVP lean process really is just about cranking stuff out rather than first filtering, thinking through, and refining what to do.
--:-- Guest (Colin Bryar): People confuse speed and activity with effectiveness.
--:-- Host (Sonal): Yeah, exactly. What I find so fascinating about that is coming back to where we started: how the 14 leadership principles balance each other. You described Jeff in this context, and a lot of these processes as ways to slow things down up front. But then you also have principle number 10, or whatever number it was, for "Bias for Action" at the same time. You think of this as a whole system. It resonates with me with Ben Horowitz's book on culture, where he describes the value system of the samurai warriors — Bushido — where there'd be something incredibly kind and generous and then something incredibly vengeful in the same framework. It's very interesting how those motions are kind of opposite, but yet in balance.
--:-- Guest (Colin Bryar): They do work together. We started off the conversation about the "I write a lot" principle. Well, the one right below that is "Learn and Be Curious." And the one right above it on the list is "Invent and Simplify." You need to do both of those in order to be right a lot.
--:-- One thing I would add: a lot of people say, "Amazon either has so much money or so much time that they can afford to do all these things." Long-term thinking doesn't necessarily mean it takes longer to get to your end goal. Amazon built a $100 billion business faster than any company. AWS got to $10 billion in revenue from zero faster than Amazon the retail business did. So sometimes to slow down is to move fast. Even with small organizations, you're more than likely to get to where you want to be quicker if you take some of these steps. Typically, what you're doing is off-path, distracting activities that are taking away from your bottleneck resources — which are typically software engineering resources at a lot of organizations.
--:-- Host (Sonal): Yes, exactly. Those resources are meant to build up long-term value.
--:-- Guest (Bill Carr): Just to pick on the AWS thing — it's so remarkable. Think about that for a minute. What was it, Colin? 18 months? 24 months? How many months from go before the team even wrote the first line of code?
--:-- Guest (Colin Bryar): It was at least 18 months. And I did have software engineers after some of the meetings come to me and say, "Hey, can you remind Jeff that our job is to write software code, not documents?" They were itching to go. It took a while to figure out what we should build. That time was very well spent writing Word documents versus writing C code. Not only that, it's an empirical fact that it was well spent because the company set the record for the fastest company to a revenue milestone by spending the first 18 to 24 months planning.
--:-- Guest (Bill Carr): And you talk about distractions. We did not know at that time what other people were doing. Web services were no secret. There were other companies who, by their own right, should have gotten there first. We didn't know if someone would come out with a set of developer APIs at that point in time.
--:-- Host (Sonal): I know. That scares me thinking about that, actually. Almost two years planning. Think how scared they were. But Jeff had the fortitude to say, "It's not ready yet. This is not what we want to build. It's not viable." Talk about what makes Amazon special. You can read these principles. Sticking to them is sometimes quite challenging, either when things are going really, really well, or when they're not going well, or when there's uncertainty. These are not just posters on the wall. They're woven into the DNA of everyone who's been at Amazon for any length of time.
--:-- I'm just struck at all the parallels and all the things you guys are saying, because essentially every business today is a creative business. I'm not going to make a comparison between podcasts and AWS, but I will say that I talk to a lot of podcasters about how to make their podcast stand out, differentiate, and how to do editorial strategy. One of the things I constantly say is: if you can't be first or leading, then you have to be very differentiated. It's really interesting when you talk about the bottleneck resources. I think of things in terms of opportunity costs and return on energy, or what I call ROE. Similarly, you have to pick which things you're going to do for the greatest possible hits, or you'll never punch above your weight or be heard above the noise.
--:-- We had Jeff Lawson from Twilio on the podcast recently. He said, "We need to give them problems, not just specs and things to work on." When you describe that they spent 18 months writing planning documents, that is treating software developers as creatives. It also creates alignment for the teams who will go out and build, because sometimes it requires more than one team. If they're not aligned on what the problem is, they're going to solve different problems and those components aren't going to fit.
--:-- I want to switch into talking about some specific advice for startups.
--:-- Guest (Colin Bryar): One tool you can use for startups — and I've actually done this at other companies — if you're trying to figure out what to build, you can write competing press releases with different takes on the problem. Then it becomes pretty apparent when you've got two or three or four different approaches that we really need solution A, or I'm going to take the first part of this solution and combine it with a great idea that came up in the second one. This is actually what we want to build. It's a lightweight way to crystallize your thinking and also get alignment.
--:-- Guest (Bill Carr): Yes. Think about how inexpensive it is to write a one-page press release versus how expensive it is to build mock-ups. To Colin's point, when you read them all as a group, the best ones are going to be clear.
--:-- Host (Sonal): This is where I just can't get over how creative the book is in terms of its application to all kinds of creative fields, companies big and small. The memo strategy, the PR/FAQ, and then this idea of being able to write competing press releases and harness the wisdom of the group — it's actually about the power of narrative for really helping instantiate things that you know when you know. I think a lot about how tools change our thinking and vice versa. You guys are essentially describing these practices for how that plays out in organizations, especially as they scale.
--:-- On this note, you have a section in your book with the subhead "Better Coordination Was the Wrong Answer." That's one of the common myths that people have as they scale: "We need to coordinate better." In fact, it creates layers and layers of crap to deal with. People create entirely new roles, like dedicated chiefs of staff, just to create stitching and seaming between teams. It's just the most ridiculous thing I've ever seen and heard. I want to hear more from you guys, especially because a lot of listeners are talking about what happens when you scale and grow very quickly. You guys really did see this phase at Amazon.
--:-- Guest (Colin Bryar): This is a case where Amazon faced the same question that a lot of growing organizations do: "We're adding more people. It seems like it's taking longer to get things done." But Amazon came up with a different answer. Some companies would say, "Let's build coordination tools. Let's collaborate better." Jeff said, "I'd love to eliminate coordination altogether." In practice, you can't eliminate it completely. So you break up into loosely coupled teams. This was a hard, hard problem to solve because it required changing the way we built Amazon technology-wise — decoupling it into what's now services-based architecture. That was hard to do back in 2000, 2001, especially as you're growing super fast. But there's also the organizational component about how decisions get made. It was a multi-year effort, to be quite honest. But if you're going to grow 10X and 100X, you're not going to spend any time building — you're going to spend all of your time coordinating. So nip it in the bud. Jeff wanted Amazon to be a place where builders can build.
--:-- The ironic thing is that we started off with a process that was conventionally accepted — traditional wisdom — called NPI, New Project Initiatives. It was horrible. It was another example of the process becoming a thing. You had to come up with your new project initiative, write it all up, project the financials behind it — most of which were totally wrong guesses, like almost all pro forma P&Ls are — and then put it in front of a big committee and try to take it upstairs. Not only was this process a massive waste of time, but then you'd have these frustrating business reviews with a team saying, "We really need to go build X, Y, and Z." And Jeff and this team saying, "Yeah, I agree. Go build X, Y, and Z." But they say, "Well, I can't because I need Browse to do this, and I need the team that works on the checkout, the order pipeline. And they already have four other projects they're working on. So I'm just sitting here in line and I can't go build it." You've de-empowered your teams. Senior management becomes like referees between teams, deciding who's going to do what. Frankly, it's not much fun.
--:-- This was a huge breakthrough for the company: breaking down the code into APIs, and then breaking down the teams into single-threaded focus teams.
--:-- Guest (Bill Carr): There are some roles that don't actually fit too well in this type of paradigm. For instance, a chief product officer doesn't really fit in this role. If you ostensibly are responsible for making every product decision in the company — from what's going on in the warehouse to how the apps should be built to how we can decrease delivery time — there's no one person who can be obsessed with all of those details. That role typically doesn't fit as you separate into these small, separable, single-threaded teams. The fallacy is that as your company grows, the chief product officer isn't really doing that anyway, because they can't get that much high-quality information to make all of those important decisions. You want the teams closest to the customers making those types of decisions anyway.
--:-- Host (Sonal): I call that bare metal decision-making — who's the closest to the metal of the thing. That's what I think is super valuable about what you just said. You're essentially describing these small units as every team being its own mini GM, every mini unit is the leader, and every product you're so close to the core in that decision-making framework.
--:-- Guest (Colin Bryar): That's right. The idea is for each one to be like a self-contained unit. In search, that's all engineers. But in some other business, like my digital video business, it was a combination of engineering and marketing people and people working with studios. But you have all the resources you need. You don't have these dependencies that basically slow you down.
--:-- During this timeframe where we were making the transition, we were using narratives. One of the questions we actually required was for the teams to put in: "What things that are not under your control do you wish you had under your control? And how are you going to organize APIs so that can happen?"
--:-- Guest (Colin Bryar): Again, this is work that's below the tip of the iceberg of where Amazon really put a lot of effort into how can you still grow and be as nimble and agile as we were when we started out. And it's not easy, but it only gets harder. So you may as well start now.
--:-- Guest (Bill Carr): Yeah, and don't take away from this that Amazon was some nirvana world where you never had to worry about coordination with other teams. It was just that we were so hard to minimize the degree to which we did.
--:-- Host (Sonal): I'm very fascinated by how organizations evolve in general to be effective. And when you think of any large company, it becomes a complex system. And you're essentially describing how modular self-contained units can thrive in these complex systems. It's a lot like evolution. And these pieces all fit together because then we have to look back to like, oh, the reason that Amazon could do that is because those teams wrote narratives in PRFAQs. So they made it abundantly clear. If Jeff wanted to see what is this team doing? Oh, here you go. So you read this document and in two hours, you know exactly what he was doing. He makes him the chief ecosystem officer, essentially, not just the chief slowdown officer.
--:-- Okay. So Colin, so a question I have for you, it's really unique that you were able to shadow Jeff Bezos and be his shadow. And I don't want to make this about the glorification of Jeff Bezos. I'm so not interested in that because I think there's just plenty of narratives out there. What I am interested in, however, is this question of what it takes for a CEO and a leader like that to evolve. And what you saw in shadowing him on that front, and then also the act of shadowing and that itself as a mechanism for learning mentorship, apprenticeship, if you will. I'm very fascinated by this whole thing.
--:-- Guest (Colin Bryar): Sure. So, my role as Jeff Shadow was primarily two parts. One was just to make him more effective CEO on a daily basis, making sure that the right issues are surfaced, that the people coming into the meetings, we cover the right topics and have enough information to make the right decisions on, and afterwards that it gets followed up on. So you're kind of bookending the day.
--:-- But then the second and more important part, the longer term part, was it was a training role. And the way Jeff put it is, I want us to be able to model each other and know how we would think in different situations.
--:-- One of the enlightening things that I realized during my time with Jeff is what he chose to work on during the two years I was his shadow or technical advisor. And it wasn't the biggest businesses at Amazon. About half of the time was spent on what would become AWS and digital. And those businesses had revenue of effectively zero for those two years. And he told me more than once, you want your top leaders and your most impactful people working on your biggest opportunities. And that may be different from what your biggest current set of activities are.
--:-- How Jeff changed during that time and just to become a better CEO. One thing he said is he learned how to become a better operator, becoming more operationally efficient. Fortunately, that is a teachable and a learnable skill. And Jeff had some great people at Amazon, like Jeff Wilke, who's a great operator. He's the CEO of the consumer business. And now in his own right, he is a great operator and insists on the highest standards as one of Amazon's leadership principles. He holds people accountable, himself included, by the way.
--:-- And I would say that I also learned that sticking to these core principles is harder than it sounds. But the time when you need them most is the times when it's easiest to ignore them. So for instance, Amazon Prime, he made the call that, hey, we're going to go launch this shipping project in the holiday season. And it was not very popular at the time. But when he explained his thinking, that customers were basically giving us a B- on our three to five day shipping, even though he had just gotten reasonably competent and spent a couple hundred million dollars doing that, Jeff looked at it long-term and said, well, we're becoming a smaller and smaller share of the overall e-commerce industry. So we've got to make the change now. And so it wasn't that Jeff had this insight that comes once in a generation for Amazon Prime. He just stuck with the leadership principles and took them to their logical conclusion when it was tough to do in the face of the holiday season of what our quarterly results were.
--:-- Host (Sonal): By the way, for the listeners, because we don't obviously have time to go into the whole book, but half of the book, the first half, is about the principles and these practices and mechanisms. But the other half is actually showing through these very detailed case studies and examples that you both participated in or witnessed or oversaw firsthand in this invention machine at work. And that includes the Kindle story, the AWS story, Prime video, and what you just mentioned, Colin, which is Prime.
--:-- And what's really interesting in that prime section is that not only that Jeff had the wherewithal to push through, but what's really interesting in the prime program story is that it's all the iterations involved to actually get it to what it is today. Because there was like a 1.0 and then the loyalty program it later became. So I think that's a really great part of the book for those that want to learn more.
--:-- And Bill, anything to add there on what you saw in the evolution?
--:-- Guest (Bill Carr): I do. In fact, I remember Jeff at least more than once talking about the way a leader needs to evolve as the company grows. And I think about this frequently, which is he said, at the beginning, the leader needs to really focus on what. Then they really need to focus on how. And then eventually their focus really just becomes who.
--:-- Wow. What is our business? What's our product? What are we building? What are the details of that, right? How is what processes? How do we do the work? What is the filter, the lens through which we make decisions? And then who, of course, is who are my leaders? How am I making sure I assemble the right team? And in doing so, how do I delegate responsibility to those people so that they can carry out those details?
--:-- That is a classic and challenging transition for any entrepreneur owner who starts off being in control of every detail and then has to slowly let go of those details. And then they have to figure out how they put in place the right mechanisms to be able to properly audit the work of the teams. And that's really what we try to describe in the book is actually this management science, frankly, that Jeff and the team developed to really solve this problem.
--:-- Host (Sonal): *Working Backwards: Insights, Stories, and Secrets from Inside Amazon* by Colin Bryar and Bill Carr. Thank you so much, you guys, for joining the a16z podcast.
--:-- Guest (Colin Bryar): Thank you.
--:-- Guest (Bill Carr): Thanks, Sonal.
