Main Topic: The podcast episode discusses the advancements in autonomous vehicles, specifically Waymo's fully autonomous car technology. The episode features an in-depth conversation with Saswat Hanigrahi, Waymo's chief product officer, who shares his insights on the technology, its development, and its potential impact on society.
Key Points:
Participants:
Takeaways:
Self-driving cars have been on the horizon for quite some time. But, they might actually be here.
We got to ride around in one all over San Francisco with Waymo’s Chief Product Officer, Saswat Panigrahi.
We discussed so much, including the five levels of autonomy, the infamous LIDAR vs video debate, regulation, UX, and the role of AI in fine-tuning these models. But most importantly, if autonomy is here… now what?!
Topics Covered:
Resources:
Check out Waymo: https://waymo.com/
Check out Waymo's Youtube channel: https://www.youtube.com/@Waymo
Find Saswat on Twitter: https://twitter.com/saswat101
Check out a16z's 8-part series on the autonomous vehicle ecosystem: https://a16z.com/2018/02/03/autonomy-ecosystem-frank-chen-summit/
Stay Updated:
Find a16z on Twitter: https://twitter.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Subscribe on your favorite podcast app: https://a16z.simplecast.com/
Follow our host: https://twitter.com/stephsmithio
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Stay Updated:
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Podcast on Spotify
Listen to the a16z Podcast on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Host
We are in San Francisco. We're not taking you to some little test facility in a desert to show you that this car can drive. We are on the road. We are on the road very much. We are in one of the most vibrant cities on the planet. That's where we are in the journey. We're fully autonomous where we need to be. Self-driving is here.
I think if I was a human driving there, like, I don't know what would have happened.
Guest
You're asking me, does it get boring for you after working? I love it. I love seeing your excitement. And we are not the only ones who have noticed. That kid just noticing that we don't have a driver. Look at that! I love seeing so many people when they look inside. Look at that. Just like, what's going on there?
Host
Today, I got to ride fully autonomously with Waymo's chief product officer, Saswat Hanigrahi, who's actually been on this journey since 2016. We discuss so much, including the five levels of autonomy.
Fully autonomous. As you can see, nobody in the front seat. No expectation of a human to take over.
The infamous LIDAR versus video debate.
Guest
Saying you love LIDARs and hate cameras or vice versa is saying you love one wavelength versus the other wavelength, right? It's not a fundamental thing.
Host
Regulation, user experience, the role of AI in all this, but especially the question, if autonomy is truly here, now what?
As a reminder, the content here is for informational purposes only. Should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. For more details, please see A16Z.com slash disclosures.
All right, so before we jump in, if you are hearing this, you are listening to our RSS feed, which is great. We love that you're here. But for this episode, this episode is such a special one. I would really recommend you go watch on our YouTube channel. I honestly feel so lucky that I got to ride around in this self-driving car for an hour and we have so much good footage. We literally saw the full gamut of possibilities. We saw someone running a red light. We saw a group of bicyclists, literally like 20 or 30 bicyclists. We saw a construction zone. We even saw a two-year-old kid who saw that there was no driver in the front seat and you could just see the confusion on this kid's face. So really, it was an incredible ride. And if you watch on YouTube, you can see all the footage from inside the car, outside the car, and from the car itself. You can literally see how the software is interpreting the world around it and transforming that into this representation simulation on screen. It's honestly so cool.
I actually just hit my one year as the A16Z podcast host, and this by far was my highlight. This is so cool. I hope you all get to experience this one day. And again, if you want to go see what I'm talking about and not just hear about it, go to our YouTube channel at youtube.com slash at, that's the at sign A16Z, or just search A16Z on YouTube. Really, this is the one episode if you're going to go on over to our YouTube channel, this is it. All right, we will see you on the road.
Sound of car door closing
Wow. Very clean. So what do I do? Do I just... Oh yeah, the KB up there. Nice. Oh, I don't even know how to interact with it. All right. We'll just jump in.
Car Voice
Hey there, Katherine.
Host
Thank you. This is cool. Can we just click start?
Guest
Yeah, do you want to?
Host
Go ahead. Oh wow. This is wild. Happy Friday. This is cool. So you can play music.
Guest
Yeah. You can just ask it to pull over if you want. Yeah, anytime. At any time. And you can call support. And there was an audio cue to remind you in case folks get in and they, you know, into a minute think like, this is a...
Host
Yeah, this is kind of scary.
Guest
But it turned out actually that wasn't...
Host
Really? Folks just got eased into it pretty quick.
Guest
Somehow, the smoothness with which it drives builds some instant trust.
Host
Totally. So... I mean, so I get car sick pretty easily, but to your point, like, it doesn't feel jerky at all.
Guest
Right. It feels really smooth.
Host
But, I mean, I'm so curious, because you've been working in this space for so long. Does knowing kind of, like, how the sausage is made, does that make it any less magical? Like, when you got into this car for the first time and there was no driver sitting there, were you also like, oh my gosh, like, what's going on here?
Guest
Oh, absolutely. I'm totally like a kid on this one. So 2017 was probably the first time there was a person in the front there, but they didn't have actual control, so that was my first. And 2019 was the first time I was in a car with truly nobody on a public street, just driving around.
Host
Yeah.
Guest
And every single time, even if, of course, the year before, the month before, the day before, and the hour before, I was in anticipation and preparing for it, still being inside it, it's truly special.
Host
I mean, it's so cool to just see it navigating.
Guest
Yeah. The slight turns on the wheel.
Host
Something I have to ask about is, it's felt like this is coming for a while. Like, I almost feel like the future is here. Like, we're seeing this, we're sitting in a car with no driver. But a lot of people would kind of say, this has been promised every two years kind of thing. And it's like, we're finally at the two years in a way. And so maybe you could kind of map out where we are in that kind of arc, the five levels of autonomy, and also where we still have left to go.
Guest
Yeah, certainly. We are in level four right now. So fully autonomous, as you can see, nobody in the front seat, no expectation of a human to take over. And in level two and three, it's really, really crucial to communicate the expectations to the driver. Because it's very easy that during a normal situation, the driver feels, hmm, the car is kind of driving well, so I can pick up a book and start reading. No, that's serious. And there is an expectation to take over. So we are in that level four with a certain scope. So right now, if we were to begin heavily snowing in SF, which it has, believe it around last season, there was a little bit of snow, we wouldn't operate it. But level five is truly defined as anywhere, anytime.
Host
Right, I mean, and even the idea of autonomy, people kind of view in this binary way, right? Like the whole idea of level puts that into perspective, like level two is lane sensing, automatic braking. Is level three where basically what we're doing now, but there would be a human in the front? Or how would you define level three?
Guest
No, no, I would say a vast difference even between level three and level four, huge. It's almost like a difference between driving and flying, I would say. It's a massive difference. Because this concept still that within seconds you need to take over versus now we can have this conversation.
Host
Yeah, we're not doing anything.
Guest
That would be challenging, right? But the assurance level you need to get to for level four is just a universe different, I would say.
Host
And if we wanted to, let's just imagine we're rolling down a hill for some reason. It feels like the car is not stopping. Could I take this over if I wanted to?
Guest
So if you did, the car would say, hmm, I'm being interfered with. I'm a fully autonomous car. I'm not supposed to be interfered with in this manner. So it'll pull over, basically.
Host
Okay, got it. So we've talked about the kind of arc of innovation. You've also worked in this industry for a long time. Since 2016, is that correct?
Guest
Yeah, that's right.
Host
Okay, so tell me a little bit more about the barriers along the way to Level 4. What would you say? Was it the technology, the regulations, some combination? Are there major factors that have really delayed us from getting to this point where we're now at level four?
Guest
Yeah, yeah. One thing you mentioned, the innovation arc, I always reflect that on any journey, when you're trying to do something that has never been done before, there will always be ups and downs. Some challenges you foresaw, some you didn't. The key is, are you clear on how massive the price or the benefit to society is on the other side of it? Because then that makes it all worth it. So we were super clear from the get-go that a fully autonomous car that does not get drowsy, that we even have physical constraints, right? So even when you are alert, let's say you were looking for parking on this side of the street, your face would be turning towards that, constantly looking for that parking spot. So you wouldn't be able to see. You don't have your sensor.
Host
Yeah, yeah, exactly. You can't keep turning back and forth.
Guest
And so we're fundamentally convinced that a fully autonomous driver is going to be safer. So once we started that, yes, the largest challenge I would say was largely, I would call it technology, but I mean two different things. One is building the driver itself that can drive under these conditions and having the high grade of performance. But also measuring that is pretty hard. This smooth sort of early stage drive under very tight constraints is now relatively with today's technology, not terribly hard to build. But to be able to do that at the scale of 24-7, busy intersections at slow speed, but also high-speed intersections of Phoenix.
So in Phoenix, for example, the streets are wider, so you don't deal with these narrow situations. But the driving speed is 45. People are sometimes going 60 on that. That means you've got to see a lot further. So very different sets of challenges, very diverse ones. And the technology, it really required the full stack, right? We build the hardware, we build the software. Because if you built just the software and waited for somebody else to deliver the hardware, the speed of learning, the speed of iteration that was necessary to build something like this was so steep that it was not feasible.
So we had to build the lasers, the cameras, the radars, and the software on top, and the massive simulation infrastructure as well.
Host
I was going to say, there's so many moving parts. And actually, maybe let's talk about that technology. So this is my first time in a fully autonomous vehicle, but I've seen them around my area. They're driving around, I see no driver, but I also see the swirly thing on top. I see a bunch of different kind of appendages to the car. So maybe you could just break that down. Like, what is happening? How is all this technology coming together? And what are the bits and pieces that you've added onto the car that allow it to be autonomous?
Guest
So fundamentally, you can think of it like, is the car aware of what's happening around it? And then can it anticipate what the things around it are going to do?
Host
Yeah.
Guest
And then reasoning on what it should do. These are sort of the three components. In perceiving what's around us, think of the example we were just discussing, you were trying to look for a parking. So you're focused on that task. This car with those appendages, as you mentioned, can see three football fields away, 360 degree, and it's getting a snapshot multiple times a second.
Host
Okay.
Guest
And it's relying on a combination of the state of the art laser, camera, radars, all strategically positioned.
Host
Okay.
Guest
To give you an idea, lasers give you a very precise understanding of everything around you. It's the smallest detail. So if there was a child an inch out of this pole, it would be able to mark, oh, this is a demarcated child away from that pole. It would get to see that. But, you know, the cameras are needed to distinguish between the red light and the green light.
Host
Oh, yeah.
Guest
And the radars can almost see around corners, even when the laser and the camera are humanized can't, because they can sense objects coming in. And so we took an approach that we want to combine the best strengths of each of these modalities to create the best picture of what you can see around the world that you're just incredibly better than a human possibly could, both due to their attention span, the range, the fidelity, and the combination of these sensors coming together. So that's what we see. But then there's a harder challenge of anticipating what the person will do. Take a look at that pedestrian. They're standing pretty close to the crosswalk, right? But you don't know if they're going to move, right? Are they going to jump in or are they going to stay? Are they going to jaywalk or are they going to obey the light? This requires a deep, deep understanding and tremendous amount of machine learning. Each stage here, by the way, requires an insane amount of machine learning. And it's not the type of thing that you can just put on the road and say, oh, let's see if it makes a mistake.
Host
Yeah, we're not trying to tell you, is this a cat image or a dog image?
Guest
So, for example, for that pedestrian, in addition to seeing that they're there, acknowledging, which was this problem one, you've got to look at even their gait, their hand movement, their leg movement to anticipate, are they about to take motion? And if you're always conservative, which was, say, four years ago, we couldn't detect the pedestrians. We could totally detect them. It was this nuance of, are we being over-conservative, assuming they may jump in, hence let's not move, versus confidently moving forward.
Car slows down
So are we at our first stop of the multi-stop? Amazing.
Host
So yeah, that anticipation of what a car is going to do, what a pedestrian is going to do, what a child is going to do, because they can erratically jump through.
Guest
Yeah.
Host
What a motorcyclist is going to do. Are they going to lane split and speed and get around you? So all these motion models and understanding of how people behave, how this gentleman is walking, and what their gait and motion tells you about where they're going to go, that is the second. And third and final, what should the car do? This feeling that you had that it gently accelerated, but not harshly.
Guest
Yes.
Host
That takes into account not just look at this gentleman. He is almost, has his feet almost in the cross, but he's not intending to cross because he has a stop sign. A very conservative system would just come to a stop. But here we sort of asserted ourselves a little bit.
Guest
Yeah, we went for it.
Host
That's such a great point because... So I learned to drive in the last year. I was actually waiting for self-driving and it took a little too long. But as part of that...
Guest
Do you want to hit resume right now?
Host
Yeah, absolutely. Let's go.
Guest
You can see here, for example, we're showing... Oh, yeah, they're jaywalking.
Host
Yeah.
Guest
And you see, within the last minute, you saw an example in which we looked and we noticed that this pedestrian is not going to cross so we didn't come to an abrupt halt. We went through smoothly and later we just noticed that somebody jaywalking and we yielded to them. That delicate nuance is pretty hard. And like this lady, you actually don't know. She kind of looks like, oh, she went back. And you can see here we are tracking them. So you know if there was an anxious passenger, is this car seeing it? We give them the feedback that we are. We're seeing this pedestrian crossing right here on the screen. We tell them why we're slowing. Because what happens is also people zone out. They take their space. They speak to their kid if they're picking them up after soccer practice or take a phone call. And then they notice when the car is stopped. And they're like, hmm, why are we stopped? And here we try to give them the feedback why we stop. It's a stop sign now. Or we're yielding for this truck that passed by.
Host
So I want to get to the comfort, like how do you past autonomy evolve this product? Because it really is a product. But first, I feel like the rivalry in self-driving is LiDAR versus Tesla Vision or the video processing. How has Waymo thought about that decision of what some people pose it as expensive hardware, more simple software, because you have so much fidelity from LiDAR versus a bunch of cameras and just like a lot heavier processing. So what went into that decision? And also, how are you thinking about that moving forward? Like, is there a future where maybe actually you don't need all of the same sensing systems?
Guest
Yeah, great question. So personally, I have found the LiDAR and video debate almost takes like an ideological sense. You know, for hard problems, the innovation arc that we're talking about, the best approach is taking a first principles approach, right? Without neither love nor hate for a specific technology. That is a technology. You shouldn't get to that level. It's a tool. And LiDAR clearly has strengths that a camera doesn't. For example, at nighttime, even the best cameras will have some challenges. And camera clearly has strengths that the LiDAR doesn't. The red-green example that we mentioned. And similarly, radar has strengths that LiDARs and cameras don't. So saying you love LiDARs and hate cameras, or vice versa is saying you love one wavelength versus the other wavelength, right? It's not a fundamental thing.
Host
Okay.
Guest
But there is a practical question. So on the first principles, does the combination of these sensors position you better than individual? The answer is yes. We can show you that there are situations in which camera will be insufficient. Now the question is a practical economic one. Is your ability to bring this public good to a large number of individuals hindered by the fact that these things are expensive? Yes. And LiDARs, 20 years ago, if somebody told you that all these cars are going to have radars, you'd be like, no, radars are expensive. Guess what? Most cars have radars today. Cameras on cell phones were a novelty. Now, cell phones have better cameras than dedicated cameras of six years ago.
Host
Yep.
Guest
LiDAR is going through the same transformation.
Host
Okay.
Guest
Right now, the iPhones have LiDARs, right? The amount we have been able to cost down these LiDARs in the last two years is incredible.
Host
Okay, so you're not concerned about that.
Guest
And four or five years ago, we had that belief. Now we have that proof. Because hardware generations, there's multiple examples outside of Waymo to see something that began. Chips are a great example. Cameras are a great example. So it would be surprising if you had hardware that you were able to package, and then with focused effort, you weren't able to. So we had that belief, and now we have the proof.
Host
Right, and Waymo actually manufactures LiDAR, correct? And I feel like Waymo in a way has chosen to your point, like manufacture the hardware, work on its own software, simulation technology. Give me the thought process there in terms of, there's always this question in business, do I build, do I borrow, do I buy? And given that this is a capital intensive business, how do you think about that? Where should you outsource and where do you really need that fundamental technology yourself?
Guest
Great question. The first thought we had, let's build this all ourselves. That was not the first thought. But we said, okay, let's see what's the best out there, absolute best, even easing a little bit the cost requirements. Say we're willing to pay.
Host
Yeah, we'll pay for anything.
Guest
What's the best out there? And what we found is that the absolute best LiDAR out there, absolute best radar out there, was not optimized for the task of autonomous driving.
Host
Okay.
Guest
And hence, we had to build it. Each hardware generation, we do evaluate that. We try to see, okay, have radars evolved to the point that we could use something off the shelf? So that build versus buy is a pretty practical choice each single time.
Host
And as we look forward, the question becomes, where do you build a moat? Because now you're not the only company that has achieved some level of autonomy. And you could imagine, like, let's just say a future where we've just achieved level five in many places with many companies. How do you think about what differentiates? Is it really a data moat based on the amount of training that a company has been able to do? Is it owning the proprietary LiDAR that is just 10x cheaper than the competitor? I'm trying to think ahead in terms of where does value really accrue in that future?
Guest
So first, just thinking of the space we're operating in, we're talking about a space with trillion miles, right? So it's vast. Imagine when, let's say, the first cars were being built, folks said, hey, is there space for only one provider or two providers, right? When you're talking about a space of a trillion miles of today, and then you think about what is the potential value add when you have a driver that drives itself. Because partly if you think about the miles today, we talk about a truck driver shortage, for example. In fact, even those commercial miles are kind of stunted by the lack of availability of drivers.
So really, we're talking about a space that is all cars, trucks and all transportation when something is that vast. And the number of autonomy players today are much smaller than three years ago, if you look at it, because it is a pretty challenging problem still. I think the universe is different, but still a valid question. We do believe that there is a knowledge curve. So by having driven 20 million plus miles in testing, by having done billions of miles of simulation, we become aware of problem spaces that others may not have discovered yet. And that goes into hardware design and software design and simulation design.
Hardware design in particular has long lead times. So that becomes flywheel effect. Your question about how did you know you had to build your own laser? Because by that time we had already driven 10 million miles and we're like, we're going to need that.
Car brakes suddenly
That was a person that you saw there. That person basically ran a red light. It was red for them.
Host
And we can talk if you or I were driving. I was going to say, like, I think if I was a human driving there, like, I don't know what would have happened.
Guest
And you would definitely not have been able to carry a conversation when that was happening, right?
Host
No, I can barely, I can't even talk to someone next to me most of the time when I'm driving. But okay, so talking about the technology, I want to talk about safety next. But first, are there any important technological unlocks that you still see on the horizon that not just Waymo, but the industry of autonomy is still trying to solve. Is it really that cost curve or is there something else that is still in the way of us really rolling this out more broadly?
Guest
Look, we are definitely, the rate of innovation even just within Waymo, which I can speak to most confidently, is massive. I'll give you a concrete example. When we came from Phoenix to SF, we did have some work to do to adapt the assertiveness and the driving here is different from the Phoenix driving. But when we went to Los Angeles, the driver worked shockingly good from the get-go.
Host
Really?
Guest
Yeah, shockingly. And now there's a portion of Scottsdale, which is the northeastern part of our Phoenix territory. It's a much denser area, lots of restaurants, lots of shops. There we were able to go in like within two months. We just went there. We decided we're going to open up Scottsdale within two months. And the reason for that is we're truly the drivers generalizing very well.
Host
Okay.
Guest
And the concept is pretty intuitive. And this is advancement in AI that's enabling it. But think of a driver that's capable of this kind of tight traffic navigation.
Host
Yes.
Guest
Lots of pedestrians and cyclists, but low speed of travel. That's where we are right now.
Host
Yes.
Guest
Now imagine in Phoenix, 45 miles per hour, three-lane, four-lane streets, lots of oncoming traffic and being able to navigate that. Pretty much every good weather city is like a linear combination of those two things, right? So in Los Angeles...
Host
You kind of got the best of both worlds.
Guest
Exactly. So when you go to West Hollywood, you're much more like an SF style driving. Lots of pedestrians, cyclists, and so on. You go to the LA's more faster boulevard. It's a lot like Phoenix. So once you have solved these two, the AI is just much more generalizable. The second area I would say, you already mentioned cost down, is making the simulator a lot better. I'll give you an example. We have billions of miles of simulation in good weather. If you want to test how would we do in rain, imagine being able to simulate rain so that you can take all the learnings in good weather, all the tough situations you encountered, and now test yourself in rain. What if rain was a complicating factor on top of that? What if I add a cyclist into that tough situation more? What if all these combinatorial questions, being able to realistically simulate that, that's also a huge area.
Host
Yeah, I mean, it's a little foggy today, but I also wonder, you know, you talked about AI, and obviously this is running off of an algorithm that's been trained on all these miles. Is it one algorithm or let's say it is an extremely foggy day, it's a rainy day, you're in a new environment. Is it a different, slightly fine-tuned model based on different situations? Or is it all one aggregate that's just ingesting all of this information?
Guest
It's definitely many, many deep models. Some very general, extremely deep learning models and some specialized models to make them really good at some very hard tasks. Specific things. Like, for example, understanding pedestrians' intent is such a vast space. It's like understanding humans, right?
Host
I know, and we're pretty hard to understand. It takes us a lifetime to understand ourselves.
Guest
So understanding human behavior and motion, there could be specific models. There could be end-to-end models on just driving like a good citizen, polite to other riders. That can be a more end-to-end model. Being comfortable to riders' preferences, that can be a very end-to-end model as well. So it's a mix of this. And there's AI at every layer of the stack, from perceiving the world to predicting other people's behavior to the driving to the testing. So, for example, you asked about fog. So what we try to do is we both observe how other people drive in fog. We also try to reason about how well can we see in fog. So if this fog were to get a lot denser, the appropriate thing to do is I can't see that far. So I shouldn't be driving as fast as I would normally do.
Host
Yes.
Guest
So that kind of learning is built in into multiple layers of the stack as well. But also some general things that the AI surprises you. You asked about fine-tuning. One of the powerful things that deep models are telling us, as well as generative AI is telling us, is that you actually don't need to hand-tune every single thing. It learns. So that kind of learning we are doing. And one thing I will say, you asked about moat a little bit earlier. You know, the high-level concept of AI may be easy to understand, but really the breakthrough engineering that you sometimes need in AI is just having the raw infrastructure to intake all this data. The amount of data you have to learn to handle to build a really well-learned algorithm is pretty hard. And that's where Google's infrastructure that we have worked with is just immense. And the machine learning investments that Waymo and Google did 12 years ago, 13 years ago, is beginning to pay off in a manner that's pretty hard to just get there. So that's a moat as well.
Host
That's actually a great point because, again, we're in a space where you need to react in milliseconds, right? And so you need to not just be able to train this algorithm, but to interpret live and process that information live. Let's talk about safety, right? I mean, that is like the foundational piece of whether we can get these cars on the road.
Guest
Right, right.
Host
And something I'd love to hear from you on is how different parties interpret safety. Being on the road, you could say, just inherently isn't safe. You can get in a crash, you can die. Unfortunately, that happens every day. So how would you say between the regulators, the technologists like yourself who are building this and then the consumers, the people, the riders, how do each of them view this like concept of safety? How are you designing the product with that in mind?
Guest
There's the analytical and quantitative version of safety. You can say the nature of collisions you have, the probability of entering into a certain collision under certain circumstances, a lot of complications in there. And good news is many regulatory bodies do have their teams that study crashes and there are great databases of crashes and so on. There's the element of risky behavior. You may have gotten lucky that you didn't get into a collision, but you undertook risky behavior. So, for example, in that situation we were in before, we could see that those two individuals were going to jaywalk.
Host
Yes.
Guest
Now, you could have argued that, well, the car has the right of way. We were driving right. No, but that's risky behavior, right? It's preventative behavior. So you can't see that solely with the presence or absence of collisions. Were you a good driver? And third and finally, do you make the person feel safe?
So we could be braking hard anytime we sense a risk.
Host
Yeah, wouldn't feel very safe.
Guest
Exactly. You wouldn't feel very safe. And then the fourth layer I would add on top of that is just because you figured out how to drive smoothly, meeting the expectations of a rider, you shouldn't falsely promise the true analytical safety either. So, for example, designing an algorithm whereby you drive smooth on a street could be easy, but you shouldn't over promise that you can detect a pedestrian jumping out of a car. That takes real heavy engineering. A consumer may begin believing that just because it can drive smoothly, it can probably protect me from that. And I think being truthful about your capabilities is really, really important, and what Waymo has tried to do.
Host
Yeah, yeah. And you can see how tight this space is. Truck on the right, a car just went by, a pedestrian just crossed, and there somebody just going in. And noticing that this fire truck didn't have its sirens on, so it didn't try to rush out of the way.
Guest
Yeah, so a slight change into that situation which we can test in simulation. We would say, right now the politest thing to do is let that pedestrian pass and wait for this turn. But when the sirens are on, it's a different environment. So anyhow, that's another data point you need to take in. It's sound, it's not just visual.
Host
Oh yeah, that's the sensor we didn't discuss. There are microphones that can not only hear that there's a siren, but also point at where the siren is coming from.
Guest
Yeah, directionally.
Host
That's fascinating. What does the data say though? Can you kind of ground us in the number of crashes that humans engage with every single year, and where the technology is at relative to that?
Guest
Yeah, so both the methodology by which we evaluate our safety, which is a combination of many, many—
Host
This will be interesting. There's a bunch of— Yeah, yeah, it's a whole ocean of cyclists here. Look at that. Went around a double parked truck. And look at that. They're like, they don't know what to—
Guest
Yeah, look at all of them. We were telling you who we are waiting for. See, they just pointed inside. Look at that. We can detect all of them. We're giving you feedback that we can see all of them.
Host
Yeah, you can literally sense every single one.
Guest
Yeah. Oh, and them coming around the corner. Yeah, yeah. Oh. And we're turning because we can see that their handles are turning leftward, so we can understand that they're likely gonna go that way. So we don't have to come to a stop. So balance between making progress, because that's what a rider expects.
Host
Yeah, exactly. And ultra-conservative behavior that may or may not be warranted. To your earlier question, so what we did is, I'll give you just two examples from many safety methodologies we employ. In the East Valley of Phoenix, we have been operating for a while. There, one thing we did is we took every fatal crash that had occurred and re-simulated and showed that Waymo could avoid that.
Guest
Okay.
Host
That was one very specific data set. What we also did is we were the first company ever to cross one million fully autonomous miles. Miles like this one. Nobody in the front seat. So there's no debate about what the car could have done. It was real miles. In that, we published our full crash tax.
Guest
Okay.
Host
And there was not a single collision with injury in one million miles.
Guest
Really? Not a single one?
Host
Not a single one.
Guest
Okay.
Host
And only two of them would meet the standards of a reportability called CISS. And, yeah.
Guest
All right. You're here. Oh, we're here.
Host
Yeah, and look at how it stopped because it saw that they wanted to cross. I love seeing so many people when they look inside, look at that. Just like, what's going on there?
Guest
All right. Oh, and it's been telling us.
Host
Yeah.
Guest
Okay, so I'm going to click resume ride. And the reason, by the way, we're switching a little bit from safety to user experience design, the reason it told you cyclists approaching is what we know is even when the car is stopped, what happens is people open the door and the cyclists watching. I mean, I've seen videos of that unfortunately. So this element of safety, it's not just about when it's driving, but it's about caring about safety every second.
Host
I love the analogy you used about even a driver. We only have two eyes. They're right at the front of our face. If we're looking a certain direction, if we're focused on music or a podcast in the car, our senses are as much as we like to believe as humans. We're all special and we're the best drivers. We are variable in ourselves in terms of what we're focused on. And if your two kids in the back start screaming at each other or something.
Guest
By the way, I know you're relatively new to SF as well. I hope you're enjoying the view as well.
Host
I am, I know. It's so cool to see around the city. We're going by some iconic locations. The city, absolutely. I love to think about these second, third order effects, like people taking city tours in these cars. I mean, we've basically turned this into a recording studio, which you would never think of in the past. I can take confidential work calls. That was not a thing I could do before, right?
Guest
Oh, it's such a good point. I was looking through the reviews on the app and I saw this one. You get a sense of the user that wants to be in a car like this. And they called it basically Uber and Lyft for introverts. And there's just these little things that you don't think about because, again, we're fixated on safety and that is so foundational. But then once you have covered safety, once it becomes safe in the eyes of the regulators and the consumers, it opens up all these doors.
Host
Oh, absolutely. You know, we work with so many founders in nascent spaces, AI, Web3, space. I would say those three industries also have a fair amount of pushback, which autonomous vehicles do as well, right? And in many cases, rightfully so, especially autonomous vehicles. Like we're talking about people's lives on the line. And so have there been any learnings since you've been working in this space for a while where you're trying to meet this safety need, but you're also trying to push regulation and welcome this technology into different cities?
Guest
Yeah, yeah. It may sound like an idealistic answer, but I do believe that there are applications that have a fundamentally different use case or a promise, and then they're trying to make sure they're not harming the public or causing unintended consequences. The reason we exist is to make driving safer. So we have deep fundamental alignment with what sometimes the regulators are trying to achieve. Now, we may have different inputs, we may have different data, we may be approaching it from a different angle. But I believe that every conversation I have had with anybody who is in state, federal or local government, or even outside of regulators, just firemen, local law enforcement. We very quickly in that conversation, I begin to appreciate, our team begins to appreciate, and they begin to appreciate that we're trying to do the same thing here.
Host
Okay.
Guest
And that is a powerful baseline to begin constructive conversations out of. But if, for example, the goal was something else, and by the way, what about safety, right? That would be a very challenging conversation. We try to say this is what we're trying to do folks, this is what we have measured, this is the common ground.
Host
Yeah, and just transparently sharing the data.
Guest
Safety. Oh, is it collisions? All right, well in collisions we have published our accident reconstruction. We have done 20 million miles of testing, billions of miles of simulation, and we're telling you every single contact we have had in 1 million autonomous miles. By the way, this week we're about to cross 2 million. First company again. First company again, 24-7, including daytime, not filtering out any of the challenging situations, dense downtowns, 24-7, 2 million miles, more than 160 years of human driving worth of data. We'll just share it with the world. And then they can see for themselves that we're clearly a safer driver.
And if ever there was an event about which they asked us, we would transparently share with them that. So I think that gives a fundamentally good basis. And I genuinely believe that even the word pushback, right? It's almost like internal debates at Waymo. When we debate, how should we design this thing? We come at it from different angles. Somebody may see the user's expectation of smoothness of drive. Somebody may see more, what's that scooter's intent? You see how the scooter swung in from the right, kind of came in between a bus and themselves? What are their expectations? What's the expectation of a pedestrian if they were to jump out on this side?
We may approach the problem from different angles. But our core mission of safety is so deeply drilled into every Waymonaut, that we believe that every person who meets us will see that.
Host
Yeah, and I mean, one aspect I love that it's ingesting all this data from, like you said, almost 2 million miles now. And when you think about us as human drivers, like, I certainly have not driven 2 million miles. I'm not even really processing exactly what happens when a scooter is coming up on the right here. And if I've spent my whole life driving in SF, which is not the case for me, but some people that is. And then you drop them in Phoenix, they are new to that road, just like you training in a new city. And by the way, the two million thing that you mentioned, that's just the fully autonomous miles. We have billions in simulation.
Guest
But yeah, here there was a lot of experimentation on how much is the appropriate amount of detail. So the vehicle and the sensors are seeing a lot more detail than what's being shown here, right? We're seeing many, many points per square inch of detail here. So we tried to experiment with how much detail we put in here. And there are folks who, when in the early days we had a version here that would show a lot more detail. They would engage like this, right? In fact, they would look more inside than outside the window and keep crushing. Did it see that cone? Did it see that thing? And what we came to a balance with is we want to put people at ease and invite them to use this space. And it pops up. So when the vehicle stops, you will notice it will try to explain why it's stopping. Like, is it a stop sign or is it yielding to a pedestrian? Because we realized through lots of experimentation that that's when people want to take a look at the screen.
Host
Okay.
Guest
So for example, if you're by yourself, we want you to get lost in the beauty of San Francisco. If you are checking emails, it's all right. Go do that. And you will take your head off your phone or from viewing the Painted Ladies when you see, hmm, why are we stopped? And we will tell you, well, there are two pedestrians on the right. There is a car crossing on the left. There's a car parking in the front. We'll explain that to you. But then you can see here, who are we slowing down for? You see that little gentle highlight? Lots of design experimentation goes into that because we don't want to be in your face. We want to be gentle, soft. And you'll notice that in the night, this will go into dark mode because the ambient lighting is reduced. So we don't want to be too bright because if you want to just take a nap, who wants you to take a nap?
Host
That's a great point. And I'm curious if there's other learnings from, again, you've rolled this out. The number one thing is safety. But then from there, it's like, how do you create a great product that people want to engage with, that people want to come back to? So it's not just a novelty, right? Where you're like, oh, I sat in an autonomous vehicle where they want to use this for their commute every day or for their daily lives. So how do you think about that? Other than maybe the screen, are there other things?
Guest
Oh, yeah. Thousands, thousands. And by the way, speaking of mods, that's another powerful flywheel advantage, right? If you have served the first 10,000 humans who have been in a fully autonomous car, the feedback they give you and the time you have to incorporate that into your learning is a great positive flywheel approaching 100,000 fully autonomous rides in a month. So that feedback does help. I'll give you just a couple of examples. We could speak for hours just on that part. One example I'll tell you is imagine a residential street like this one that we're passing. Remember when we were getting into the car you were asking which side of the street should I be on?
Yeah. Now imagine you're just getting out of home going to work. Some riders you would imagine expect the vehicle to pull up right on their side of the street. Actually, it turns out to be largely incorrect because the way you would achieve that is let's say you're coming from this direction, you would make a U-turn and come back to them. But on narrow residential streets, folks were like, that was not necessary. It takes me a second to walk across the street. I could have entered on the other street. You didn't need to turn around. I got to get to work. But that same reasoning does not hold on a street like this one.
Host
Yes.
Guest
Because you're like, why are you making me cross two lanes of traffic? I was going to the coffee shop on the other side. Just drop me in front of the coffee shop. So you can see how it's not just a single rule that you code in. You don't say, hey, always park on the side that the pedestrian is on. No. It depends on the context. If you're on a busy street like that and somebody is going to a business, they would like to be dropped very likely in front of the business they're going to. Whereas if they are getting out of home, going to work, and it's a narrow residential street, just go to whichever side is closest. That's just a thing. Just that rule, training it, required speaking to many riders what they expect, learning that, and then being able to articulate that from a machine learning standpoint and overall rules-based standpoint. That's just one example, many more.
Host
Are there any other learnings about what makes people feel comfortable or want to come back? Like one thing that's coming up is like, I can see the driving wheel, right? And I can imagine, especially once L5 is hit, then we don't really need the steering wheel, right? You don't need the same car design because in the past, for 100 years, cars have been designed around the driver.
Guest
Right.
Host
And now we don't have a driver. So it introduces all these questions. But I'm curious. I know we haven't removed the steering wheel per se, but are there other dynamics that just from us growing up in cars designed a certain way that we expect certain things? And then are there other things where you're actually like, oh, no, we can start to get rid of some of this.
Guest
Yeah, yeah. Specifically about the steering wheel, it's blocked currently by regulations beyond a certain number of vehicles and so on. And our next generation vehicle that we did design with CVT and Geely is a pretty powerful platform taught with the rider in mind. So we spend months and years with designers on all teams trying to visualize that. I personally do believe that thinking of all the screen and software aspects. He was like waving the car. Did you see that?
Host
Yeah.
Guest
And that was beautiful, wasn't it? Like it's polite, but it's also responsive. And that, you know, today we spoke about everything from safety to artificial intelligence, to design, to user understanding. In that three seconds there, all of that got exercised, right?
Host
Right.
Guest
Because we were confident that there's enough gap, there's no imminent contact. We saw a collaborative fellow occupant of the road, not an adversarial one, because sometimes it could be the other kind.
Host
Have you seen that, by the way, where people will start yelling at?
Guest
No, I mean, yelling is expression of emotions. That's okay. The physical demonstration of turning in or like that example that we saw right now of somebody who ran a red light, if you recall. Those are the ones that are truly could result in danger, which is what the vehicle positions it. It thinks about many small details. Where is it positioning itself? How much gap is it leaving? What's its velocity so that it has the greatest optionality if somebody were to behave recklessly. So when we are crossing that sign, we're not assuming that everybody's gonna obey the light. We're trying to monitor, hmm, their light is red. Which means they should be slowing down. Why aren't they slowing down? That's an anomaly. Let's prepare for this anomaly. Let's protect. Let's be defensive against that anomaly.
Host
Something that comes to mind is just as a rider you wanting to feel confident in the car and seeing it be a little more assertive is actually really reassuring at points. When it makes sense, because to your point, if it's constantly stopping, if it's constantly pulling over, then I don't have confidence that this thing is going to know what to do in a kind of trivial age.
Guest
Yeah, and you have a busy life. You want to get where you're going, right? So, by the way, iconic place coming up.
Host
Oh, yeah. Are we at the Painted Ladies?
Guest
Yeah. I think I see them up there.
Host
Yeah, yeah.
Guest
By the way, that's the other thing you were asking early on about where are we in the journey. We are in San Francisco. We're not taking you to some little test facility in a desert to show you this content. We are on the road. We are on the road. You are in one of the most vibrant cities on the planet. And if you wanted to come by to Los Angeles, we would take you to the most important part of Los Angeles. And in Phoenix, we would take you to Scottsdale. And when you land at the airport, we would pick you up at the airport. That's where we are in the journey. We're fully autonomous where we need to be.
Host
I love that it tells you finding a spot to pull over. Look at that.
Guest
Right? And it sees all those little kids there. Do you see that?
Host
Yep.
Guest
And it's more cautious because it understands that kids can jump out more at a kid than an adult can. They're unpredictable.
Host
Yes.
Guest
Oh, and vehicle approaching. So that's what you were saying earlier about, so you know not to open your door.
Host
Exactly, exactly. And now you can continue. All right, so those are the Painted Ladies. Beautiful. But yeah, I love the point that we're just, we're fully on the road. We're not in the middle of nowhere practicing.
Guest
And you should see the eyes of that kid just noticing that we don't have a driver. Look at that. Like he's young enough where he can't even articulate, but he's like, this isn't, my pattern recognition is off here. And by the time he grows up, this will be the more common route.
Host
Well, I mean, you mentioned you're in SF, you're in Phoenix.
Guest
Yeah, Los Angeles.
Host
LA. How have you decided which markets to address first? Is it just a matter of what cities will welcome this technology? What goes into that calculus?
Guest
In the very, very early days, we tried to make sure that we're picking a city that challenged the system in many different directions. We were in development stage, so we tested in 20 cities just to make sure that from the very early days, we're building a generalizable driver. Not one that just works in one location, but generalizes. Double-parked truck, by the way. And so is it even sensing maybe the lights blinking? When, for example, somebody's on parking, we notice their noses starting to jet out. It's just so bizarre to see a wheel moving on its own like that. Like, it truly does look kind of fake in a way. But yeah, so you're trying to find cities that kind of test the system, push it forward.
Host
Yeah, and then what we found is that we have tested, for example, in Miami as well for the heavy rain. We have tested in Death Valley for extreme temperatures. We have tested in Tahoe for snow. So there was a testing phase in which we went to 20 cities just to make sure with enough of a diverse data set to be building off of. Look at that gentle because this car was coming, that tram was coming. Anyhow, it's a beauty to see every second ride out. It really is something that it's like I'm watching. You can see this pedestrian crossing, the car going, another one approaching, but green light. So turn right after the pedestrian went off. Anyway, sorry. You're asking me, does it get boring for you after working?
Guest
I love seeing your excitement because, I mean, this is my first time. So I really am, like, as they say, taking it all in. How many times do you think you've been in a... Like just the first year in 2019, I spent a ton of hours in it.
Host
Actually, were you scared at all when you were first testing it? Because now, I mean, I guess I've seen some of the data. I've seen these on the road. So there's a level of like, oh, I know these work. But when you were first getting into the vehicle, was there any fear, apprehension?
Guest
Not fear. The closest feeling I can describe is when you have prepared for an exam for six years. Whatever is the biggest exam you've given. The little feeling that you got that you have prepared as best as you possibly could for it. You left nothing on the table. But it is exam day. You're like, oh gosh, I hope I can show up.
Host
Right, right, right. Well, it's funny because a lot of people, they'll see the stats and they'll be like, that's the average human. But they think for whatever reason they outperform the average human on the road. But I'm the opposite. I'm like you where I'm like, I want this.
Guest
The fundamental attribution error, by the way, is that more than 50% of people think they're better than average. It's just not supposed to be feasible, right? So maybe on the city selection, just to close that out. So yeah, LA, 2 billion plus 10. A huge diversity of use cases, everything from commute trips all the way to sports events, to sightseeing tours that you were mentioning. I think San Francisco similarly, both SF and LA are two among the top five ride-hailing markets in the US and among the top in the world. Phoenix is the fastest growing city in the United States. Phoenix Airport is among the top 10 busiest airports in the world. So when we commit to a city for a launch, that's different than going to a city for data collection and testing.
Host
Something else that's coming to mind is, I wonder, we are in the early stages, we're only in a few cities. But as consumers do see these on the roads, like this is something that some people really will want to see in their cities. And so have you seen any shift in terms of regulators, maybe being a little against it to actually being like, this is a competitive advantage if my city offers this?
Guest
We definitely see that in the place where we have been the longest is Phoenix East Valley. And everybody from passengers to neighbors who even haven't taken a ride, they notice that it's a much more polite driver to law enforcement, all the way to city and mayoral level, all the way to state level. Absolutely.
Host
I love thinking about just what does this unlock? And there are a few industries that can unlock so much because people do spend so much time in cars. I think that was an amazing, beautiful thing we saw from remote work. It's just the second, third order effects. Like what do we have when we get that commute back? But then there still are people spending a lot of time on the roads. And so I'd love to hear from you, like, what are some of those impacts, the wider impacts? Maybe it's on the way the insurance industry works. Maybe it's on trucking. Maybe it's on city design. Now that we have data, like, that's another aspect. We now have data about how people really move around a city and interact and how things are designed, parking, right? So maybe pick and choose what excites you the most.
Guest
Pretty vast. Yeah, right? So I do believe it's pretty profound. And only some of those aspects we can see and some we will be shocked by what it'll do. Because again, like you said, not only is the car designed around a driver, life is designed around driving, right? Like look at how much parking space. You know, that house right there costs insanely high amount in dollar per square mile, as I'm sure as a new SF resident you're aware of. But think of the parking. So there are so many things in our cities that are designed around assumption of not only a human driver, but also of a highly underutilized, expensive asset just sitting there. All these vehicles just add up the cost of these vehicles and think of how much space in the city they may be consuming while not adding to the productivity of the city. Because at this instant, I'm sure each of these vehicles added to the mobility and freedom of individuals. That's great. But at this instant, they're not utilized. They're not adding value, neither to their rider, they're giving a promise that when the person who has gone in for an eight-hour workday when they come out it'll still be there.
Host
I heard this quote the other day. It was like we dedicate in cities more space to sleeping cars than sleeping humans, which is kind of crazy when you think about it.
Guest
And first and foremost, there are people who are traveling today that we believe over time the roads will get safer. And that in itself, I know I'm saying safety so many times, but that truly is central, right? 1.35 million people are killed on the streets. How is that acceptable, right? So that's first and foremost for all kinds of, for society at large. It's a pandemic almost, right? And this is an antidote to that. And then there are classes of individuals for whom this freedom is not available. Above 65, many folks can't drive anymore or drive while taking risk or would have the freedom of mobility. There are folks with visual impairments.
Host
Yes.
Guest
Then vulnerable populations being able to take jobs that they otherwise wouldn't have. That nighttime use case that we said. Giving economic opportunity. Which they simply would not be comfortable. Or you shouldn't ask somebody to own a car before they can take their entry level job. Like that's a challenging economic catch-22, right? So there's that. Then the third is yes, how much pollution idling cars and city centers are causing. How much city real estate is being lost to idle assets. So yeah, layers and layers upon that. And yes, that's just in passenger vehicles. Then you consider in-city delivery. Then you consider long-haul trucking where already tremendous amount of economic opportunity in the United States is being lost to lack of drivers. And by the way, when we do have drivers, we have mandatory brakes because fatigue is a real thing.
Host
We recently moved. I drove from near San Diego up to San Francisco and we happened to do it at night. And just the number of truck drivers that we saw on the route, I just, I knew this already, but seeing it, I was like, something feels wrong here. And I just imagine a future where some people might not like this, but that's automated. That sounds beautiful to me.
Guest
And there's a beautiful transition point as well in the sense that truck driving could become a local job, which would be powerful in many, many different ways in the sense that. Oh, I see it since the pylon. And you see that?
Host
Oh my gosh, that's incredible. Because see, I was wondering actually if it could tell if that's a human. So they can tell that those are humans.
Guest
Yeah. But then these are pylons. And it can sense every single one. And there was like a roughly written keep right and understood going there.
Host
Yeah, because you could imagine how it might think to go left.
Guest
Yeah, yeah, yeah. We think there's enough space there. So I think construction zones, by the way, are interesting change to an otherwise structured world, right? Because no construction zone is the same, right?
Host
Yeah, exactly. Each one is unique and the degree of intelligence required to figure that out is pretty substantial as well because it's suddenly you're breaking the prior structure of the world.
Guest
Yeah, so I mean you mentioned how these vehicles can actually make our world safer. I mean maybe I'm getting a little ahead of myself but do you imagine a future where actually right now it's like most humans drive. Where actually it becomes illegal or the minority of people who are able to drive because we get the technology so far along where it's just, again, it's a no-brainer for us to have the tech drive us around instead of vice versa.
Host
The mission we have is being able to provide this option, safe and easy, for people and things to move around. I think it's good to have the choice.
Guest
All right, so maybe to close things off, I'd love to hear, you've been working in this space for a while. What gets you excited?
Host
Seeing the riders the first time, the fifth time, and what they say about us, that gets me excited.
Guest
Amazing. Well, thank you so much. This was an excellent first ride. I'm so excited to do this more in San Francisco. I love the music too.
Host
They're welcome. By the way, we had lots of music as well. We were having a conversation, so I didn't show you. But yeah, a lot of effort has gone into this one as well.
Guest
Yeah, and I can't wait. I imagine this being personalized as well in the future. The climate control, the music, hooking up to Spotify from your app.
Host
We did both of these screens as well because there would be cases in which somebody would be in front synchronizing all of that. But for another time.
Guest
Well, this is great. And I guess we just hop out.
Host
Yeah. We're here. This is great. I guess I shouldn't. It knows I'm here.
Guest
I'm imagining myself on the screen.
Host
Yes. The little dot. So how did you find the ride?
Guest
I really enjoyed it. I mean, I feel like... It's funny because I only really noticed the lack of driver for maybe two minutes. And then, as you saw, I was so involved in the conversation, you don't even notice there's no driver there. And that's what we want. What's that quote where it's like, any sufficiently advanced technology is indistinguishable from magic? And it really does feel like magic. Wow, we're here. We're here. We just did it.
Host
Well, I'm glad to have been there when you had your first experience.
Guest
Yeah. Well, thank you so much. This was so cool.
Host
Yeah. All right.
The episode transitions to a closing monologue from the Host.
If you made it to the end here, I just wanted to say thank you. This episode in particular was actually really special to me. I got my first learner's permit when I was 16 in Canada. And I actually waited until I was 29 to get my driver's license because, quite frankly, it scared me. And I was waiting for self-driving.
And it's finally, at least in some places like San Francisco, it has arrived. And I am so excited. I am so happy that I could share this with you.
And if you are just as excited as I am to see how this whole thing unfolds, let us know in the comments what you are most excited about, how you think autonomous vehicles might most reshape society. Because there really are so many implications, whether it's public infrastructure, energy, finance, shopping. I'm so interested to see how this technology finally comes into play.
All right. On that note, thank you again so much for joining me. I'll see you next time and we'll see you on the road. Thank you.