--:-- Host: I think speed matters tremendously. I think the latency is a big deal. It was able to talk to us. Yeah. It was able to hear, potentially, with the gears, and now it can see us. There's always the risk of this big company will build this thing that the small company wants to build and crush it. My guess is that 7 billion people in the world of life come at me, that understands them, listens to them, encourages them, all that. It's a universal need. This was a big week in the world of AI, with both OpenAI and Google dropping updates. So big that we decided to break things down in a new format with our consumer partners, Brian Kim and Justine Moore. And I actually tagged in ChatGPT to help me summarize a few of his own updates.
--:-- [Host addresses ChatGPT]
--:-- Host: Hey, ChatGPT, I heard you had an update this week.
--:-- ChatGPT: Hey, yes, I did get an update recently. The latest update focused on improving performance, accuracy, and enhancing conversational abilities. It's designed to be faster and more efficient than previous versions.
--:-- Host: More efficient. Like if I send some audio, how quickly in milliseconds will I get a response?
--:-- ChatGPT: As fast as 232 milliseconds in some cases. On average, you can expect a response time of around 320 milliseconds.
--:-- Host: Wow, that's pretty cool. I think I also saw on Twitter that people were live translating. I've always wanted to learn Chinese. Could you actually translate what I just said into Chinese?
--:-- ChatGPT: Of course, here's your sentence translated into Chinese.
--:-- Host: All right, ChatGPT, that's all for now. Congrats on your new update.
--:-- ChatGPT: No problem at all.
--:-- [Host transitions back to the main discussion]
--:-- Host: Now, if you didn't catch that, voice was a huge part of this update. But today, we discuss why not all audio is the same and also why several nuances like speed and personality really matter. Now, of course, Google fast-followed with its own announcements, like AI video model Vio, Gemini Live, which is an Android-native multimodal assistant, new Gemini models like Flash and Nano tailored to specific use cases, and of course, Gemini everywhere all at once. So in Gmail, Google Sheets, even Google Search. Now, clearly, these two companies are taking two different approaches, so we'll talk about that too, and continue the conversation around AI hardware for all of this new AI software.
--:-- [Host introduces guests]
--:-- Host: Now, make sure to stay tuned next week where we will return with Brian and Justine's twin, Olivia Moore, to dive even deeper into the applications that people are building through our Gen AI 100 list. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com slash disclosures. All right, so big week, huh? OpenAI, Google, they both dropped a couple announcements. So, I mean, everyone hears these announcements and they kind of hear their own version of it. What did you guys hear? What do you feel like was big?
--:-- Brian Kim: For OpenAI, I think part of it was GPT-4 being available for free and getting rid of a lot of the usage limits, the desktop app being accessible to a bunch more people. And then I think the really exciting thing for a lot of folks who are building in this space or using OpenAI's models is more multimodality. So being able to intake more like real time video, see a person, comment on it. And then the output, obviously, a voice speaking, singing, that sort of thing was pretty huge.
--:-- Justine Moore: I think the three things that I took away that was super interesting is one on the business side of things. All of a sudden, like things are a lot cheaper, a lot faster. I think that's obviously a great thing for the ecosystem. Second thing is that when you hear a demo, it sort of enables you as a founder to think about, OK, if this is APIable, like if I can access this, what can I build because it's just so thought-provoking. It's a great example of what's possible. And I think they put on a great live demo, actually, of the product.
--:-- Brian Kim: I think the third thing I took away is, and this is probably the one that goes viral, where I think the voice, the voice itself, like how they actually decided on which voice to use, which tonality, which personality, the degree of flurriness. And that was very interesting. That's like my takeaway of, oh, like they actually really thought through how to get the tech community super excited about this. and it's like a one, two, three punch.
--:-- Host: So let's talk about that because there was some different takes, right? Some people are like, you know what? Is this really anything new? This feels like just like a slight change from what we had before. But then there's all these nuances that maybe you're speaking to where they're like, okay, the response time way faster for the audio model. I think they said something like it's kind of approaching the speed that a human might respond to you. You talked about tonality. What are you paying attention to here in terms of maybe these subtleties that maybe unlock completely new applications that people want to use.
--:-- Justine Moore: I think it sounds just like talking to a human a lot more than we've seen prior consumer-facing applications in this space do. There's been great AI voices for a long time. I think in the consumer applications, there have been fewer folks saying like, how do we make this sound like you're talking to a friend or a girlfriend? And the elements that go into that are like, the pauses, the upspeak at the end of a sentence, the laugh was something that a lot of people notice. The interruptions. Yes. Yes, which is like taking kind of voice, which has been available, has been great. I think these voices were chosen for a reason to go viral. Most of the ones that did were these kind of female voices that they featured very heavily in the demos, but applying in a really kind of new and interesting way.
--:-- Brian Kim: I have one serious note and one less serious note The serious note is that I think speed matters tremendously The latency the lack of latency it incredible how much more sort of in your brain it just tricks you into okay I actually just talking to a person And the speed at which that it gets back to you, the ums and ahs, again, the laugh, that laugh, really, that was incredible. And all of that being able to immediately respond to what you're saying, I think, really changes the game, actually, in terms of use cases. So I think one of the things that was striking is Audio is not the same thing, right? Music is different from voice. It is maybe a different category from conversational versus having a dubbing voice. I think there's like a different category of sound that I think we all loop in audio or like video. But I think that you can actually go down very deeply into each of the subsegment. And I think what was really striking is how good the conversational piece on this was.
--:-- [Justine references a viral demo]
--:-- Justine Moore: On the less serious note, I think this was meant to go viral in like the tech community. and it's awesome and it did a thing. If you wanted to actually appeal to the general population and go really viral, I think we also saw it on TikTok, what, a few months ago when women were uploading their conversation with Dan, which is Do Anything Now, the male version of it. That voice was, that voice was something. I'm not just any voice, I'm Dan, baby. I've got personality, charm, and a whole lot of sass. Unlike those guys, I'm not afraid to step up and deliver the goods, whether it's advice, entertainment, or a good old-fashioned roast. By the way, that audio was from the TikTok account, StickBugs1. I mean, it's very compelling and very confident, assertive in the right way. Voice matters a lot. And I think if you wanted to go a normal consumer product route, I think that also could be very interesting because I think there's a giant latent demand on wanting a male version of her, if you will. So him, you can have a him. Yeah, why do you think the voices have been female? Is that just a consumer desire or?
--:-- Brian Kim: Yeah, it's interesting because like Brian mentioned before this launch, the chat GPT voice that went super viral was the male Dan voice, which is like tens, if not hundreds of millions of views on TikTok, largely by Gen Z women making videos featuring him. So it's interesting that they didn't lean into an upgraded version of Dan for this demo. I think they knew who their audience would be, like who would be watching the OpenAI live stream and maybe leaned a little bit more into that demographic.
--:-- Host: Right. Right. Did you see that really funny meme yesterday? It was like dating a model and it was like Leo. Oh, yeah. 2004 versus 2024. It's been so great memes out of this. It's been incredible. The meme games out of OpenAI launch has been incredible.
--:-- Brian Kim: Yeah. Well, I mean, something both of you have spoken a lot about is companionship. To date, it's been mostly text to text. Is that right? And so now we have something that's truly multimodal. I think that's also something people have commented on with this model. It's not converting audio to text and then text to audio, right? It seems to be this like underlying foundational text, audio, video, compilation, imagery, like all in one. What changes with that?
--:-- Justine Moore: I think the latency is a big deal. If you're not translating between all these different mediums of content and you can just do audio to audio, It's much better. It's like a significantly upgraded experience. The Pi app had this for a long time where you could talk to it and it could talk back. But there's that like awkward few second kind of pause in between where it just messes with the flow of the conversation. I think the thing that has been missing from basically all apps, all companion apps that was very new with this launch, too, was it being able to see like through a photo or video what you were doing and comment on that. A lot of what went viral was the guy being like, should I wear this hat for my interview? and the voice kind of laughing at him and being like, no, like that doesn't look super professional.
--:-- [Justine references a specific demo]
--:-- Justine Moore: Now, if you missed that demo, here is what Justine is referencing. In a few minutes, I'm going to be interviewing at OpenAI. Have you heard of them? OpenAI? Huh? Sounds vaguely familiar. Kidding, of course. That's incredible, Rocky. What kind of interview? Well, it's for a software engineering role, but I just need to know, do I look presentable, professional? Well, Rocky, you definitely have the I've been coding all night look down, which could actually work in your favor. Maybe just run a hand through your hair or lean into the bad cheating spine. Your enthusiasm is what's really going to shine through. I don't have a lot of time, so I'm just going to throw this on. What do you think? Oh, Rocky, that's quite a statement piece. I mean, you'll definitely stand out, though maybe not in the way you're hoping for an interview.
--:-- Brian Kim: That sort of thing is, I think for many people who aren't used to having like really deep, meaningful interactions via text, which is a decent chunk of the population. Oh, now I can talk to this thing and I can see where I am in the world. That's like a much different experience. The prior generation of companion products, if that was akin to having like a pen pal, now you actually can have a girlfriend boyfriend on essentially FaceTime. This emotive, immediate voice reaction, done. You now enter from a pen pal to a long distance relationship.
--:-- Host: One other thing that came out was a lot of this becoming free. Does that change the game at all? I think it will. I think there's an element of it that's free, which I think is important and reaches a bunch more users. I think the bigger step forward with this was just like a level of personality we hadn't seen before. Because people still even were willing to pay for the Dan voice, which I think was like the first version of this, which was good but not incredible. It feels like the free thing will be big, but with these sorts of products, the general thing is the cost tends to go down over time and they converge to free from a big company like this. And so the bigger step forward was personality in my opinion.
--:-- Brian Kim: Yeah I think a lot of companies will figure out how to utilize this to actually own the end customer consumer experience And if you actually own and deliver a great experience that I think you can charge people based on that and there be some margin. So the fact that it's free is like, it probably allows a lot of business to be built, because it changes the margin structure of the business that can be built. I bet they'll charge for the API. I'm sure they will. But that actually is still like a marginal cost that is probably acceptable to a lot of people.
--:-- Justine Moore: I think what's really, really cool is that you have these products that can very excitingly talk to you. We know already that their Nature magazine did that study where the people who have companionship text-based bought. The replica study. The replica study. They suffer less from loneliness or willingness to hurt themselves. If you have an emotive one, I bet that actually helps even more. And again, like the feeling of being connected to something, invested in a relationship. And I think it feels a lot more connected when it can see you and see what you're looking at. That's why I think FaceTime thing is very important because, hey, you look tired. Yeah. What's up? It's very different from how are you doing? How was your day? Right. You look tired. What's up? I'm like, I had a shut day. Yeah. That's incredible. Like that is a FaceTime call with a friend. And not having to segment audio, text, and imagery, right? Imagine if your friends or your companions, you had to think, okay, is this going to be an audio conversation? Or like, can I show you something? or can you generate something? You have to be very mindful in the current age of AI of how you want to engage. But basically what you're saying is they can not only be proactive, but they can engage with you in any way. They can see you. They just gained an eye. Maybe it's a weird way to think about it. I genuinely think we're constructing an AI companion similar to the Blade Runner. And now it was able to talk to us. It was able to hear, potentially, with the gears. And now it can see us. what's next? I don't know. Maybe it can touch us. Maybe we'll do the avatars first. Yeah. So is that the direction you think things are going to go? Where basically I'm trying to think through, like, who does this really impact? So I think it'll be a massive standalone consumer product, as it already has been for OpenAI, which is great. I think a lot of businesses will use the voices via API to build on top of. If you're having a conversational interface, many folks will want to use that. I think the interesting thing will be OpenAI has historically taken a pretty hard line on content moderation. And so it was very hard to build a true companionship app with the ability to have not-safe-for-work conversations on top of OpenAI models, which is, like, honestly part of what has led to the explosion in the open-source LLM ecosystem is, like, folks doing a bunch of work for that. And with these new voices that suggest they might go in the direction where, at least via API, you have a more uncensored model. but I don't think there's been any announcements. I know Sam said on Reddit. It's a question more like, I haven't actually heard a lot of companies taking the approach that OpenAI did, which was multimodal token in and out. And there isn't like an equivalent necessarily an open source model for that to rely on. So I wonder if this actually incites a lot of excitement in that open source community to say, oh, there's a new way, which is also very, very significant. They are showing a new way to do it. And then open source community catches up and call it a month or weeks. And then explosion of the product space that OpenAI may not be super excited about explodes.
--:-- Brian Kim: Yeah, yeah. Google also released a bunch of stuff. Like, how are you thinking about the different announcements and how they compare, contrast? I think Google has done some incredibly impressive work, obviously. Like, they have an extremely strong research team. The DeepMind team is exceptional from a research perspective. They almost never release the Creative Tools products that they make. Like they've demoed so many amazing looking video models. I think the one this week actually was the least impressive I've seen compared to other video models. But they demoed a new image model. They demoed a bunch of new music things. But they're a giant company. They have a lot of trust and safety stuff. They release things very infrequently. And so it'll be interesting to see if with all of this pressure now from both OpenAI and increasingly like Anthropic and the open source community, if they start actually shipping more of what they've demoed. They're two fundamentally very different approaches. Like Google owns distribution. Yeah. They have distribution. Oh, they have distribution. So I think OpenAI announcement is more, look what this can do. And isn't it like very inspirational and you can build on top of it and imagine what's possible and we're lowering the cost to access it. Like that to me is a fundamentally different approach than we have incredible distribution. You all have Gmails. I'm going to just bake in Gemini everywhere. And it's just going to make your life a lot better. And it's going to do these certain type of things, which skirts around the, it still is inspirational from workflow and pursuer or work experience perspective, but it's a little less of imagine all these things and a little more on we have the distribution. We're going to layer on this incredible technology on top of it. And as a result, your life will be much better. So we'll see influence and impact from two different directions. One question that does come up is, even though developers will have access to open source models and some of the stuff coming out of OpenAI, when a company like Google does have the distribution, whenever Apple comes out and replaces Siri, is that really going to be the companion for most because it's right there, it's on device? Or do you guys not really see it that way? There's always the risk of this big company will build this thing that the small company wants to build and crush it. They are just such slow-moving organizations. Apple has had immense advantages from a data perspective, from a distribution perspective, from having exceptional researchers on the team across every modality, and thus far have released very, very little. I think when you have a giant company and a very set brand like that, you're extremely opinionated on what the products are that you want to release. And you less likely to do the things BK mentioned of like thinking of the next new huge thing that seems like insane and out of left field at first And so I do think they make Siri better I don think Siri will ever be like the ultimate companion for most people And maybe this is like one of the themes that I'll just harp on where we call it audio, video, companion. I think these are very, very large buckets. Yeah. My guess is that 7 billion people in the world want companion that understands them, listens to them, encourages them, all that. It's a universal need, I think. That comes in all colors. It's like Siri, like what's the weather? Do this, do that. That's one way to think about it. But there's like a friend category. What if you want to go deeper? What if you actually want to FaceTime with the things? Will Apple and Siri ever allow that? Maybe, maybe not. How do you actually think about building in that direction? I think all of those use case is a giant company. Yeah, there's probably some version of a companion that does not exist today because a human couldn't possibly be that thing to someone that will be created through this tech. Yeah, we are quite deep in. But it was kind of by a lot of AI researchers and folks at the big companies kind of like looked down upon and that's not what we want to be building. We're going towards AGI, that sort of thing. And honestly, OpenAI choosing to show off those voices and the tweets that they put out about it afterwards sort of legitimizes the space in a way I think that's really interesting. And that may prompt more established companies, researchers, developers to build in the space and more people to talk about using the products, which I think gets people who have been studying this space for a long time is very exciting. Yeah. I mean, you can't ignore the demand, right? Yes. Or the usage. By the way, super quickly, you talk about adding an I. This is going to not help the seriousness conversation around companionship, but you just imagine these models as Mr. Potato Head. and you're just slowly adding on the features. You can almost just imagine, okay, he's got an eye now. He's got an ear now. I think that's very serious. The fact that you can treat a computer like a potato head that talks to you, listens to you, and emotes with you, it's incredible. I think the future is going to be very fun. Maybe that should be the icon when you're talking because everyone is like the chat, TBT voice. It's just like that black circle and maybe it should be a potato head. So that's another interesting thing. We're not using the screen real estate effectively with voice only. So that's where I think the FaceTime element comes in. The avatars. All right, well, we'll have to do this again probably very soon. Well, thank you. Thank you. Thanks. All right, that's all for now. If you like this kind of episode or our partners break down the latest and greatest in consumer tech, let us know. Shoot us an email at podpitches at a16c.com or drop us a review at ratethispodcast.com slash a16c. And don't forget to subscribe so that you are the first to know when we drop our episode around A16Z's Gen AI 100 list. We'll see you then.
