--:-- Julie Yu: Yeah, the funny thing about that idea is, I mean, it's very exciting. Idea because we could gain so much knowledge and we could improve healthcare so much, but it's a ridiculous thing to imagine doing without something like AI. Without AI, I don't even know how you do that, how you could pay for that. What do you mean by that? What parts of that do you think AI is? Really in many ways this becomes a data problem in terms of slicing and dicing and then keeping track of this person got this drug and at this moment had this response and sort of understanding the causal nature So what we love about clinical trials, and I'm going to get a little wonky, is that you have to have some sense of causality. Like, I took this drug, this happened. And people say correlation doesn't mean causation, but that doesn't mean we can't do causation. That's what trials are about. Trials are all about causation. So we have to understand the causal pathway. But with all this data, AI is great, especially certain types of AI like Bayesian statistics are really good at causality. And so we could actually have causal understanding. And it can even be complicated like you're not supposed to take like grapefruit juice with your birth control pill because the P450s will nullify this. I don't know how someone figured that out but who knows what else is out there that we just didn't find yet. And if you could have an app that knows your diet and knows the basic things and knows your drugs and has that information, that's kind of mind-blowing. That how much we can just have no new drugs, no new treatments and just optimize. We're such an unoptimized nature because that optimization is almost impossible without AI. But then on top of that, once you've built out this infrastructure, now new drugs go into that infrastructure and can optimize. And finally, we don't just optimize for health, but we can jointly optimize for health and decreasing cost. This drug is 10x more expensive than the other drug, but maybe the outcome for me is going to be no different. So maybe I should get the other one. Exactly. And how to think about that, that's such a complex data problem and logistics problem, which is also another part of AI, that I think we could actually really finally tackle. So as you can tell, I'm very excited. I remember I had a conversation with the CIO of the VA many years ago where one of the ways he looked at his population, he's like, Julie, a million clinical trials are just organically running in my population every day and I have no idea how to harness it. And so even with just EHR data alone, you can imagine the possibilities, let alone if you were also to layer on top of that just your daily behavioral data and all that kind of stuff. And that's where the LLMs could come in is just a conversational way to capture the day-to-day journal of what you're doing, what you're eating, who you're interacting with, all that. And it's doctor behavior. It's like the whole system we can finally debug. And the funny thing, coming from a tech background, if RCT seems unfamiliar, this is basically a giant A-B test. Exactly. And so this is deeply, deeply entrenched in tech. Even every pixel is A-B tested. I wish healthcare could be optimized the way pixels are in webpages. And you'd have an A, B, C, D, E, F, E, on and on in terms of multivariate nature. That degree of optimization is something that would be just fantastic. And that gets to the other sort of holy grail problem that you hear people talk about is, could you actually ever have almost a spot market for pricing in healthcare? And on any given day, at any given hour of the day, you might have a very different dynamic of supply that's available for a given service. Why could you not price differently for it the same way we do in other industries? Yeah, so why can't that happen? I mean, today it's probably deemed illegal, honestly, by many of the contracts because you're sort of bound by these, again, these monolithic agreements that highly specify. And the fact that you have this claim system, there's really no notion of a real-time adjudication of the actual price that needs to be paid for that service. And maybe this doesn't require the fanciest type of AI per se, but certainly the notion of being able to run machine learning on these things and say, how many of these rules are just useless because they don't actually move the needle on cost or price, but which are the ones that are most consequential that we actually should keep and therefore have some kind of automated and systematic way to adjudicate them. So huge opportunity and that again represents a very significant portion of what gets wasted in our system today. So that's a fun one to think about. And then relatedly, I mentioned at least in my career in healthcare, one of the general themes of the problems that I'd like to go after are where there is a fundamental mismatch between supply and demand. And I think a lot of the companies in our portfolio represent that problem space. And I had built a company that was in the scheduling space, which when you think about the phenomenon by which I'm sure you've experienced this, you as a patient are told to wait weeks for a doctor's appointment, and you assume that that's because every doctor is booked out solid. But it actually turns out that a lot of the capacity in our system goes completely wasted. And if you could just simply get better visibility into the underlying data streams, then potentially you could really mitigate the wait times and the experience for the consumer while also helping the providers best use their time. Are there any examples that you've seen that you think are an interesting representation of that? I know we have a lot of companies that are trying to both increase transparency of supply, companies that are using coaching or provider groups, and then bringing that to bear in care models. Well, here I think you're describing something which is even like maybe before we even get to AI, which is like we got to get like off the whiteboards and onto some more modern sort of computer approaches for things. Yeah, like systems of record. Systems of record. And that actually, you know, we often talk about like technology versus people. Where's the weak spot? Where's the problem? Maybe the most heretical thing one could say is that maybe the way of doing medicine has to change. Tell me more. What do you mean by that? Yeah, well, and so for instance, like just the workflow of a doctor, you think how a doctor goes through their day, How can we support the doctor to do what we all want them to do and what they want to do, which is maximize patient welfare and maybe view it as something that takes it off of them? So I think about Devoted and their infrastructure or Inoco. They address this system internally and to the extent that they're not optimized, that's something that even they know and they can see. Right. Yeah. And that's an example in the case of companies like Devoted who actually did start with Clean Sheet. Yeah. Right. And built their own scheduling system to accommodate the exact care model that they were going after. Yeah. I remember from some of the work that we did at my old company, you would see the way that doctors design their schedule. And very much to your point, they would design the templates of their schedules specifically in a very protective, like almost a defensive way, because they felt wronged by the way that the system sent patients to them. It's a learned behavior because they've gone... They want to protect their time. Yeah, and they've gone screwed in the past, perhaps. Exactly. We understand what this feels like when, let's say that we have 10 pitches with new entrepreneurs that we've never met in a row in a given day, and you're just on for hours straight, and you know how physically and mentally exhausting that is. It's the same thing for doctors. When they say, if you give me four new patients back-to-back in the morning, that is far more taxing to me than if you intersperse repeat patients or other tasks and whatnot. After seeing the pictures, then we have to go into Epic and write about the pictures. Exactly, and that's even worse. My God, if we had to do that. So you could see how the unfortunate side effect of the way that the systems traditionally have been designed has now caused this sort of ripple effect and defensive behavior. But if you were to just kind of start from a clean sheet, as companies like Devoted are able to do, could you actually design a much more logical system that actually learns from historical data? And there could be almost like a reinforcement learning component to that where the doctors could provide feedback and you can learn over time. So we talked about one idea, which is a clean sheet of paper idea. Is that the only way? What I'm hoping also happens as part of this wave of AI is that it's really a forcing function for people to actually take advantage of the data that we have now digitized, right? So we're actually only, what, 10, 11 years post-meaningful use in healthcare, which was the act that incentivized doctors to actually adopt electronic health records to begin with. And it's kind of remarkable to think that even five years ago, less than 70% of doctors had an electronic health record. So we're actually relatively new into the era of even having digitized forms of information about patients over a longitudinal period of time. And so in many ways, we haven't at all scratched the surface of exploiting those data sets. And there really hasn't been that much incentive to, historically, I would argue. But certainly not necessarily the technology capabilities at kind of the middleware layer of the stack to be able to do anything meaningful and useful with that data. But I think that's where the advent of the tremendous technology shift that we've seen on the AI side and what you can do with that information, how you can synthesize it, how you can present it to someone in a way that's actually usable and friendly. That could be the tipping point that actually gets people to unleash the data. We're also, by the way, in a period of time when provider organizations, hospitals are struggling financially. And so many of them are looking at, OK, how can I actually monetize my data assets? Right. Like what do all of our companies want? They want to eat data. And one of the ways by which they can do that is actually partnering with these provider organizations who have these systems of record that they haven't been able to exploit and actually pay them and give them rev share or equity or whatever it is to get access to proprietary data sets to train their models. It's interesting to ask of the various crises, like the staffing crisis versus the issue hospitals are dealing with, which crises are going to be catalysts and which will be impediments. I think we kind of feel like the staffing crisis really is like tailwinds for AI. Strangely, yes. And COVID, I think, is tailwind for AI because we're so used to virtual. But maybe not all these crises will be helpful. I think that would be particularly interesting to see how it nets out. Yeah, I think that's a great point. I think certainly there are many who would view it not as a tailwind, but I think it's a good forcing function because now people are at a breaking point where the status quo way of solving that problem, which again is how do we produce more doctors? How do we produce more nurses? We just, we can't do that physically. And so that is driving, I think, a lot of this adoption. We were remarking as a team that at this last JPMorgan conference, like 100% of the incumbent payers and providers got up on stage and talked about not just what they want to do with AI but how they actually deploying AI in practice because they found no other way to be able to solve those kind of more fundamental problems I think the other tailwind that some might call an impediment but certainly builders in our universe call a tailwind, is the business model change in healthcare, right? So movement towards value-based care fundamentally breaks the kind of the schema of like how healthcare has worked for decades. And I think incumbents are more likely to be on their heels with that dynamic versus the upstarts who themselves can build their entire care model and operating model on the basis of those new payment domains. I actually don't envy organizations who have to have one foot in each world, right? Because having half of your shop in a fee-for-service model and then the other half in a value-based model is very, very challenging to do. Well, it's fun to think that like in a fee-for-service world, AI is nice, but maybe actually goes against what you want. And in a value-based care world, actually AI is the catalyst, right? Because if you can do things better, you can see the value to it. Yeah, it's actually funny. The example that that reminds me of is how the AI are getting sued right now. So there's a bunch of major national payers who are using AI algorithms to automate prior authorization. And so all they're doing is taking the rules that humans wrote, that humans were executing slowly, and doing it faster. And so now all of a sudden everyone is complaining, the denial rate is going up, but the rate of denials is not going up, It's the speed with which the denials are happening that's going up. Don't blame the technology. Blame the humans who actually wrote the rules and you're just seeing kind of an exacerbated version of it. We're going to deal with a lot of that of kind of the finger pointing at the technology where it's actually just implementing the broken system underneath it. And that's why this kind of move to like new business models gives you the opportunity to clean that up and start with logical ways to control spend. Okay, so we talked about healthcare administration. We talked about scheduling opportunities. Let's actually talk about the EHR itself. So LLM as an EHR. What do you think about that? I think the thing that's really underpreend about the LLM is people think of it as this Oracle or something like that. But I think it's maybe, at least for us, I think of it as a UI. And it's kind of funny because we start with command line interfaces for those who have ever dealt with that. And then we have GUIs because that's better than command line. But now we're back to text and typing things in except instead of some weird command that you have to memorize. You just tell me what you want. Right, you just speak English. And we're so optimized for speaking English to each other. I mean, that's easy, it doesn't require training in the same ways. And so I think as a UI now, that makes sense. And now when you're saying as an EHR as an LLM, then I guess you're kind of meaning that the data is in there and it can be queried like this and maybe synthesized. That's all very natural. I think obviously you want to be very clear about partitioning things. So maybe you're doing it with like RAG or something like that, where it's getting information coming back. But maybe the question to turn around is, again, the technology sounds very plausible. You can imagine a hackathon that would put some pieces together and get that done. But I think you need more than just connecting to GPT-4 or Gemini or something like that. You need something medically specific. Yeah, absolutely. This is where I think one of the prime examples where we certainly believe that a specialist model is necessary to understand the specific nuances of how to interpret medical information versus general internet information. And I mean, that's certainly a big area of development for builders as we see it. Companies that are building tools that can do everything from summarize existing medical record data. How do you tell the story of Vijay Pandey before he walks in the door so that you understand like his journey and not just look at a bunch of numerical records and sort of sporadic information about different visits and whatnot, but really truly the story of him, including things like your social determinants and what happens in the home and outside of the clinical setting. We see a lot of companies kind of building that. The other obvious use case is the scribing use case where have a conversation with your doctor, actually look them in the eye rather than them sitting at a keyboard during your entire visit. And that also gets written as a story. It's a story that then gets added to your medical record and it can create that flywheel effect of continuing to add to the narrative of your journey. One way to sort of think about AI in healthcare is take the existing jobs and then see which ones can go. And that makes a lot of sense. I'm also curious if you sliced and diced it a different way, because we don't have people with AI, how would we do it differently? Because people are assigned specific jobs because of the way humans work. But maybe if when we're finally said and done, when AI can do everything, maybe the resident isn't the role that it would take. If you were to unbundle the job of the doctor, what components could you re-bundle into a different thing? There was actually a time where this concept of a dataist was popularized, where a baseline component of basically what every job in healthcare is doing is some degree of data interpretation. And so if you were to unbundle that component and create almost like a horizontal job that was just doing data interpretation and this kind of thing, and maybe that's actually the better analog to what I described earlier. It's like, what if there was a dataist role that effectively is an LLM that is synthesizing all this information? I think the thing that's missing right now to make this a reality is today's information architecture is very sporadic, right? So you're a pretty healthy person, you probably see your doctor maybe once or twice a year. And so how do you augment that with more continuous information, whether it be self-reported, whether it be remote patient monitoring data, whether it be just other information sources to create that more holistic picture. But I like that notion of flipping the jobs on their head and thinking about the components a different way. Well, the fun thing about the data is, I think the UI is pretty important, right? Because we're talking about a team of people. And medicine often is done by a team. There might be a nurse or a PA or a doctor or a specialist and all these people. And where the AI comes in, one idea is a co-pilot, which is like each one of the team members has a co-pilot. But what's interesting about the dataist is like the AI is a peer, a contributor, part of the team, and has its role that actually everyone feels pretty good about. And you think about it, you don't put a person to do the dataist job. I mean, in principle, with a calculator and a lot of time, maybe it could do all what's necessary. But you'd never have a human being do that. And that might be a very easy first entry where it's like they're doing what they're good at. Yeah, that actually reminds me of a company that we saw that what if every nurse in the inpatient ward, because the inpatient setting is very chaotic, very active, surprises happen all the time. And a lot of nursing teams have sort of either live walkie-talkie type devices just on their shoulder or they'll have some real-time communication mechanism with the rest of their care team. And this company was saying, why not put an LLM into the same walkie-talkie signal and actually literally just have it be almost a Jiminy Cricket sitting on everyone's shoulder being like, hey, I'm sensing X pattern by virtue of listening in to your conversations. Let's all remember that this is happening with this patient over here and I think there could be a safety issue over there. It's almost like the way that I talk about Baymax all the time. So could everyone just have kind of a Baymax companion and hang out in their care team and be sort of the steward of all the information flow, synthesize it and read it back when they find something that probably warrants an alert within that group. I don't know if you spent much time in the ED. I break this and cut that. I have lost scars and stitches and so on. So I remember it was a few years ago, actually right around here I cut myself with a chef's knife. I was probably showing off to the kids. So it wasn't looking good, I was like, I should get stitches. And I go to the ED. I'm there for like two hours. Wow. And I look around. Yeah, just waiting. And I look around and I'm like, this could be another four hours. And I'm doing my math. And maybe for the first hour I'm bugging the nurse and the inbound. But after all, I just leave. And there's various situations where I just want to talk to somebody. But you can't have everybody talk to somebody because they're going to be overloaded. If I could just be texting somebody. I just want to know where things are. And if it's busy, that's fine. Or maybe I don't even need to be there. But that triaging too could be huge. Absolutely. Going back to the concept of unbundling the role of a clinician, there's one part which is actually the treatment part. So that's the part where maybe we can't necessarily today build an LLM that will stitch your finger. But the notion of triage, getting you to the right site of care. So should I stay home? Should I go to an urgent care clinic? Should I go immediately to the ED? Or am I okay just going to my PCP? That is actually a critical role. You know, we wrote a piece about this where we said, my version of that today is I call my doctor cousin, as all of my family members do. The poor lady, she's a cardiologist, but she gets every single call about every single specialty under the sun. And she'll tell me literally like, should you take your son to the urgent care or does this need to go immediately to the ED? That is like one of the roles that LLM construct could play, which actually would also do a huge service to doctors that they don't have to be the ones who are fielding those questions. Well, especially the alternative is Dr. Google, right? Correct. Or WebMD. Or whatever and then the patient which will tell you you have cancer the amateur is deciding and especially with my friends you've probably been through this your kid is sick and you're like I probably don't need to go in but it's my kid so if it's even 5% maybe I'll go and that's just a drain on everybody doctors and patients so we talked about EHRs and this notion of the patient story we now getting into this notion of if you were to take almost like the front door experience to healthcare and what are the big opportunities for AI to make an impact there One is simply instead of going to Google you know going to a specialized tool or whatever it might be that's trained in this way. One of the questions that always comes up is, what are like the regulatory rails on this? So like, at what point do you sort of cross the line into actually clinical decision making? And how should I think about this as a builder? I know you've obviously done a ton of thinking on this and a ton of work, including like talking to the regulators and understanding what they think, what kind of advice would you give to entrepreneurs who are trying to figure out where that line is and whether or not they should cross it? Yeah. So a couple of things. I think some of the lines are more clear than others. But in the cases where there is any gray zone, I think the regulators are, I think, eager to chat with startups. Especially maybe it's on the software side, that might be ONC, but so on. But to try to figure out where you are. And we've seen a lot of successful founders who have done that type of collaboration. And I think generally that's a pretty strong approach because then there's no surprises on either side. The tricky part is when nobody knows. And so I think it's not just about consultation, but it's also leading. And sort of taking the framework and the philosophy for how we regulate things right now and really understanding, is this software as a device? Is that the right framework? And really sort of being a leader in terms of how we should be thinking about this. And I think there's actually welcoming of that as well because it's new for everybody. As you're alluding to, we are actually an industry that has a regulatory framework when it comes to AI specifically. And so in many, these are like some of the rare cases where healthcare is actually ahead of the curve as far as technology goes. What's your sense of, you know, does generative AI specifically, do LLMs constitute enough of a sea change relative to historical waves of AI that it should
