Most of what happens in your health happens outside the doctor's office, right? It's happening to you day to day. I know for me, it's hard to remember, "Did I have this symptom four weeks ago, or three weeks ago?" -And it's so true. Right? Most of how you experience your health-- your health, isn't at the hospital or isn't at the doctor's office. It's at home. It's on a jog. It's going to bed. It's disrupted sleep. /-Historically in the past. The kind of wearable information, the fitness and wellness apps were separated from the medical field. But we're starting to see those connections across now in terms of how can this be really helpful and useful, where you can actually forecast out multi-year for what's going to happen; then those behavioral changes that you're trying to ask people to do in the near term, won't seem so big. -So a weather forecast, but for your health. Yeah. /-A lot of the decisions that get made in healthcare right now are based on rules and diagnoses that were derived from very small scale studies; As more folks engage with healthcare in a digital way, the medical community can bring that to bear back on medical science and start coming up with more tailored, more personalized recommendations. And I think it's just a matter of time before we get there. -AI often gets described as the next industrial revolution for humanity, where the world could change in ways that are almost unrecognizable to us today. So just how different is the world of healthcare about to become? What will it mean to have AI assistance dedicated entirely to your health; or AI systems that work around the clock to catch the earliest signals of disease? Today, I'm sitting down with two of the leading minds at Google Research who have each dedicated decades of their careers to health; to ask what exactly is Google building at the intersection of health and AI, and whether we're approaching a completely different era of human health? I'm Sinéad Bovell, and this is The Google Research Podcast. I want to start with understanding. The moment that we're in. Because AI has been a part of healthcare for close to a decade now. A lot of it's around access. So, when we were working on this ten years ago, images were the technology that AI could work from, right? Now that AI can handle text, it's much more able— It's a much more useful tool in facilitating communication. And communication happens everywhere in healthcare, right? Directly with patients. You know, folks are using consumer tools and trying to ask for health information there. And so really the sort of-- both the opportunity for access and increasing access to care, but also the challenge and sort of building and maintaining trust is much bigger nowadays. -Well, so we're talking about accessibility. That's happening not just with the patients and the physicians in terms of how it can interact with the computer, but it's also happening for developers. You can develop these foundational models and put it out there. And you've seen this with the open source models that have been put out there. We have MedGemma, for example, and I was really happy to see how much adoption it can take off compared to before. Now a developer takes these foundational models, tailors it in 3 to 4 months and is able to push it out there. And we have 8 million downloads, like 60,000 developer active developers, and I think of each one of those developers as helping their community, whether it's in India, whether it's in Global South, different parts of Africa. Sometimes they're using it for medical education, sometimes they're using it for helping to screen where they just don't have the specialists. And so this is nice to see it light up around the world so that everybody can actually get access to expert-level care. -What can MedGemma do if someone is hearing about it for the first time? Because you're saying computer scientists, developers have been downloading this model, which I think is really curious, so what can this model do in healthcare? -So part of the thing we do at Google is figure out what's the useful role we can play? And one of the things we're really good at is building models. One of the things that we're not well positioned to do is figure out how to adapt that model, move it out into the world, get it in front of patients in a way that helps them. And so this is where we rely on on the rest of the healthcare ecosystem to do that. The folks who know this best. So developers take these models, like MedGemma, that are foundational models that can be used for many different tasks. They'll figure out what's the problem that needs to be solved in my local area that helps patients and doctors in my area. They'll build that product, they'll figure out how to get it onto the market and get it in front of patients. And by sort of us investing in these foundational models, we can sort of help the rest of the world bring to bear that- that AI technology for patients. At scale. -It's actually kind of costly to evaluate and be able to get to this higher level quality that you want. That's important when you're going into the healthcare space. So these aren't just any developers, they're healthcare developers. Usually they're trying to develop something for community healthcare workers who are out there that are bringing the care to the last mile edge of care for underserved populations. And so they don't have a lot of resources to put into that. We do the heavy lifting and provide these technical building blocks, and then make it quite easy for them to turnkey that into something that they now tailor to their local population. -And what are some of the different ways MedGemma has been used in practice? In a clinic? Are there- do you have certain benchmarks that have come through, certain examples of how it's performed, and what is it performing in? Diagnostics, illnesses? -Diagnostics, treatment pathways, there's indice-- I think it's called indice derma (DLQI), which is one that was picked up by a India Institute (AIIMS), and they were trying to address the problem, where they have a ratio of 1 to 140,000 patients; one dermatologist under 140,000 patients, which is far beyond what you want the ratios to be. And so they were trying to, help reduce the burden on their specialists. And they were able to develop this screening program that helps really triage and make sure that the hardest cases and complex ones go to specialists, and the rest are actually able to be addressed by community healthcare workers not requiring the full specialist to be able to deliver that care. -And it's happening in very different ways around the world. One of the challenges and opportunities is that healthcare looks very different in different parts of the world. So, for example, in India, one of the ways they're using MedGemma is to— they want to build apps that everyone can download on a phone, on their phone that helps them manage their health data. And so as part of that, you need to be able to scan in any kind of complex medical documents and have a machine be able to understand that. And so they're using MedGemma as actually the engine to be able to build apps they can understand, organize, and bring to bear people's healthcare information for their own use. -There's another case that is worth mentioning, which is the Taiwan Medical University used it to essentially bolster their challenge that they had around nurses burning out for delivering care. So they actually needed something to help also, again, triage and screen and help offload some of the, um, kind of patient demand. But they were able to evaluate it not only helps the nurses reduce burnout, but they were able to check for outcomes improving, when they're providing both the diagnostic support through the clinical decision support. And so is it because then that it's a foundation model, that it can be used in all of these different ways? Because you mentioned it's helping with some of the the medical documents and forms. And then you mentioned triage. So, and it's the same underlying model. So it can be used in all of these different ways. -There is still like a family of models in the sense that we have different size models. But for MedGemma, you have at the core sort of this language model that allows you to converse easily. And so then it becomes much better and tailored for healthcare. - And can I ask you about a very specific study that I had read? So, it was published in March in 'Nature,' and it was a study where you had combined forces with, I believe, it was the NHS and Imperial College London, and you developed an AI system for mammogram and breast cancer detection. And this AI system was able to detect 25% of the interval cancers that radiologists missed. So an interval cancer, you go to the doctor's office, you're given a clean bill of health, but then a tumor starts to grow between now and then. So what are you learning in this work that AI models can see or detect that humans can't? So I'll say that I think that question's interesting to me because, AI is helping in multiple ways here. One is— is it helping when like, any human can get tired and they might make a mistake? Is it helping in catching that error, so just out of sheer like— the doctor normally would see it, but they missed it. There's also cases where it's not exactly what they were looking for at the time. And we sometimes refer to this as like where AI can come in and be a really great disease surveillance. So it can also catch kind of these conditions where you were looking for one thing, but you actually— the AI picked up something else because it's knowledgeable and it knows. And then there's a case where it actually sees something that doctors and humans can't see, because it's working on a multidimensional level and it's able to detect micro patterns that humans can't. And so I think there's these different layers of ways that AI can see things that humans can't. But I'd say for this particular study, it was more in the realm of it was missed on the initial scan. And so it was able to be picked up. And it's because, you know, the interval cancer is like much more subtle. And so it's a combination of not what they're potentially looking for and also more subtle and not as obvious to the doctor when they're seeing a lot of scans at the same time. -Yeah. The other interesting thing about that work, and in general about the work we've done in mammography is, it's been surprising to me— So when we first started in this space, as computer scientists, the first thing we think of when we think of what can I do in medicine, it is that diagnosis and the sort of can it see what's going on. And so we thought that that would be the useful thing it can do and, that it would be about the accuracy of that diagnosis. But we found through this study and other studies that there's all these other side effects, beneficial side effects, to the automation aspect of automating specific workflows. Like, for example, in the mammography automation, it's actually often the biggest impact is the time to read. So we've done multiple studies now in multiple different imaging domains where we find that if you can get a patient's read much faster, often in especially in lower resource settings, reads take days. It takes days to get the image to a doctor who can read the image and get the result back. If you can shrink that days into 15 minutes, it unlocks the ability to then immediately schedule the follow up with the patient, and you see much higher rates of patients actually going to follow up, not being lost to care, these kinds of things. And some of the if you look at the system as a whole, I think that may be a larger effect than just being accurate with the diagnosis. But being fast, surprisingly, has a huge impact on the healthcare system. And this is because healthcare is complex, which is like a theme we we keep coming back to I think. -I mean, 25%, catching 25% of, breast cancer, things that would have been missed. That's very statistically significant. Doesn't that seem like we're stepping down a path where eventually it becomes a non-negotiable to at least have this backstop with AI? The same way, I mean, with electricity, AI is a general purpose technology. And over time, those just get-- those become the expectation. And to me, that's an early indication that that's too statistically significant to not include. -I think the way to think about it is if you look back at the history of medicine, there's been many examples of new tools that are developed that doctors use, that allow them to do some part of their job much better. I think you'll see that AI becomes a new tool that doctors can use to to make much better diagnoses. And to your point, it'll be, you know, still under the purview of the doctor, but they'll have this kind of tool they can deploy where they were staring at an image or reasoning about a set of symptoms, they'll be able to hand that to a machine. The machine will do it for them. And then they can focus on, "okay, what's the next step?" -And then I think. As you had mentioned to the AI, it could pick up things that the doctor, that specialist is looking at one very specific focal point, you know, this organ or this area of the body. But the AI probably has seen many different aspects of the body, and it's not necessarily siloed to a specialty. So in the event that it can spot something that the doctor wasn't even looking for, those types of scenarios will probably also start to increase, I imagine. -Yeah, it's happening right now. We've done a lot of work in this space to say, like, how can we pull up a level and not be driven into these silos of specific pathologies and actually just come at more holistically? -Yeah, I think the other angle there, from talking to doctors who are the leading early adopters of AI in their practice, a lot of doctors go into a specific specialty, right? And they see patients for their specialty. In the past, they would often have to go to other doctors with other specialties for, like, an off-the-cuff kind of consults to make sure that they're covering all their bases with the patient. And a lot of them are finding that the their ability to gain expert information by asking AI about fields that they aren't specialists and has helped them sort of sure that up and to make sure that they're treating the patient holistically and looking at them from all angles. -And this also saves time because every specialist you have to contact, well, they have their own list of patients and their own things. And outside of the breast cancer detection, are there other symptoms that AI is getting really good at classifying that you're aware of? that we give off? Or maybe it's the skin, something that AI is starting to be able to classify in a different way than us. I'm pretty excited— Actually, just yesterday we talked about the launch of Health Guardian, which is part of our personal health assistants. And the reason why I like this work very much is it's contributing to this growing body of work of how can behavioral changes like sleep, diet, movement, these signals that are coming off of your body that are telemetry data sets, how can those help inform what is the underlying physiological trend that you're going on? And this last set kind of points to metabolic dysfunctions and how that relates to pre-diabetic and insulin resistance. So you're starting to see the dots connected. Historically in the past, the kind of wearable information, the fitness and wellness apps were almost kind of a disjoint and not-- separated from the medical field. We're starting to see those connections across now. in terms of how can this be really helpful and useful. And so this is where I think there's lots coming online right now that can help make these connections. -I think the interesting framing for this, for me, is-- most of what happens in your health happens outside the doctor's office, right? It's happening to you day-to-day; you're experiencing symptoms. And so, sensors are one way that we can sort of help capture some of that and bring that into the healthcare system so that your doctor has that information. But the other way is just what's going on with you day-to-day. And a lot of folks are turning to chatbots to sort of talk about, "hey, you know, I'm feeling sick today. What might-- I'm trying to figure out what's going on with me." I think what we can do in the long term is we can help people collate that information, record it over time, and bring it to the doctor so that doctor gets a more complete picture of what's been going on in your life and your health. I know for me, it's hard to remember. Did I have this symptom four weeks ago or three weeks ago, and when did this thing happen versus this other issue? And I think we can help people record all of that and bring that to their doctor. And doctors can help them better with their health because of that. -And it's so true, right? Most of how you experience your health, isn't at the hospital or isn't at the doctor's office, it's at home, it's on a jog, it's going to bed. It's disrupted sleep. And I think we all feel like, okay, isn't there something that I can wear or buy that can carry in some of these symptoms and some of that data? And so when you say sensors, you mean things that are on our wearables or things that we can wear, that we can sit on, that we can plug into a mirror. What are some of the behavioral things that AI is able to catch and maybe spot? Oh, there's something that you're eating or doing, or there's some way that your heart rate is moving or not moving in the right direction, that we it could be attached to some sort of metabolic condition that you had mentioned. -I do encourage folks to look at the work inside the personal health assistant, because they've gone deep on this from sleep to movement to activity. And, you know, eventually they'll get to things like nutrition. But all of these factors have a pretty strong, impact on health outcomes. And it's widely acknowledged that behavioral change is responsible for at least 40% or more of what could have been a better health outcome. And yet, I think we haven't fully connected the dots until more recently between, hey, you have this chronic disease, let's just say diabetes, type two diabetes. That if we could have just showed you that, you know, your sleep and maybe eating habits and also things earlier before you became pre-diabetic, you just had to make small adjustments. And this is where the AI again comes in to be very helpful, because it can help with understanding your personalized motivations and drivers to help nudge you in the ways that would actually make the change from your day to day. In a way that's not against your own personal value system. And I think every person's uniquely different in how they're motivated. But this is where I think having this conversational and very much accessible AI system to tailor to you your approach and, and and focus your outcome and share that. That's at the core of a lot of these health assistants today. -We're still early stages in this area. The behavioral change, the effect of coaching... These are, I think, the promise of the future of AI. Like everybody wants that personal coach who, to your point, and kind of knows exactly what you care about, what you want, and how you want that to show up for you. Some people love to have, for example, to be to be poked at, and prodded, and nudged and say, "hey, did you remember to do this?" For other people, that's super annoying, right? So I think we have to figure out how do we how do we custom tailor this technology to every individual so that it's helping people meet their goals and starting from where they are and meet their goals. But and I think it's just a matter of time before we get there. -And so if I was to use Personal Health Assistant and say, theoretically, this may or may not be true, that I'm someone who likes sweets and baked goods and knows every bakery in New York. That may or may not be the case. How would Personal Health Assistant assist me in making better choices? -Well, so I think there's kind of these both clinical guidelines as well as known practices in terms of small changes that can be made. So let's just take your sweets example. Things that you don't think might affect the outcome. Do you, after you've eaten, have you done some level of movement right after? Are you causing certain glucose spikes because you haven't had anything else during the day? Are you pacing it out? I think people— it's complex; they're too much for people to want to think about day-to-day. But I think if you tell it, your goal is, well, I'd like to make sure that I have a family history of, you know, maybe diabetes. And I want to make sure I'm being proactive about not going down that path. It can help. But to Dale's point too, right now, there's studies being done actively to see how much of this is actually causal and how much of it is is more just placebo effect. And part of what I would say is that we're seeing very good growing body of evidence that there's many things that you can do from movement to when you sleep, and having regularity around this, to understanding your body and how much of its, you know, deep sleep versus like REM cycle versus light sleep. We did these studies with sleep physicians at Stanford. Like these are not just like, you know, arbitrary pieces of signal that we think are important. It's being done with experts out there. So I think it's nice to start seeing these roles converge, which, you know, previously, everybody's in their respective silo. But as a human being, you know, we're every aspect of what I do is happening all the time. And so it's weird to just only take a snapshot and pinpoint and extract it out and say, this is your health forever and onward. -Right, health isn't at a particular moment. It's just the— it's an ongoing continuous system. -Very much. -Yeah. And I think maybe to your example for sweets, it brings up a personal example for me, which is I'm lactose intolerant and I'm waiting for the day, which I think is probably not that far away. Where I can— when I go to a restaurant, I can take a quick photo of the menu and have it turn red or green, right? It's like, hey, these are the things you should probably be steering towards. And I think the same thing for sweets, right? If you have a goal to reduce sweets, "I would like to eat less than this amount per day." I don't think we're that far from a world where you have tools that go support you where you can. You don't have to do the-- invest the hard work and time of trying to dig through the the ingredients list for everything you do, but you get an assistant that is actually effectively does that for you. And it kind of helps you make better choices for yourself based on your own goals. -And I know, the future of healthcare is a fan favorite among all three of us. So we're going to talk about that in a second. But I think when most people envision the future of healthcare and the role AI might play, they're actually envisioning a system like AMIE I feel like. Based on the papers that I've read about it. -Yeah, because I think they're thinking of an AI doctor, possibly. And, you know, I would say, AMIE, for us is in part to answer the key research question of how far can you push AI to deliver upon expert care and make that accessible to everyone? And so we're really trying to see how far AI can go to, provide diagnostics and disease management support. And what we found from AMIE is that it does outperform primary care physicians for diagnostics. And it's seen as valuable from specialists like a majority of time when they're looking at disease management. And it's become useful as a force multiplier with the physicians in terms of it then helps them prepare for their conversation with the patients when they're engaging because of the summary that's able to provide and also even adjust their plans based on what AMIE's been able to provide them for treatment. So this has been really promising from our perspective. But I do think that where this is going is really how can you make expert level care accessible to everyone? And that's where a lot of the work that we've been doing, AMIE is pushing that frontier. -Yeah. AMIE's a good example of one of the kinds of research that I find really fun, which is we get to imagine roughly what a future world might look like, right, where AI is somehow facilitating communication between doctors and patients and helping them both out at the same time. And then we get to find great partners. We're partnering with the Included Health, for example, which is a company that's a telehealth company to just go to run studies in very controlled, very safe environments where we can simulate and examine how it works and where is it helpful and where is it? Where is it not? And that's one of the tools we use to make sure that when we are deploying AI out into the world, we can deploy it in a way that's safe, that's adding value, that's not, you know, not uh— capturing as many of the upsides as we can without the downsides. -And when you say that AMIE is outperforming primary care physicians in some of the benchmarks that you've done, is that so— is it a patient would interact with AMIE at home and give them their preliminary symptoms in that type of hypothetical environment. And then AMIE is able to do a preliminary diagnostic. And in those settings, it's actually just as good, if not sometimes better than a primary care physician in some illnesses. -Yeah. So, we run different kinds of studies that the sort of stage of maturity of study we're running at the moment is roughly what you described. We'll compare two different groups of people. One group of people go through their normal process with their doctor with a telemedicine encounter. For the other group of people, they'll first go through this interaction with AMIE as a chatbot where they can type to AMIE. And we actually just released some new research where we start to experiment with video and understand, okay, if we we actually create a video interface between you and an AI, what happens then? But in this— in these scenarios, they'll experience the AMIE experience, they'll then go and actually be seen by their doctor the way they normally would, and then we'll look at what happened, what was better, what was worse, these kinds of things to understand the impact of the technology on the experience. -And what are some of those data points or some of the the benchmarks of performance? Do you have any of the numbers around it? -So in general, we find that both doctors and patients tend to like the AI when we get it right. It helps facilitate the patients' feeling informed when they go into the conversation with their doctor. The doctor in some scenarios will get a summary already of what's been discussed with the patient so they can kind of instead of starting from zero in their conversation, they'll get a better sort of preview summary, these kinds of things. We've seen in general, that these chatbots are surprisingly empathetic. This would be one of the more surprising things we found. We kind of expected that if you're interacting with a chatbot, it would feel cold, but it turns out that human language can express the right things for a conversation, even with a chatbot to feel empathetic. And these kinds of things. So it's been fascinating to discover what is it good at? What is it not? Where does it surprise us? -And when you said video, so it did— did people share different information when they were able to interact with it in a video format? And what are they seeing in the video? So in our experiment so far, we don't try to simulate a fake doctor or things. So they typically don't see much as a as a patient. It's more about the AI being able to see the patient, then kind of read the situation. I think longer terms this is, you know, distant future, we're hopeful that— we mentioned we talked a little bit about news like what can the AI see that people can't; it's not crazy to think there are some things just from a video that you could pick up that would help the AI better understand you as a patient. When I talk to doctors, a lot of them have this sort of gestalt of, you know, they can walk into the room and already know something about the patient just by looking at them. Oh, their color's a little off, these kinds of things. -Right. -We'd love to do studies to sort of explore over time, can the AI get good at these kinds of things, too? So if we are in a situation where, for whatever reason, someone can't have a doctor in front of them, they still gain the benefits of some of these kinds of things that they would get from a doctor. -And there's a lot to the sound, as well. You know, there's— when we're doing medical scribe, what we also found was that the voice has different kinds of conditions that come through. And the more studies we do on this in the future, the more I think you'll see that even the voice alone— so we're doing audio and visual; the visual side, there's a lot to be learned about your baseline and how you're changing, visually day- to-day. And from your speech; you're going to hear a lot in just the nuances that we-- actually, as humans we do pick up, but we don't think too much about it. -Wait, that is very interesting. So if the AI can see you and hear you, the changes in your voice, pitch, and tone, and how you're moving, and maybe you're a little bit jaundiced; all those little signals and cues the AI can pick up on. In the papers that I read about AMIE, you described it as being multi-agent. So AMIE, it would present as a single AI system if I'm sitting in front of it and I'm talking to it. But is it a team of AIs in a way that are kind of a panel of different experts within that same model that are having to come to agreement on what it is that it thinks you have? -Yeah, there's a couple different things we've tried. And the panel kind of version is one of them. The current version of AMIE, the thing we're finding that works best right now — and these things will change over time — is actually less specialization of the different medical specializations, but more, kind of a team trying to handle the encounter with one patient. So the current AMIE work focuses on... There's in this audiovisual AMIE, we talked about there's one agent whose entire job it is to pay attention to the video on the audio and sort of be registering what's going on there. There's another patient whose job is to communicate, it's communication essentially. What are we going to be saying? What are we doing? And there's a third agent. That's the reasoning agent that's sort of, you know, thinking about all these things and communicating with both the other agents. So, there are many different ways you can configure this. And it's one of the interesting areas for us to explore is how do you— Health is complex. Interaction with people is complex. How do you sort of bring a team of agents to bear on these problems in the most effective way? -If we're to step back and put all of this together, right? The pipeline. So you have a personal health assistant that's helping you with your behaviors, your lifestyle. Then we'll be wearing different sensors that are continuously picking up on different signals. What is this future that we're moving towards? -Well, the theme here, I think, is really about connecting all of this together and having the ability to— and we sometimes refer to this as a road model, but being able to actually, really comprehensively understand the progress from all aspects and dimensions of the human health and how that applies to the individual patient; and the constellation of symptoms that you have in just trying to figure out what your conditions are today. That's considered what we go to doctors for right now. Diagnose me, at this moment. In the model that would be like now casting, say predicting at t0, right? And this is where I think about the complexity of how we do that for things like weather models. Weather models for us, you know, trying to figure out what's going to be the weather for the next three days. Is a cyclone going to come? That's considered state-of- the-art, very challenging problem space. Why? Because there's so many different observational data sets that are important in feeding into that. Very complicated. AI is doing a fantastic job with that. And then if you want to forecast out 14 days, if you want to forecast out sub- seasonal to like multi years like what's going to be climate change several years from now. Huge problem challenge in terms of complexity. And for us, that's where a lot of the research, kind of frontier work is for us. Looking forward into that for human health, where you can actually forecast out, multi-year for what's going to happen, then those behavioral changes that you're trying to ask people to do in the near term won't seem so big. And you have that very strong connection point back to show them, people can potentially change very much their outcomes today and have agency in that. -So a weather forecast but for your health. The seven day, 14 day, ten year forecast. And then it's connected to your behavior and your lifestyle. You have wearables. You have different signals that an AI can detect in real time that you may not even have to think about. Yeah. -The other angle to this that I'm excited about is like the personalization angle. So a lot of the decisions that get made in healthcare right now are based on, on rules and diagnoses that were derived from very small scale studies that weren't particularly representative of all the people that were being applied to. So the other interesting angle here is, as more folks engaged with healthcare in a digital way, where where it's easier to sort of record what happens, what were the outcomes for for which patients we can I think then the medical community can bring that to bear back on medical science and start coming up with more tailored, more personalized recommendations for treatment, for when to diagnose, how to diagnose, these kinds of things. So I'm excited for a future where every time someone goes through a healthcare encounter, the information from that and the learnings from that kind of feed back into the system and helps dial in better outcomes for people like them. -Right. So what happened when you were seven is still a part of the systems analysis when you're 27 and that data isn't lost. It's cumulative. So we're moving towards a world where your DNA, your blueprint, will also be factored into the different signals that your wearables are maybe picking up on you. And then you could have a system like AMIE, theoretically, and you're engaging with it and it's monitoring all of these different data points. How does this come into the medical workflow? Because I see it almost like the invention of the microscope and the discovery of bacteria, where it was an entirely new world that previously was not a part of how the medical community viewed health. So now we're going to have all of these different signals that are coming in in real time, new types of data that were just unthinkable before. How does that get incorporated into how we think about health? -I'll say that I think the medical education system comes into play here in terms of how they train up their next set of doctors. I think it's become very important for the next generation to be able to understand, all of the probabilities, because to some degree, what we consider fact now may not be the same truth that we have, you know, two years when we have more scientific discoveries and a better understanding. And so how do we keep adjusting ourselves? Well, I think while the AI technology is coming into play, you can utilize it to help with that; that constant change of information. But you have to understand how it works. And I think there's huge benefits to both. understanding statistics, biostatistics in combination with what AI can and can't do. I think those pieces coming in early stage during medical education will be very helpful as we look into the clinical workflows of the future. -I think the other element here is the way people experience care today, I think often looks like for a lot of people, the first step is you go search or you go talk to a chatbot, right? Try to start figuring out what's going on with you. Then you go to to to one version of a doctor. Maybe in the US, it's a primary care doctor and you lose all that context, so the conversation you have and you maybe have to remember it and repeat it. And then maybe if you have to follow up with a specialist, you then go from your primary care doctor to a specialist. Then you've sort of lost the context of these two things, and you have to spin them back up again. And I think we can move towards a world where that happens much more seamlessly, where people are sort of guided and supported throughout that journey, and all of the information about them and what they're experiencing goes with them through that journey and helps the doctors around them bring the right care to what they're experiencing. -And I would say that there's— today, what's happening is usually there's some family member who is the caregiver and they collect all that information and they're the ones that are partnering with the clinician to try and help give the best possible care. And this can get offloaded. The caregiver's role can start being unburdened, hopefully in the future, where you can offload that to the AI assistant that's helping the caregiver. And that's sort of what Dale's describing here is that that will then hopefully be a partner to the clinician and to the family caregiver. -And you had mentioned that in medical school, doctors may need to start understanding probabilities of, and how AI model— what's in and out of the AI models distribution. And so how does the doctor's role evolve in this model. I think all roles are going to evolve with AI. And now you have this team of supercomputers around you, and wearables, and sensors, and all of these different things. We had chatted about synthetic biology, you know, a couple hours ago. How does the doctor's role evolve in this new ecosystem? -My hope is the most boring parts of doctors jobs will get automated. The things where they have to spend a lot of time doing a thing that's not very rewarding, and their time and attention can go to the things that they care most about, which I think is usually, you know, connecting with the patient and sort of understanding them at a holistic level, dealing with the tricky or more challenging cases. I think it's-- the promise of automation for doctors is largely, I think, going to show up in the mundane tasks that they don't really enjoy doing, that's my hope. -I think there's sometimes there's this fear that AI is going to come in and it's going to solve all the problems. But what you're actually describing is the role of the physician also becomes even more complex in some ways, because there's going to be this convergence of statistics and biology. There's new data sets to to take into account. It's continuous now, medicine;it's not just in time zero. You're going to have to factor in, "oh you have data now-- Seven years ago, on March 16th, there was an abnormal spike somewhere in their heart rate." All of that now comes into the picture. So it's not that things get easier, but they just change. -Yes. -Yeah. That's what I'm seeing. I'm seeing this in the software engineering all of which is you can do a lot more with your time. You can process a lot more information. You can you can create a lot more output. The people around you are doing the same. And so the the way you interact with other people do your job, which I think you know, for doctors will be caring for patients. It'll change or become in some ways more complex but hopefully more interesting. -And you know, we look at the durable skills of the future. And part of the durable skills is the critical thinking and the judgment. You still don't want to defer the judgment to the AI because the AI doesn't necessarily know what your goals are or what your values are. And again, it makes as it makes mistakes, too. So you really still need that ability to do critical thinking on top of it. You also, will— I do think, I hope this is the direction, which is that connection that you have between the doctor and the patient should grow. Hopefully there's an increase in empathy in this sort of like positive relationship between them, maybe a lifelong like relationship. That aspect of it, I think is also something that we talk about in terms of durable skills through communication and then creativity. That aspect of it is coming through on the life sciences side, because we're unlocking just such an opportunity and future for understanding human biology in a way that we've never been able to before. -Right. Imagine your doctor coming to the appointment with also your your genome, as well, and say, "Well, I see how your epigenetics have changed based on what you did in kindergarten." And you had mentioned sometimes AI can make mistakes. So how do you how do you think about trust in this model then? Because on the one hand, some of this is quite new for the physicians, these models that they'll be working with. And then these models also have their own limitations. They can hallucinate, they can sound super confident and be factually incorrect. So how does trust come into this healthcare model? -It's a great question. I think, so we tried we try to start from a place of trust where anytime we're putting something out there, we're trying to do pretty careful research. We talked a little bit already about running studies and safe situations to measure when the AI is successful, when it fails. I think it's important because trust is something that's very difficult to build and very easy to lose. And so essentially, the approach we're taking is study things as much as we can. Try to move as slowly as we can. You know, when things are already out there, people are already asking questions about their health, we kind of rush to study that, understand it, put some science behind it so we can make sure it's constantly improving and better supporting people over time. -And, you know, some in the past, people would look to paper publications as the outcome they were going for. But we see it as a means to ensuring that we get peer reviewed. feedback on the approach; have this more robust way of validating. Is the technology doing what we want? Also working with the community directly, the users themselves to ensure that we have a feedback loop so that the technology is going in the right direction. Because I often say that technologies is a means and not the end. And so you really have to have a very tight circle of back and forth with your users, and those are patients and those are doctors, to be able to ensure that you're actually designing the technology the way that would be most helpful. -And so how do you then set those benchmarks for an AI model? How do you decide this AI model we've built for healthcare is ready to go from simulation into pilot. Does it have to meet the performance of physicians? Does it have to exceed? Sometimes physicians do make mistakes, too. Sometimes they don't agree. So the idea of ground truth. So how do you decide when a model is ready to step into the game in healthcare? -The best approach is we start by looking at what are what are people asking, right. Understanding who's going to be using this tool; what kinds of questions are they going to ask? From there we work with with specialists, with doctors, clinicians to say, okay, well, when someone's asking this kind of question, what is the shape of the right answer? And this is— this can be challenging when it's a freeform text answer. Right? How do you describe a good answer? And so we spent a lot of time on answering that question. What are the properties that the answer needs to have? It needs to be empathetic in tone, it needs to be factual and correct. It needs to to sort of point the patient in the right direction. And we'll write up big rubrics around these. And then we work with clinicians to sort of try to run, run initially retrospective studies where we're looking at past chats and sort of simulating what would have happened and seeing if the model is doing the right thing on past data. Then we'll do prospective studies where we're saying, Okay, we're going to we're going to work with a group of actual patients in a safe setting to sort of go through this and make sure that the in production, the model is doing the right thing, and that our assumptions that we might have made about what's a good answer or not, play out with real patients. And only then do we think about deploying the technology out into the world. And even once we do that, we do monitor it afterwards to make sure it continues to perform in the right way. -You mentioned that there's doctors sometimes make the mistake to. And so there's sort of two different kinds of data that we look at evaluating. There's the data that's coming from humans and or labels. And we— that's where we use a lot of the adjudication where we make sure if we see discrepancies, we even built tools to help like resolve those discrepancies and allow for communication across the doctors to do that. But then there's also data that's just raw coming off of the sensors, and it's sort of reflecting what it's picking up in the— whether it's your signals and how do we do that. And a lot of it's that's just where all the statistics comes in. And we look for like area under the curve and maximizing that out. And so making sure that we have like very high percentage accuracies according to the outcomes that we're trying to evaluate for. And so it does depend on which use case you're using it for. But now we can approach on so many different dimensions that we have to benchmark all of them at the same time. -And you ask specifically about what's the bar? For us, it's approval from the healthcare community. Like we look to doctors and key opinion leaders in the healthcare space to understand what do they see as good enough, like what is the bar? Because they are the they are the folks that society trusts with their health. And so they're the right people to make decisions on when something's good enough or when it's not. -And I'll have to add to that, which we use the clinical guidelines, that's a good, strong baseline. But there's also this interesting phenomenon that's happening, which is those guidelines don't get updated with the most frequent information. Sometimes it can be seven years before they update it. And so there's this real opportunity to work with these experts who actually have access to a lot of this knowledge that's coming online, that's proven in trials. And bring that as the new baseline in terms of how you're going to perform for the AI. So those are things we're looking into. -And then how do you incorporate? Because sometimes the the art and the science of being an expert physician, it's getting the diagnosis right. But then it's also the art of the questions, right? And some of that comes into med school but not always. And so doctors are connecting these patterns. Oh, they heard that you were on a meditation retreat, or you were on a party retreat with friends for three months. That's going to light up a different set of questions from the physician. So how are you training AI models to bring that into the architecture? To know what questions to ask, to pull out different points. -When we saw that people were starting to use chatbots, not as a question and answer, but more of a conversation, we asked the simple question of what makes a good health conversation? What makes a good conversation in the context of health? And this is exactly what popped out of that analysis as we looked at all the things. And it turns out what makes a good health conversation is, you know, whoever's playing the role of caregiver or doctor here has to ask the right questions. Like, one of the biggest challenges is just getting the information out about the lived experience that that needs to be brought to bear on the health question. And so, so we trained we found that that was the missing gap with the early models. And we've spent the last year or so, trying to encourage the model to ask more questions in the right context, helping you, have clinicians help the model, ask the right questions and learn how to ask the right questions. And to Kat's point, bringing in the medical guidelines is one big component of that, because a lot of this is specified in medical guidelines of what are the right questions you should ask? So it's an ongoing journey, but I think it is that back and forth between the caregiver and the patient that matters. And that's a big area of investment for us. -Both of us went to Harvard Medical program, that was one of the first things they gave us was this paper that had all these biases that if you're a doctor, you really have to avoid going down any one of them. And you know, everything from like recency bias to, bias around, you know, prior demographics having similar things. And so there's so many different kind of biases. I was shocked by the number. But I think when we're what— when they're trying to train, what is the right question to ask? Part of it is making sure your questions are helping eliminate all those kinds of biases, as well. -Right. Each question also has to play a very important role. And so with this, I mean, we've talked about a really extraordinary future that we're moving towards and how exciting it could be. But I think for a lot of people I know, myself included. Who's going to get access to this future? So is AI, and we have all of these fancy wearables coming, is this going to be available to everyone or is it just a few? How do we make it accessible? And then the second part, how are you thinking about— everywhere around the world is different. So the demographics, we're here in California; the distribution of disease — all of that is different than, say, Eastern Europe or a region in India. So, how do you ensure that the AI models that you're building and the products that you're building can be used in these different contexts and can be used by more than just a select few. -So there's kind of maybe two angles to the access part. One is we do have products that are used by people around the world at Google right now. And so a big part of our focus is how can those products best support folks in their health? That's kind of one angle. The other angle is, okay, well, for the healthcare system, how do we help the healthcare system manage this? And on the healthcare system, I think the access does come back to the sort of foundational models, building those in a way that they can be used by folks who are local who do understand what matters in my part of the world for health? What are the things that you know that people in this part of the world need to know, need to understand, to navigate the healthcare system. These kinds of things. We can sort of push that to the developers and the people that are building the tools in those local areas, because they're the experts and that kind of thing. -Yeah. And an open source is something we play into a lot here, because it's both helpful for places where they need to have sovereign models to be able to really ensure that the highest quality care can get to those places. But also it's what allows for an ecosystem to be built together. And we did MedGemma, which is open source, but we also did Open Health Stack, and that's really targeted towards care that's being delivered in the underserved communities where they really can't go or travel to the places where there's high quality care. And so what we're trying to do then is bring the high quality care to the community healthcare workers, who are individuals in the neighborhood that are looking after their community. -Right. So by being an open source, not only does that make it a bit more financially accessible, it also allows you to be more adaptive to the environment that it's in. So that final last mile is in the hands of the region that the model is being used in, which I think is really important. And is there anything that you think in a few years from now we won't need to see a doctor for at all? I mean, or that there's something that AI is able to— there's a solution right in front of us, but we just can't see it. And maybe because we're looking at an illness in an incomplete way. But are the things that that you're seeing in the pipeline that, oh, just wait until this becomes scaled or reaches a certain intelligence point that it changes everything. -Today, there's already therapeutics that people are not able to access or able to address, like let's say, leukemia or like these cancers that are quite severe. And it's partially because to produce them, that therapeutic, is actually very costly and it's hard to get right. So I think you're going to see that role start to shift as well in terms of there's this acceleration that we're seeing happen across the board is happening in every field. So we should be able to see some of the production for therapeutics accelerate. And so there's a future where I hope that we don't even need to be treating these chronic diseases because we knew about them and we knew what you could do much earlier on, similar to brushing your teeth. You know, people probably, back in the day were like, "What can I do about the fact I have no more teeth?" And we just said, "Well, if you just brush every day, you will actually be able to keep your teeth for a very long time." -And what's the bottleneck between now and that vision? Is it the intelligence of models? Is it more data that you need? Is it different types of hardware to get to a point where chronic disease— we can see which ones don't ever have to be experienced by anyone, because there are little things that we can do that change everything. -I think I'll tie it back to behavior change. So behavior change is the lever— the biggest lever that you can use to improve health for the next 20 or 30 years, because most of the disease burden that we experience is associated with diseases where there's a strong like environmental or behavioral component. And so I think in order to unlock that, we need to figure out the right interface between AI and people, like how does it show up in their lives in a way that helps them change themselves and benefit themselves and achieve their health goals? There's some elements of that may be interesting in new hardware. I personally think, like, there's probably a future where I would love for something to be able to track what I eat, for example. Right? And just to know and I a while ago, I got a continuous glucose monitor for a few weeks and that was super eye opening to see. You know, I knew in theory that when I eat a certain kind of food, it spiked my sugar. But it's very different to eat a food and watch a graph go up. And I'm like, oh, this is like real for me. So I'm hopeful that there's additional kinds of hardware that will help you monitor yourself and understand what's going on with your own body. Because for me at least, that's a big component of behavior change, I think. -And I think the behavioral change has been overlooked, partially because it's operating at your tissue and your organs and your, like, cellular level. And we haven't had all the connection— dots connected from every stage of that in the human biology. So AI and the road model that I think is where we're headed to in the future has the potential to make those connections, which again, makes it easier to then come back to the behavioral change. The sensors are really important because it really is eye opening to see it happen live. But then knowing what that means when you see a spike, when you see a change in your baseline, that's the part that's super exciting. -And what drives each of you in health at the intersection of health and AI, because you've both been working in it for years. What is the drive? What's the motivation here? -I started thinking about it, interestingly enough, I'm very close to my grandparents, and when my grandfather one day became hemiplegic and had a purple ear because he was old, the healthcare system was like, "Well, he's just old." I was like, well, clearly it's more than just old, but but I think it then led to this path where I was trying to understand what had happened to him. And interestingly enough, it was actually a medical assistant that offhandedly asked, did he have shingles at some point? And then I went and did all this research on Google at the time about what that could possibly be. And it was a study in UCLA that found where it had the exact same symptoms, and it was shingles in this in the brain cerebral, that can cause it. It's treatable so if he knew about it earlier, he could have done things about it. And so I think that there's several things here. One is the power of a care team, which is many different people observing the person. There's also the aspect of being empowered as a caregiver, where I could go to Google Search and be able to actually find the information that was relevant, and also being able to then say, is there something that could have been done sooner and could I have acted sooner on it? So, those are all aspects they think we're trying to address even right now. I'm very excited about because I think it's very doable. -For me, I started my career in the biological sciences, and then I came to Google, and in the first two years I spent at Google, I spent in Google Aearch. And it was really interesting to watch how with Google Search, we've sort of built an engine where every time somebody goes on search and searches for something we can we can try to measure whether they were successful in that search. And then we look across millions and millions of people and say, okay, here's the kinds of people that or the kinds of experiences that were successful. And we make more of those kinds of experiences. And this allows the product to improve over time. And then I started working in healthcare, and I saw some of these examples of where that process doesn't work in healthcare. The time it takes for information about one person's journey to inform the next person's journey is decades. And so for me, the motivation is, you know, we've done it for Search. We've got to be able to society to figure out how to do this for health and medicine. How do we get to a world that looks more like Search where where every every patient experience informs the next one and improves the next one? And we have kind of a worldwide engine that's optimizing for better health. -Well, I think I'll certainly be counting down the days till I get my first weather forecast for my health. Kat, Dale, it's been a pleasure. Thanks so much. -Thank you. -Thank you.