The Hidden Risk of AI-Powered Clinical Decisions
Loading...
You already know not every AI tool belongs in your workflow. The real question is whether your filter is sharp enough to tell the difference.
In this episode, Dr. Junaid Kalia and Dr. Harvey Castro sit down with Dr. Hansa Bhargava, Chief Innovation Officer at Healio and a physician who has spent her career building trust at scale, to examine what trustworthy clinical AI actually looks like from the inside. They trace how the content integrity crisis predates AI, why clinician-in-the-loop development is the clearest signal of a tool worth adopting, and how evidence-sourced curation is the only real defense against sophisticated noise at the point of care.
The stakes go deeper than adoption. Who carries the legal weight when AI-assisted decisions go wrong? And are the physicians leaning into AI today quietly trading long-term clinical judgment for short-term efficiency? This episode doesn't evaluate the tools — it reframes what's actually at risk.
"When there are AI products in healthcare, one of the top three influencing factors that makes me trust that product is that there is a clinician, a doctor, a nurse at the table in the development of the product. Super important. Immediate trust for me."- Hansa Bhargava, MD
Loading...
What You’ll Discover
[00:00] Intro: Building Trust in Medical Literature
[02:02] Dr. Hansa’s Journey in Architecting Trust
[04:49] Her North Star: Better Patient Outcomes
[06:16] Is Clinical Research Already Compromised Before AI?
[10:34] Why AI Is Built to Agree With You
[12:59] Clinician-in-the-Loop is the #1 Trust Signal
[15:06] The Biggest Risk & Opportunity in the Age of AI
Transcript
Junaid Kalia, MD:
Good morning everyone. I am very grateful today for you to join us at Signals and Symptoms podcast. And we have our good amazing friend Hansa. Dr. Bhargava has actually had a significant long career in basically building trust in medical literature.
And she's going to help us understand in the age of AI what trust in medical literature means.
So Hansa, do you mind actually reintroducing yourself to the whole our audiences that what's your journey been and how did you get to this point?
And then we're going to start, essentially the today's topic, which is, architecturing trust in healthcare AI. And the idea is that how medical literature, which is by itself which going to come into it, you know, authoritatively and everything and discuss how we can move trust again.
Because honestly, we are losing trust in AI, and healthcare because of the hallucination problem. So start with your journey and then we'll move forward.
Hansa Bhargava, MD:
Yeah. Well, Junaid, thanks so much for having me. I really appreciate it.
Thank you for having me on this. Look, I mean, I'm a physician, just like the two of you are, and I worked in clinical medicine for a long time over.
Over 12 years actually, in many roles. And that included ER, as well as hospitalist, as well as even, you know, helping residents, training residents as well. Well, and a year in private practice. So I kind of took on a lot of hats at that point.
I was also involved in leadership over there and it's really interesting because, some of the leadership I did was actually work on protocol committees for, issues such as fever, in a newborn or concussion pathways. And so those are protocols and protocols are essentially what algorithms are. Right.
So, except that we were looking at the research and the evidence for a very long time before actually doing this. So, putting together guidelines or policy statements or protocols for hospitals takes, you know, a couple years sometimes. Right.
And now here we are in the juncture where we can do that, you know, in a snap, as we all know, because we can get the Literature immediately, especially with the generative AI tools that we're using, for physicians and scientists.
After that, I actually did kind of pivot to digital medicine, Junaid. And, the reason I did that is because I was really frustrated with the health system as it stood.
There are a lot of gaps in care. So I was fortunate to, take on the role of medical director of a partnership program with WebMD and a large hospital system in the Midwest, Sanford Health, kind, of looking at obesity.
And we built up three websites, we built up an app, we built up, like, digital tools to help patients engage where they were. So that's the problem, right? Like, we go into. We go in, patients go into a doctor's office, they see the doctor for 15 minutes, and then the rest of their journey is just out there, right?
And now, so using digital tools, you kind of follow them. And, then I went on to build mobile apps. We did some partnerships with Apple, with scripts, and basically, again, same idea. Meeting the patient, where they're at. That was at WebMD.
And that way we could deliver trustable information. Junaid, I'm going to talk a bit, little, little bit about trust here. I cut my teeth as a journalist at WebMD. Trust was the essence of everything they did. And it was super important to, have that trust.
As soon as you lose trust, you have nothing. And that's true of any product, any, publishing piece, anything else. And so, we basically made sure that we took a lot of resources, we had unbiased content, and that no matter what, whether we had partnerships with pharma or anyone else, there was a church and state.
The way we built our content did not have bias. And so you can take those principles into anything today. And I do, I do like when I write and when I produce content, when I build a product, it's super important to have authenticity and trust.
Harvey Castro, MD MBA:
I really want to go deep with you, in a sense that. What is your why?
I know you and I talk a lot, and we have a passion for AI and healthcare, but what is your why? Why are you doing this? On? Why should we listen to you? So tell me.
Hansa Bhargava, MD:
Yeah, great question, Harvey.
Look, I mean, the why is very similar to a lot of our colleagues. You know, we actually. I'm an optimist, and I do believe that most of my colleagues, my physician colleagues, have the same why, and that is we actually want to make patient health care better. And how do we do that?
And for me personally, it was seeing patient care having problems both as a physician, where, A, they couldn't get access to care, or B, we were just running around seeing patients in the ER Not spending enough time with the patients because we were overloaded with EMR or C.
After the patient left the room, what happened? What was the follow up? Did they actually take the medicines? I mean, I had patients taking the wrong medicine all the time, because they just. They and D. Like patient communication. Like, you know, when you give a patient tough diagnosis, like the child in my patient, room that I had to deliver the diagnosis of type 1 diabetes or tell the parents their child had cancer, you know what happens?
So the why is huge, right? Whether it's for patients or it's for clinicians, the clinicians have been overburdened for years. We know that.
We've seen the rates of burnout, and burnout still exists. So what if we've had a little bit of improvement? That's still up to 40, 50%. Is that okay? That one in two doctors has burnout? And the admin load that they had. So there were so many whys. But as a patient or having family members, myself, who've gone through the system honestly, and I'm sure You guys have seen this too.
I mean, it's been really tough with all of what we have seen and what I have seen and just looking at some statistics. We went to Aimed, we saw the keynote speaker talk about medical errors, talk about medical reconciliation issues.
There's a lot of whys, whether it's for the doctor or nurse or it's for the patient. We need to make a difference.
Junaid Kalia, MD:
Oh my God, So much to, unpack right now. So the problem statement today is that we can see that content generation is very easy and high, quality authenticated content is extremely hard.
And then, we actually call AI slop. That is extensively going through social media and everything, which again, erodes trust.
But then that's the whole point of this podcast.
Signals and symptoms that we can decrease the noise, increase the signal, and then put it back to the patient.
So first paint the picture for our audience again that how much true clinical research is also slop, because people don't understand that clinical research, goddamn it, is actually academicians trying to push publications so they can from associate, you know, assistant professor to associate professor and everything.
Before AI, the process of content generation, what you suggested, which builds trust, authenticity, etc. And then we're going to talk about AI. So do you want to give that sort of background to the community, our audience, that how does this look?
Hansa Bhargava, MD:
Yeah, I do agree with you, Junaid. There is a lot of sloth out there. And, the tough task is to kind of filter through that and make sure that at the point of the question that the right content is being generated that is actually true and fact based on
And so I think it's a harder task. I know Agentic AI can make a difference there. But look, I mean you've seen the articles about certain LLMs and patient information and how much there is accuracy. We've seen the cases of suicide or giving, you know, the wrong information to an oncology patient, you know, generated by certain LLMs out there.
So I do think that the ones that are, you know, medically based have probably a lot more trust in that they're only pulling from certain content now to your content, to your question, Junaid. Yes, they're pulling from PubMed. You know, they're pulling, you know they're pulling from drug resources, they're pulling from the news, the health news that's out there at, I mean Healio.
But you know, other organizations like Open Evidence and you know, all that's out there, are doing a good job in that they are pulling from restricted sources. Right.
But I do think that there is some value to peer reviewed journals and going through the evidence and looking at the data and having that rigor. So maybe I'm maybe I'm an optimist, but I do think the majority of information out there from PubMed, although what you said is somewhat true, it's probably good information, right?
Junaid Kalia, MD:
So first I concur with you that definitely PubMed is better than the general Internet.
Unfortunately what I'm seeing is that, that the recent, more recent articles that I'm getting to review, I mean the first step I do is actually make sure that I run it through my custom, you know, AI to ask them that hey, was AI used to write this article in the first place and then I would write to match it.
And Interestingly we're seeing 60, 70 again. That doesn't mean it's bad.
Ten years ago when I had to write it, you know, I had, I'm come from Pakistan, I mean my English is great and overall but still I would get an English grammar person to check my work before even I submit.
The level of rigor that was required was insane. And they're like six month process. So interestingly that that flow is going towards AI generating more research which is again we are actually looking at the advent of AI being at the PhD level.
Then the second part is that that publishing pipelines are getting broken.
But that comes to the opportunity part because when there is a mess, with the advent of AI, do you think this, all this mess is solvable?
Given that we now have this new generative AI technology, large language models, et cetera, they can basically go through each and every line, mark it out that this is fluff, this is not, etc.
Hansa Bhargava, MD:
Yeah, I mean, I mean a lot of what you say has, I do agree that there is definitely subjectivity, whether it's policy or whether it's publication. But I don't want to just, I think that we're walking a dangerous line, if we just say, oh well, let's just hand it over to AI.
And the reason is that the human in the loop and oversight of the human just being like having the blended intelligence as one expert has called it is, is super important Junaid.but AI by itself is, is not something that we should also lean into.
Certain AI engines are, are built on user engagement.
The algorithms want you to come back for more. Right. And so what does AI do? They say, oh yeah, yeah, like we're, we're going to agree with That a person asks a question and they, they want to be agreeable. Because you and I both would rather hang out with somebody who's agreeable than non agreeable.
Right. That's just human, human behavior.
The second aspect is that, you know, depending on where the AI is built and where the data is drawn from, and the data can change. Right. Things change all the time. The AI can actually have that bias.
So you're talking about subjective bias in publication policy. AI certainly can have bias too. And, you know, the AI that's built in the United States might be very different than the AI built in China, for example, or even in Europe. Right. With the regulations that exist in Europe. So, the AI is going to be different wherever you go.
And then of course, culture, like you and I are from different cultures. Yes, we're, you know, raised here. But, you know, that cultural influence probably has some impact on how we, you know, how we interpret things.
So I think it's very, you know, it's a tricky line that we walk. Most of the time it will pull in objective data, but it's super important to have the humans involved in the loop because we just don't know.
Lastly, Junaid, you probably know more about this than I do, but there could be shifts in the AI algorithms too, with machine learning and also with data change.
But I'm just saying that we have to be aware. You know, I am a very big AI optimist, and, you know, I believe in AI, you know, solving a lot of issues, but we we can't just hand it over.
And no, no, that was never the thought in terms of that handed over. But human in loop is absolutely important in terms of trust. But the way that AI needs to function is that, that what I'm trying to accomplish, and I, I'll demo it to you, is that even the reviewer actually gets AI help and then confirms in a user experience that matters.
Junaid Kalia, MD:
But the idea is that, that even at that point in time for me to be provided clinical decision support, how does that actually look like? And more importantly, whoever is providing the clinical decision support, how did they make it?
But what I'm saying is that we're going to talk about how the source is made and I think we need to bring the AI backwards. Not just the content production with trusted sources, but more importantly bring it back to the reviewer and the reviewer can actually review with the help of AI, constant citations, etc.
Junaid Kalia, MD:
The future of copyright in the age of AI is a big issue.
And then lastly, Helio is amazing.
And of course I trust you. And that's where it comes with me. I don't trust Helio. I trust answer. And that's where the trust problem also impacts. So the third one is you said human in loop, but human in trust production. How is that going to be impacting in an era of temporal shift of data, temporal shift of model, and then of course this onslaught of so many things.
Hansa Bhargava, MD:
So, so Junaid, you said something interesting. I, I trust Healio, but I trust Hansa. Right. That's what you said so I'm just gonna hit on that point. Thank you for that trust. What I was going to say is that that is another reason there needs to be human oversight and humans in development.
And I talk about this a lot. Like, when there are AI products that are in healthcare, what makes me trust that product?
One of the top three influencing factors is that there is a clinician, a doctor, a nurse at the table in the development of the product. Super, super important, immediate trust for me, you know, and investors have said, savvy investors have said.
The first question I ask is, you know, was there a doctor or clinician, you know, in the development of the product? If the answer is no, then I'm not investing that product.
there's so many AI healthcare products out there. Where do they fail? They fail in the implementation of it, right? And the scaling of it. They can have. You have the best shiny product out there.
And then lastly, the last part of why clinicians are super important in the product cycle is that what happens after you implement it and. And there's workflow, you have to have somebody monitor it, right? That's where, like, shifts can, can make a difference.
Who's going to carry the legal risk? Whether a hospital has the standard of care, AI or not, who is the risk of that? So for all of those reasons, the human is the clinician that needs to be in the loop, right?
And at Healio, like as a leader at Healio, that was something that was a very significant part of my leadership. Let's do the physician advisory board. Let's get the doctors at the table, let's have them actually vet everything we do. Super important in any product Junaid. Yeah.
Junaid Kalia, MD:
So Ansa, I completely agree with you. So, Hansa, last question. Of the day. Five years from now, a, what is the biggest risk and what is the biggest opportunity you see in this space of, trustworthy content becoming the baseline for clinical decision support, cdi, all of those.
Hansa Bhargava, MD:
You know, I think that it's important for AI not to, be completely in charge, that we need blended intelligence, we need clinicians completely involved from the very beginning to the very end to every single day that they use the tool. Super important.
I, do believe, Junaid, that the younger generation of doctors and nurses might be at risk, just like our kids are, for automation bias and all kinds of bias. Like, you know, eventually the human mind says, oh, well, the AI is right most of the time, so I'll just trust it all the time.
Junaid Kalia, MD:
So you're so on, point that not only future generations have, will have a skill gap.
The current generation is deskilled. Keep going. I'm sorry, I had to say it.
Hansa Bhargava, MD:
I really think it's a risk. And we need to address it now. Like these studies are coming up. But we knew it intuitively, right? Of course every generation is going to be deskilled, but how do we intervene?
And I think that's where that question lies with the medical school schools and, you know, the CME organizations and all of that. The second part of your question, the good is incredible. Like it's AI, is a tool. We can use it for good or we can use it for not good. And I think it could So, yeah, there's a lot of good, but I think we have to tread carefully.
I'm sorry, that's just my nature. I mean, it's. It's important to think before you leap.
Junaid Kalia, MD:
I agree 100%. Alan, thank you for your amazing comments. I love it. You're absolutely right. At some point in time, we as physicians need to figure out what coders and coding companies are responsible for.
They should become, at some point in time, liable, as physicians are. You can't keep everything shooting from my shoulder and using me as a thing. And I agree with the right. Learned helplessness is going to be something that is going to be an issue as, as Hans has said, not for physicians, but everyone as we decrease, democratize, LLM usage, but more importantly, democratize and demonetize intelligence itself.
© 2025 Signal & Symptoms. All rights reserved.