How Do You Measure Clinical-Grade AI?
No guest this week, just the two besties with a Healthcare AI Update. DR GPT™ Harvey Castro opens with the six questions he now asks of every healthcare AI tool, adapted from WHO and NIST: can I rely on it, who is responsible, will people actually use it, who sees my data, who can afford it, and what actually improves? Then he holds the evidence to that standard. A Kenya study where AI made documentation faster but showed no benefit for the patient. An odds ratio whose range includes “no difference.” A forecast, at 60% confidence, that within a year an AI will show lower morbidity and mortality than standard care. Dr. Junaid Kalia pushes on liability: if an AI flags a breast mass and the radiologist disagrees, who is liable, and when does not using AI become the lawsuit? Harvey’s answer is the hospital across the street. The episode closes on a live audience question from MaryAnne Rizk, Ph.D.: how do you measure responsible AI for clinical-grade use? Junaid’s answer is practical. Reset the expectations, because even clinical trials are not 100%. Give every model its own KPI. Keep a physician-verified golden dataset to re-score each new version.
What you'll discover
- Cold open: even clinical trials aren’t 100% (Junaid)
- Welcome: no guest today, just the two of us (Junaid)
- WHO’s six questions for any healthcare AI (DR GPT™ Harvey Castro)
- What the evidence actually shows: the Kenya study and the odds ratio (DR GPT™ Harvey Castro)
- Three questions every hospital CEO should ask (DR GPT™ Harvey Castro)
- The 60% forecast: AI that lowers morbidity and mortality by next year (DR GPT™ Harvey Castro)
- Who’s liable when AI and the radiologist disagree? (Junaid)
- Legal will change the culture: the hospital across the street (DR GPT™ Harvey Castro)
- Audience question: how do you measure clinical-grade AI?
- Reset the expectations: clinical trials aren’t 100% either (Junaid)
- A KPI for every model: Dice, F1, job and task completion (Junaid)
- Golden datasets and re-scoring every new model (Junaid)
- The close: it’s all about human plus AI (DR GPT™ Harvey Castro)
Referenced in the episode
- Six questions for every healthcare AI discussion (trust and safety, governance and accountability, culture and workflow, privacy and security, equity and access, outcomes and value), adapted from WHO and NIST guidance; framed on-air as questions, not a certification or validated score
- A cluster-randomized primary-care trial in Kenya comparing an electronic record with and without LLM assistance: documentation improved, primary patient benefit uncertain (one tested implementation; do not generalize)
- Reading odds ratios and confidence intervals: when the interval includes “no difference”
- FDA guidance in progress on evaluating AI-enabled clinical tools
- Model-specific metrics: Dice for image segmentation, F1 for medical coding, job and task completion rates for voice agents
- Golden datasets: physician- and multi-clinician-verified ground truth for re-scoring new models
Medical disclaimer. This podcast is for general informational purposes and is not medical advice. Listening does not create a doctor and patient relationship. Views expressed by the guests and co-hosts are their own and do not represent their employers. No specific vendor or product is endorsed. No PHI is shared.