Run the Model Yourself: A Clinician’s Guide to Local AI
This one is a roundtable no guest, just the three of us. Dr. Junaid Kalia sits down with co-hosts Ed Marx and DR GPT™ Harvey Castro for a hands on, slightly geeky conversation about something most clinicians have never been told they can do: run capable AI models yourself, on your own hardware, without ever sending a patient’s data to someone else’s cloud.
It starts with a provocation Junaid running a local model on a $2,000 Mac at ~100 tokens/second and asking why he’d keep paying Azure for the privilege. From there the three of them walk through the whole terrain: why Harvey now updates his emergency medicine book with an AI agent every month, what “edge vs. near-edge vs. far-edge” actually means, whether an Apple laptop can really keep pace with a data center, and the uncomfortable catch that the best cheap open models right now come out of China, which opens a security and standards question the U.S. hasn’t answered. They close on the part that matters most in medicine: what a model can never show the FDA, and why a rural clinic with no road to it might be the best argument for local AI there is.
What you'll discover
- A $2,000 Mac at ~80–100 tokens/second (why bother going local?)
- Welcome + why write a book when an AI can rewrite it every month
- Who the book is really for: patients (and why the ER is a lot of people’s primary care)
- 13 minutes with your doctor, how AI fills the gap (and visits trending down)
- The home setup: 3 Macs, 128 GB unified memory, one tailnet
- The model onslaught: what you can run for free (routers, Qwen, Kimi K2)
- Edge, near-edge, or cloud, where should the model actually run?
- Can an Apple laptop out-run a data center? (Apple Silicon + MLX + Core ML)
- The catch: the best cheap models are Chinese and who then owns the standard
- Boeing vs. Airbus: the same fight, now national, now for AI
- What a model can never show the FDA (SBOM + prompt injection)
- The rural clinic more advanced than any academic center
- Three things you can do with AI today
Referenced in the episode
- DR GPT™ Harvey Castro’s AI in Emergency Medicine book + the “Pivot or Perish” series
- Open-weight model families discussed: Qwen (Alibaba), Kimi K2 (Moonshot AI), DeepSeek
- Apple MLX + Core ML on Apple Silicon; Tailscale for a private multi-Mac “tailnet”
- Inkling (referenced by Junaid) — a fine-tuning/infrastructure approach for bring-your-own-data models
- Signal & Symptoms Ep 38 — “Why Will Digital Twins Change the Way We Treat Patients?”
Medical disclaimer. This conversation is for general informational purposes and is not medical advice. Listening does not create a doctor–patient relationship. The tools and models discussed are general-purpose technology, not cleared medical devices; nothing here is an endorsement of any specific vendor or model for clinical use. Do not enter protected health information into consumer AI tools.