What Happens When Your Healthcare AI Framework Meets the Real World?
We’ve spent 24 episodes of Practical AI in Healthcare building a way of thinking about what works and what doesn’t in healthcare AI. We’ve identified themes. Named frameworks. Developed a shared vocabulary.
Then we tried to check whether any of it held up.
Turns out we both saw the mismatch. The disagreement was about whether we should have seen it coming. Steve thought our framework should have held up better. I’d been expecting the cracks. It took us a good 10 minutes to even understand where the other person was coming from, which doesn’t happen often.
That disagreement became the crux of the episode.
What does it mean when your model doesn’t break — but also doesn’t fit?
None of our guests told us we were wrong. What they did was tell us stories that kept spilling past the edges of what we’d built. A domain-specific revenue cycle AI that outperforms broad frontier models by 30% on its specific task, generating real money for hospitals — but operating inside a reimbursement system that punishes efficiency. An FDA-approved retinopathy screener that works best for African American patients and Medicaid populations — the people least likely to have access to it. And a health system disincentivized to use it. We know how to screen. We know it works. But who’s going to pay?
These aren’t failures of AI. They’re collisions between technology that CAN work and systems that weren’t designed to absorb it.
The thing we didn’t see coming
We expected friction from the usual sources — regulation, workflow, data quality. We’d built our framework around those.
What we hadn’t accounted for was a threat to the knowledge base itself. One of our guests described AI-powered paper mills manufacturing fake articles (and even networks of fake journals) that cross-cite for artificial credibility. If the scientific literature that clinical AI trains on gets corrupted, it doesn’t matter how good your algorithms are. That’s garbage in, garbage out at the epistemological level — and our framework had no category for it.
The aperture problem
Steve put it well during the recording: “I am opening the aperture a bit because I think … we need a bigger model.” Not more themes. More dimensions. Our original framework was a list. What we need is something closer to a map.
The honest version of intellectual progress isn’t arriving at “we were right.” It’s arriving at “we were less wrong than we thought, and now we can see where the gaps are.” The greatest moments in science aren’t eureka — they’re “huh, that’s weird.”
That’s where we are. Huh, that’s weird. And we think that’s worth sharing, because too many podcasts and too many frameworks in this space pretend they’ve got it figured out. We don’t. We’re updating the model in public, and the discomfort of that process is the most honest and valuable thing we can offer in this moment of vertiginous change.