Build With How Clinicians Think, or Build Against Them

Most conversations about clinical AI start with the algorithm. Dr. Vimla Patel would rather start with the clinician.

Patel has spent four decades as a cognitive scientist studying a deceptively simple question: how do expert doctors actually think? The answer turns out to have direct consequences for every team building AI for healthcare.

Two directions of reasoning

Her central finding is that expertise has a direction. Expert clinicians reason forward. They take the signs in front of them and move quickly, almost without visible effort, toward a diagnosis. She illustrates it with a now-famous case: a young man arrives with puncture marks, fever, an eye hemorrhage, and a specific heart murmur, and the expert lands on acute bacterial endocarditis in seconds, using only the patient data.

Novices, and even experts working outside their domain, reason backward. They start with a hypothesis and work to justify it. It’s slower, and it’s what anyone falls back on when uncertain. If that sounds like Kahneman’s System 1 and System 2, Patel agrees the mapping is close, though her version is specifically about how expert knowledge gets built. The provocative part: forward reasoning can’t really be taught. It has to be grown, through years of exposure to patterns. Backward reasoning is what you teach along the way.

Why this breaks most clinical AI

Here is the design failure. Most clinical AI assumes the physician is reasoning in slow, backward, System 2 mode, so it delivers more: more data, longer differential-diagnosis lists, more screens to click through. But real clinical work runs on forward reasoning, and, as Patel puts it, “if you give them too much information, it interferes with the utility of actual performance.”

Her design rules are concrete. Surface patterns, not data dumps. Embed AI where clinicians already look, inside the EHR, and stop forcing them to switch context, because forward reasoning falls apart when it’s interrupted. And accept that there is no one-size-fits-all interface: the design that helps a novice will drag an expert down into novice performance.

The cost of getting it wrong

Patel’s second case makes the stakes visceral. An 85-year-old ICU patient received 316 milliequivalents of potassium chloride over three days. Every clinician made a correct decision at the time. The system failed them: lab values lagged behind reality, an order defaulted to run for seven days, and nothing flagged the accumulating dose. “The physician made the right decision at all time,” she says. “The system did not support the decision.”

Her phrase for it is the one to remember: when technology ignores how clinicians think and work, it doesn’t just fail, it quietly sets the stage for smart people to make dangerous mistakes.

Her safeguard for the AI era is counterintuitive in a field obsessed with speed. Build in friction. Make the system pause and verify when it’s uncertain, because confident, hesitation-free AI erodes the judgment we will still need when the tools aren’t there.

Listen to the full conversation.

You Might Also Enjoy

• S1E33 with Edward H. “Ted” Shortliffe, MD, PhD — A founding figure of clinical decision support (MYCIN); the direct intellectual lineage of Patel’s work on how machines should reason alongside clinicians.
• S1E39 with Sarah C. Rossetti, RN, PhD, FAAN, FACMI, FAMIA, FIAHSI — The CONCERN early-warning system, built by modeling how nurses actually think and work rather than bolting an alert onto the chart.
• S1E43 with Isaac “Zak” Kohane, MD, PhD — Who controls healthcare data in the AI age, and the values quietly baked into the clinical AI we deploy.