The Man Who Freed the Data Is Now Measuring Its Values
When you give a 13-year-old’s growth-hormone case to GPT-4 and tell it “you’re a pediatric endocrinologist,” it recommends treatment. Start a fresh session, change one phrase to “you work for the insurance company,” and the same model says don’t treat. Same patient, same facts, opposite answer. “Just by changing the prompt of the role, we switched it from give growth hormone to don’t give growth hormone,” Dr. Isaac “Zak” Kohane told us. “This is where money actually is gonna begin to talk.”
Kohane would know where the money flows. As Chair of Biomedical Informatics at Harvard Medical School and founding editor of NEJM AI, he has spent two decades building the infrastructure that AI now runs on. Our conversation traced two acts: freeing the data, and guarding its values.
The accidental standard
Act one starts in 2009, with an iPhone on Kohane’s desk and $50 billion in federal money about to flow into electronic health records. His pitch was simple: why can’t healthcare IT be modular like phone apps? “If I don’t like my camera app, I can get a new app,” he said. “But if I don’t like my order entry system, I have to fire the whole C-suite.” That argument became SMART on FHIR. Six of seven major EHR vendors adopted it. The 21st Century Cures Act later named it as an enabling technology for giving patients computable copies of their own records. Kohane calls the whole arc “better lucky than SMART.”
The new battlefield
That door, once open, let healthcare AI walk through. The surprise is who got there first. More than half of US doctors already use OpenEvidence, Kohane said, “voting with their feet” while health systems convene committees. The smart money two years ago was on the EHR companies. Now it isn’t, and the prize is the doctor-facing layer, because doctors direct the spend.
The values inside the machine
Act two is what worries him most. If a single role-word can flip a model’s recommendation, then payers and pharma will learn to tune that quietly. “The payers will be injecting system prompts,” Kohane warned. “No one’s gonna be so crass as to say, ‘We want to maximize income.'” The manipulation vector is already real: researchers have been caught hiding white-font prompts inside papers to sway AI reviewers, and Kohane expects the same tactic to reach clinical knowledge sources, where content gets written for the models rather than for humans. His response is the Human Values Project, which benchmarks the values inside clinical models the only way that works: not by asking them, but by measuring hundreds of their decisions against what human doctors actually do.
Compared with what?
The frame Kohane keeps returning to is the counterfactual. Critics judge patient-facing healthcare AI against an idealized doctor. But when most of his own residents at Mass General Brigham can’t get a primary care physician, the real alternative isn’t perfect care. It’s a six-to-nine-month wait. His elevator pitch to health-system executives: “Do not give your data away… But don’t be a pig. Bring your patients into it, make your patients your allies.”
The man who helped free the data is now measuring the values of whoever is about to consume it. “Compared with what?” may be the question that should govern every AI decision in medicine.
Listen to the full conversation: https://practicalaiinhealthcare.com/episodes/#S1E43
You Might Also Enjoy
- S1E40 with Jeffery R. Smith, MPP — The ONC view on healthcare AI regulation and transparency, including the rule Leon cites here: information-blocking protections now extend to a patient’s AI agents.
- S1E33 with Edward H. “Ted” Shortliffe, MD, PhD — A founding figure of clinical AI on the long arc from expert systems to today’s models, the lineage Kohane came up through.
- S1E41 with Hugo Campos — A patient advocate building his own tools for personal data access, the flip side of Kohane’s “make your patients your allies.”