The Unsexy Problem That Could Break Healthcare AI

For 37 years, Charlie Harp has been telling healthcare organizations they have a data quality problem. For most of those years, they told him he was wrong.

“People for a long time have been saying the quality of the data is fine,” Harp told us on Practical AI in Healthcare. “And they were right — they were looking at it from the perspective of their EMR, their ability to bill and schedule.”

The data was fine for its original purpose. It tracked medications, generated claims, and scheduled appointments. What it wasn’t fine for — and what nobody needed it to do until recently — was powering analytics, training AI models, and flowing between organizations through exchanges like TEFCA.

Harp, the founder and CEO of Clinical Architecture, spent decades building terminology and interoperability tools. He describes his early years as “Don Quixote jousting with the windmill.” Then AI arrived and changed the conversation overnight.

Measuring what we’ve been ignoring

About two and a half years ago, Harp developed a healthcare data quality taxonomy — a structured way to talk about what’s wrong with patient data. It breaks down into four categories:

Availability — Is the data there at all? Are all the parts present?

Accuracy — Is it in the right format? Does the NPI match its check digit?

Conformance — Are the codes in the code system we agreed on? You said LOINC, but you sent local codes.

Plausibility — Does it make clinical sense? Is the encounter date before the patient was born?

From this taxonomy came the PIQI framework (Patient Information Quality Improvement) — an open standard now going through HL7 balloting. It uses composable “Simple Assessment Modules” (SAMs) that chain together. If a lab result fails a conformance check, the system traces back through the chain to find the root cause. Maybe the code wasn’t in LOINC. Or maybe the field wasn’t populated at all.

The distinction matters. “Your labs were terrible” is useless feedback. “50% of your failures are missing data, 30% are wrong code systems” gives someone a path to fix it.

The numbers are not great

Real-world deployments of Harp’s PIQXL Gateway paint a sobering picture. Lab data quality averages about 70% against USCDI standards across assessed organizations.

One facility in a pilot deployment had mapped every single hematology test to a single LOINC code: glucose. Every CBC, every differential — all coded as glucose. Someone needed a LOINC code and picked one.

“We haven’t yet decided whether they’re sewer pipes or water pipes,” Harp said of healthcare’s data exchange infrastructure. “And I think we have to ensure that they’re water pipes.”

The information blocking argument

Harp raised a point during our conversation that stuck: if you share data through TEFCA that nobody can use, you are effectively information blocking — a regulatory violation the current administration takes seriously.

This reframes data quality from a back-office IT concern into a compliance question. Tie quality scores to economics — pay less for bad data — and you have a market mechanism for improvement.

“AI is much faster than we are,” Harp said. “And if we don’t give it something good, all it’s gonna do is be artificial stupidity at scale.”

He told us about the moment he decided to open-source the framework instead of keeping it proprietary: “We can make a product or we can make a difference. We are at a point in healthcare where we need something like this. Because if we don’t improve the quality of the data, we’re all gonna go down with the ship.”

Whether the rest of healthcare meets him there will determine whether AI in healthcare runs on water or sewage.

🎧 Listen to the full conversation: https://practicalaiinhealthcare.com/episodes/