JAMIA Study Reports 22-68% Interoperability Across EHR Platforms: 7 Implications

by Vince Kuraitis, JD and Ian McNicoll, MD

A recent study of EHR interoperability found that 68% of data was “understood” when exchanged across different sites using the same vendor, but only 22% was “understood” when exchanged across different EHR vendors.

The study was published in the Journal of the American Medical Informatics Association (JAMIA). In this post, we will:

• Summarize the JAMIA study and its findings
• Interpret the findings
• Discuss possible solutions
• Describe seven implications

While we mostly agree on the study’s findings, we’ll offer some nuanced interpretations. Vince is a U.S. based healthcare consultant focusing on platform strategy and business models. Ian brings a European perspective – he is a former Scottish GP turned medical informatics expert.

Summarizing the JAMIA Study and its Findings

The objective of the investigation “was to quantify the interoperability of real-world EHR implementations with respect to clinically relevant structured data”.

The investigators “analyzed de-identified and aggregated data from 68 oncology sites that implemented 1 of 5 EHR vendor products. Using 6 medications and 6 laboratory tests for which well-accepted standards exist, [they] calculated inter- and intra-EHR vendor interoperability scores.”

As defined in the study, intra-vendor interoperability refers to the ability to share information between instances of the same vendor’s product (e.g., Epic > Epic). Inter-vendor interoperability refers to the ability to share information between instances of different vendor products (e.g., Epic > Cerner).”

The authors did not reveal the names of the five EHRs chosen, but they noted that the EHRs were selected from the following subset of CancerLinQ supported EHR vendors:

• Allscripts
• Varian Medical Systems
• General Electric
• Cerner
• Epic Systems
• IntrinsicQ
• Elekta
• NextGen
• Flatiron Health

They emphasize that “although we did not include all EHR vendors, our sample included 4 of the 7 most frequently implemented vendors based on expenditure-weighted Meaningful Use attestation rates of certified EHRs.”

Their summary of findings: “Overall, interoperability was relatively poor….The mean intra-vendor interoperability score was 0.68, compared to a mean of 0.22 for inter-vendor interoperability, when weighted by number of sites implementing the particular vendor’s product, and 0.57 and 0.20 when not weighting for the number of sites implementing that vendor’s product.”

“…we define ‘understood’ to mean that the specific text string used to define the meaning of the data element was used in the recipient site.”

The study also found high variability across and within vendors (Figure 4). We’ll refer to this figure in our discussion below.

Interpretation of the Study Findings

Initially, we have somewhat differing perspectives on the study’s definition of “interoperability”.

Ian: What was measured was the lack of standardization between the *internal* representations of clinical information of the various products examined. I think most health product developers would say that these internal definitions were never meant to be exposed to the outside world; they would expect to be measured against their level of external standardization, which they achieve by building adaptors to those external standards such as HL7 FHIR and SNOMED-CT.

Vince: HIMSS offers a broad definition of interoperability. I’m not a medical informaticist like Ian, so while his view could be technically accurate, I prefer to focus on practical interoperability.  I’m willing to accept the authors’ interpretation of lack of “interoperability”.

We broadly agree that are two likely categories of “causes” of the study’s findings of poor interoperability:

1) The inherent difficulty of standardizing complexities and idiosyncrasies of medicine as practiced in different locations, which results in inevitably differing interpretations among EHR vendors, and

2) A lack of incentives in the U.S. market to standardize terminology and to share data across various stakeholders. The lack of incentives is broad: EHR to EHR, EHR to provider, provider to provider. As we will note, these incentives are changing dramatically.

While we agree that both of these explanations are plausible and contributory, we place different weights on them:

Vince: I’d weigh the lack of incentives more heavily in explaining the findings. Care providers in the U.S. have viewed patient data as “their” asset. Fee-for-service reimbursement doesn’t incentivize appropriate sharing of data. Providers haven’t demanded interoperability, and EHR vendors have been slow to provide it.

Take a closer look at the data shown in Figure 4 (shown above). Two of the vendors have very high levels of intra-vendor interoperability:

B > B = .93

E > E = .98

My take on this is that it documents that intra-vendor operability is possible if vendors and providers do the work to standardize terminologies and data definitions.

Ian: What this study does reveal is the huge job that has to be done by system vendors to build the adaptors out to CDA/FHIR etc, and to make those internal representations interoperable. Often vendors are accused of blocking data flows for commercial reasons, but the intellectual and technical burden of analyzing the mismatches, agreeing consensus standards with others via FHIR etc., and then building and quality checking any software transforms, is enormous.

Discussion of Possible Solutions

In the U.S., there are many companies developing workarounds to the perceived weaknesses of EHRs. Exchange-based interoperability is improving thanks to innovations like HL7 FHIR,

However we jointly believe that a rather different approach, built around standards-based data platforms, would be the ideal solution to deliver a seamless patient-centric digital experience for clinicians and patients.

openEHR (pronounced “open air”) pioneered this new approach and is gaining traction in Europe, but seems unlikely to gain adoption in the market-based U.S. health care system. EHR switching costs are VERY high.

It is a set of open specifications and free, standardized clinical data components designed explicitly to support the data platform approach, separating applications from data and storing the patient data in specialized datastores called “clinical data repositories” (CDRs).

A similar approach is Vendor-Neutral Archives (VNA), which are common-place in the clinical imaging world for structured/coded clinical data.

We are now seeing attempts to build similar CDRs — but based on HL7 FHIR, e.g. SmileCDR and Zus. So regardless whether you believe in openEHR, standards-based data platforms and applications are becoming mainstream in both the US and Europe

To power these new ecosystems, we need to ramp up the global development of clinical content standards, The JAMIA paper highlights the enormous scale of that challenge. As an example, it mentions the lack of consistency between the various representations of smoking and tobacco
— “Tobacco use” is not the same as “Smoking use”, and neither are the same as “Nicotine use”.

All of these might need different information sets depending on whether one is doing a risk-assessment or keeping a daily diary of use. Teasing out these kind of issues needs detailed consultation with a wide-range of professionals, vendors and indeed patients.

Implications

So what? Let’s consider seven implications of the JAMIA study.

1) Patient Care Suffers When Decisions are Based on Incomplete and/or Inaccurate Data. The study documents that decisions are being made with incomplete and/or inaccurate data. It adds depth to the EHR patient safety challenges reported on in KHN’s Death by 1,000 Clicks.

2) We Have a Long Way To Go On Interoperability. The study’s authors correctly describe overall interoperability as “poor”. There is no doubt that we have a very long road ahead in understanding how to harmonize the very different ways that clinical information is collected in this highly complex ecosystem — even if all of the commercial and other barriers were magically removed

3) Real World Interop Could Be Even Worse Than the JAMIA Study Suggests. The JAMIA study relied on a narrow set of data — a subset of medications and lab tests from oncology data. These data sets are among the most standardized and mature. The authors note that “less commonly prescribed medications and less frequently checked laboratory values are likely to be even less interoperable.”

The problem might be less significant if the vast majority of patient data was confined to a single EHR, but that’s not the case. A study in International Journal of Medical Informatics found that “only 4.5 % of expenditure-weighted individual Medicare beneficiaries had their MU medical records associated with a single vendor, while 19.8 % of expenditure-weighted beneficiaries had their MU medical records stored in 8 or more vendors.”

4) EHRs Must Do a Better Job as “Platforms”. While all the blame here doesn’t fall on the EHRs shoulders, EHR vendors must take a stronger leadership role in assuring interoperability across locations and across vendors.

5) We Need More and Better Measures of Interoperability. Simply documenting the exchange of data isn’t enough. Data can be transferred from point A to point B, but that’s insufficient if it’s not “understood”.

The authors call for the informatics community to “develop a nuanced and relevant measure (or multiple measures) of interoperability.” We wholeheartedly agree.

6) Market Consolidation across EHR Vendors Will Continue. The burden of standardizing data helps drive smaller vendors out of the market. It makes many current EHR systems a poor fit for a data platform approach, where the platform must be able to adapt very quickly to the demands of its client applications and support new or rapidly evolving types of information.

7) Interoperability WILL Improve: There’s a Huge Upside. We want to end on a positive note.

In the U.S. and across the world, market and regulatory demands for interoperability are increasing. HITECH legislation, the 20th Century Cures Act, recent ONC/CMS regulations, and the growth of value based care (VBC) and value based payments (VPB) are all drivers. We expect that data sharing and interoperability will improve dramatically over the next decade.

We welcome your comments.