By Matthew Michela

There are three big hurdles in the way of effective use of Real World Evidence (RWE). Each of them is more complex, harder to deal with and takes longer to implement than most people think.

Fortunately, solutions for each one of these issues are also closer to being available than most people realize.

The three problems are:

• The vast size, scope and heterogeneity of the data required
• The difficulty in incorporating the results of machine learning (ML) and artificial intelligence (AI) algorithms into real-life clinical workflows
• The seductive trap of adopting proprietary technology stacks

Without an adequate approach to manage each topic, many RWE efforts will fail to achieve their promise of supporting innovation and improving safety in order to provide an understanding of outcomes in real-life clinical practice and risk exploding budgets and timelines.

Real World Imaging (RWI), or medical imaging, will be an increasingly important part of RWE. Radiologists must be aware of the ways their clinical practice can create RWI to mature evidence arguments, and how the results will affect their work.

The importance of heterogenous data
Healthcare is moving its focus from procedures to outcomes. RWE will be the way real-life clinical practice demonstrates these outcomes. Randomized clinical trials (RCTs) will continue to be the basis of clinical practice, but RWE will increasingly help show which directions are likely to create the most value and minimize harm when applied to large, non-controlled populations.

In order to provide the best long-term statistical validity, RWE should be based on a wide range of data sources.

First, it really needs to be a large enough volume of the right kind of data. The trick here is to realize that more is not necessarily better – unless it is more of exactly the right data. For RWE to start becoming usable, specific information about procedures on each patient is needed, as are enough patients within different clinical contexts. Given the technical and legal barriers in accessing and sharing medical data from even one setting, let alone many settings, it is easy to underestimate the cost and time delays experienced in acquiring enough data from different sources.

Second, each type of data must be diverse. Sometimes large health systems or hospitals will assume that since they provide a large number of a type of service in a year, and therefore have such large volumes of a type of diagnostic data, for example, they can themselves independently generate valuable RWE.

But health systems by their very nature strive to be homogenous given their efforts to standardize practice, reduce clinical variability and manage technology with a high degree of commonality. Therefore, their data may be rich and deep, but it also contains difficult-to-account-for bias. They increase the efficiency of their business by having similar operating suites and imaging equipment, following specific protocols, measuring activity by certain criteria, and hiring and promoting people with specific skill sets. These practices bias the data in a variety of ways, resulting in models and algorithms that operate well in those environments, but not as well when exposed to the real world of varied approaches.

To develop reliable RWE, each type of data should be as multidimensional as possible. It should be diverse in patient characteristics — such as ethnicity, age, geography and economics — as well as in clinical settings which influence variables — such as equipment, procedure and supplies. The only way to ensure this is by pulling data from a large variety of facilities, geographies, care models and patient mixes.

Third, it is crucial to be able to access a variety of data types, including non-structured data where clinical value exists. One example is medical imaging, a data source which is clinically important enough to be called RWI. Other sources such as disease registry, claims data, pathology, text, reports, and particularly electronic health record (EHR) data will also be significant.

EHRs, in particular, are widely seen as solutions developed primarily for reimbursement, and secondarily to support clinical care. Evidence of this focus is seen in the maturity and stability of these systems in tracking procedure codes, billing information and order accounting in very discrete ways, but leaving much of the actual clinical data embedded in unstructured reports, PDFs and document scans. But it is precisely this data, such as physician notes and medical images, that provides most of the clinical context that is key to meaningful RWE.

Fourth, all of these various types of data must be curated, which includes activities such as normalization, standardization and de-identification. Healthcare’s persistent interoperability problems will also impede RWE. RWE cannot privilege one source of data over another simply because of conversion difficulties with an essential format, otherwise biases will plague the results.

Fifth, all of this data must be indexed to individual anonymized patients in order to successfully analyze medical interventions longitudinally. Patients may have gone to a variety of places for lab tests, pharmacy and clinical treatment at various times under the care of various physicians and their data may be stored in variety of places. The ability to accurately attribute all of these encounters to a single patient over time is essential.

As key contributors to the data underlying RWE, radiologists need a seat at the table from the beginning to guide how this powerful and potentially contentious source of outcomes information is used in their practices and facilities.

The recent availability of de-identified medical image datasets is a tremendous step in the right direction, vastly simplifying the process of incorporating this type of data into RWE. While there is a lot of work to be done, the way forward seems pretty clear.

The primacy of clinical workflow
Algorithms sometimes seem to work well — until they leave the testbed and come up against human beings in the real world, in this case caregivers and patients. Many ML models have shown a lot of promise while in the hands of data analysts, only to fall short in the trenches of clinical settings, and thus fail to see any uptake.

Clinicians tend to be uncomfortable with black-box models that lack transparency and interpretability. They want to see and understand the reasoning used to reach the presented clinical conclusions. And they are right to be suspicious since such models can reach flawed conclusions.

But they can also reach unintuitive conclusions that are nevertheless clinically meaningful, and should have an impact on treatment and outcomes.

Every effort must be made to show the connection between the results and clinical practice. Radiologists can contribute by insisting on clear demonstrations of how the model reached its conclusions, and aiding in testing the results in practice.

Radiologists likely make more clinical decisions per day than any other medical specialty, analyzing up to 100 scans per day in 10- to 12-hour shifts. RWE, along with other technologies such as workflow orchestration, should assist in prioritization and decision making. It is particularly important that radiologists be attentive to the decisions being made, and how they affect radiology workflow.

The need for technological agnosticism
The third hurdle may seem relatively minor, merely a matter of procurement, but its effects on RWE uptake will be quite significant over time. There is a lot of potential profit in creating useful RWE and providing it to users in an effective form, so the field has attracted many innovative competitors.

Anyone who works in healthcare knows of the perils of being locked into a proprietary system with specific formats, interfaces and required behaviors. At first, each system promises a great step forward in increasing information, lowering cost and improving patient care. Over time, unfortunately as those solutions gain market share and consolidate into only a few vendors, a lack of flexibility appears, new features become less frequent and expenses increase. This cycle has been repeated many times in medical technology, with clinicians realizing that initial exciting short-term savings often turn into long-term expense coupled with reduced innovation.

The same will become true of AI (built with RWE) if purchasers are not careful. Vendors have already appeared with specific technology stacks. Time and money are always short, and the temptation to sign on to a workable solution built on proprietary technology interfaces can be powerful. It must be resisted.

The landscape of healthcare data is changing with extraordinary rapidity. The pace of change is likely to increase with no end in sight. Flexibility, expandability and the ability to evolve are key elements in any RWE solution, because what is state of the art today is obsolete tomorrow. Do not be seduced by vendors who promise benefits in exchange for holding your workflow and data hostage.

Nothing is easy, but most things are clear
For the foreseeable future, RCTs will still define the boundaries of clinical innovation, and right now RWE is a developing approach with a mandate to implement.

Effectively applying RWE at scale, requires some effort, but what that work entails is pretty clear. A lot of that work is already getting done, largely behind the scenes, and solutions are beginning to emerge that offer access to large de-identified medical imaging datasets.

Radiologists will be key participants in both data contribution and in deciding how RWE is applied in actual clinical circumstances. If they keep an eye out for these hurdles and know what they need to do when you get to them, RWE efforts will be rewarded with improved patient outcomes.

About the author: Matthew Michela is the president and CEO at Life Image.