But health systems by their very nature strive to be homogenous given their efforts to standardize practice, reduce clinical variability and manage technology with a high degree of commonality. Therefore, their data may be rich and deep, but it also contains difficult-to-account-for bias. They increase the efficiency of their business by having similar operating suites and imaging equipment, following specific protocols, measuring activity by certain criteria, and hiring and promoting people with specific skill sets. These practices bias the data in a variety of ways, resulting in models and algorithms that operate well in those environments, but not as well when exposed to the real world of varied approaches.

To develop reliable RWE, each type of data should be as multidimensional as possible. It should be diverse in patient characteristics — such as ethnicity, age, geography and economics — as well as in clinical settings which influence variables — such as equipment, procedure and supplies. The only way to ensure this is by pulling data from a large variety of facilities, geographies, care models and patient mixes.

Third, it is crucial to be able to access a variety of data types, including non-structured data where clinical value exists. One example is medical imaging, a data source which is clinically important enough to be called RWI. Other sources such as disease registry, claims data, pathology, text, reports, and particularly electronic health record (EHR) data will also be significant.

EHRs, in particular, are widely seen as solutions developed primarily for reimbursement, and secondarily to support clinical care. Evidence of this focus is seen in the maturity and stability of these systems in tracking procedure codes, billing information and order accounting in very discrete ways, but leaving much of the actual clinical data embedded in unstructured reports, PDFs and document scans. But it is precisely this data, such as physician notes and medical images, that provides most of the clinical context that is key to meaningful RWE.

Fourth, all of these various types of data must be curated, which includes activities such as normalization, standardization and de-identification. Healthcare’s persistent interoperability problems will also impede RWE. RWE cannot privilege one source of data over another simply because of conversion difficulties with an essential format, otherwise biases will plague the results.

Fifth, all of this data must be indexed to individual anonymized patients in order to successfully analyze medical interventions longitudinally. Patients may have gone to a variety of places for lab tests, pharmacy and clinical treatment at various times under the care of various physicians and their data may be stored in variety of places. The ability to accurately attribute all of these encounters to a single patient over time is essential.

More Voices