IBM’s silicon trivia champ Watson is expected to do big things in medicine. Dr. Eliot Siegel, a radiologist who oversaw the first all-hospital PACs transition in the early 1990s, is working with IBM on Watson, and said the computer technology now has knowledge analogous to a “well-read medical student.” But he wants it to get smarter, and he thinks the technology might one day be able to help radiologists prepare for image interpretation and even, potentially, help with diagnosis and treatment. This is part two of our interview with Siegel. For part one on next-generation PACS, go here.

DBMN: I know you'd like to set up a Center for Computational Intelligence in Medicine at the University of Maryland -- how far along has that gotten?

SIEGEL: The center hasn't gotten very far because we're still trying to figure out whether the center should be centered in the school of medicine, or whether it should be on campus here, which is the professional campus, which would encompass pharmacy and nursing informatics and medicine, and potentially even law and dentistry. And so we're still trying to set that up administratively. But in the meantime, we've been working with the University of Maryland, the Baltimore city campus, and we've continued our talks with IBM and a number of different groups.

We're making slow but sure progress, trying to make it multi-campus and working to include not only the medical school but other schools on campus such as pharmacy and nursing and also to have it encompass the excellent computational resources and expertise at the University of Maryland undergraduate campuses.

Why is "computational intelligence" needed?

If you look at how hospital information systems communicate today and how they integrate, it's really amazingly limited. We're just in the nursery school stage of beginning to get our information into a digital format and starting to look at using existing standards to help with interoperability and making the record computer intelligible. Once we do that we can build intelligent clinical systems that can sort through the complex health record and recognize safety issues such as drug interactions and inaccurate or incorrect medications, become more effective by allowing automation of implementation of medical guidelines, and become more efficient by allow the computer to help synthesize and distill and interpret important clinical data.

Speaking of Watson, you've done some work with it...

As far as the project with Watson, I initially had the privilege to work with IBM, even before the "Jeopardy" match, using the technology that IBM developed, which they refer to as DeepQA, deep question and answer. What's different and unique about the technology is rather than having a set of rules or situations that are predefined, Watson software is able to take advantage of the system's tremendously parallel, computational speed, to be able to take a database that exists in a location and create hypotheses and mine information from that database on the fly, without having any preexisting rules or suppositions. The advantage of that is as new data become available, you don't have to rewrite new rules, and it's able to discover things that people may not have thought to program into a system.

How far back does your relationship with IBM go?

I had been doing research with IBM for a number of years on a number of things related to medical imaging, and they asked me if I'd be interested in talking with the team that was working on a "cool" project. So at that point, it hadn't been announced to the public, but they introduced me to the "Jeopardy" team and to some of the strategies they were employing. And I mentioned that I thought it would be a fascinating technology with some fairly profound implications for medicine as well.

What are some of those implications?

Well, even though most of their efforts were homed in on the "Jeopardy" match, we started with a subset of their developers looking at how IBM could enhance the database and the software's ability to be able to answer questions in the medical space. As time has gone on, IBM has tested it against different types of quiz questions, for example, that are posed to internal medicine candidates for certification...And its knowledge in the medical domain has increased. At this point then, it's kind of like a really super smart, well-read medical student, maybe first year, maybe second year, and what it really needs is much more empirical experience.

How does it get that experience?

We've been working with IBM recently on taking deidentified or anonymized patient data with IRB approval, triple-checking to make sure all patient information is stripped, and then having the IBM team begin to look at what would be required to do natural language processing, and to do analysis, not only on structured data, such as the lab data and diagnostic codes, but also to look at unstructured data, such as progress notes, discharge summaries, admission notes, and radiology and pathology reports. So that's where we are with IBM currently. And what I'm hoping to do is connect the VA's database of more than 28 million patients, which is referred to as Vinci, and to essentially bring to bear the computational processing power of the IBM DeepQA technology on that.

In what ways would this apply to imaging?

Well, what I really would like to do is start looking at how, from the medical imaging perspective, we might be able to extract important and relevant information from a patient's chart, for radiologists who need to see a synthesis or summary of what are the important and pertinent details. The history that we typically get is fairly limited. It might say a patient has a new fever, but what they don't tell us when they request the study, is that the patient has lymphoma and is HIV positive, and has had a recurring history of bouts with pneumonia, for example, and all the other things that would be really important for us to help make a diagnosis.

So what I'm really interested in from the imaging perspective as an initial step, is to simulate or emulate what in an academic setting my residents and fellows do currently - by spending a half an hour reading through the medical charts, talking with the patient, gathering information, looking at previous studies, collating all that knowledge and then presenting it to me in such a way that I can be more accurate and more efficient in making a diagnosis.

So you sort of foresee a DeepQA technology in radiology, at least at first, acting as sort of a synopsis-maker - giving the doctor a quick capsule medical history of the patient?

One of the things I'm going to show at the Dwyer lecture is a kind of a timeline where there are three pictures of a stick drawing of a patient, and some of this concept was originated at Massachusetts General Hospital by Dr. Supriya Gupta. She essentially thought of the concept of having a little graphical stick figure, and on that stick figure you can create systems drawings. You can create a lung nodule, or a brain tumor or a myocardial infarction, and then from one image to another, you can see whether the tumor's getting bigger or smaller, whether that problems' gone away, and so in maybe five or 10 seconds, glancing at three pictures, I can essentially have a summary that I can drill into if I want to, that so much supplements my just looking at a new imaging study cold.

Just the creation of that synopsis itself could change the way we practice radiology to a substantial degree by making important and relevant data available to the radiologist routinely. And I just see that as phase one.

What are the next phases?

At the VA there is a data warehouse of structured and unstructured information on all patients that goes back more than 12 years. So now the VA has billions of transactions or bits of data on patients, that there's the capability to mine, either in real-time or preprocessed. So step two after being able to synthesize the data and summarize it and create those synopses, is actually to have the computer make suggestions. The first small step would be to have the computer look at discrepancies. To say, hey, here's a problem list, where it says the patient is diabetic, but I can't find any recent examples of the patient's glucose being abnormal. Or here you have the patient essentially diagnosed with having myocardial ischemia, but all of the cardiac studies that have been done seem to suggest that's not the case...

And the next step after that would be diagnosis: "Hey doctor, have you considered A, B or C, and here's some evidence that would suggest those possibilities." And of course there's the potential for the computer to provide treatment options... I'm hoping to do some work to help push the envelope with the IBM implementation of the software and move it further away from being just a question and answering device, and further away from being just a super-focused medical textbook, but to also have it start to do these tasks related to synthesis, vigilance and surveillance and then ultimately diagnosis and treatment.

You mentioned earlier the billions of transactions recorded in the VA's archives. What's the big obstacle of getting that knowledge to Watson - scrubbing the identifying information?

Scrubbing identifying information from the data is a huge and critically important issue. We have spent weeks just scrubbing data from 10 patients, having to check, recheck, triple check, because you can scrub the name of the file, but somewhere hidden in a progress note, someone refers to Mr. McGillicutty, or Mr. McGillicutty's next door neighbor Mr. Jones. There's a lot of identifying information in unstructured data such as progress notes or discharge summaries that's difficult to scrub. Our recommendation to IBM is to consider bringing the technology in house. So once it's inside the VA's firewall, then it just becomes another information system that's mining data, so you don't have to deidentify the data if it's just essentially mining the data in order to be able to answer your questions.

Is that going on now?

We're in the process of getting an IRB, investigational review board approval, for being able to mine those data. We're applying for that at the University of Maryland. We've already talked with the Vinci team who already do really impressive natural language processing for health services research and development, using some other software developed by IBM that does NLP. The VA's Vinci developers and experts are really enthusiastic about the potential of this rich database; and systems that are fast and sophisticated enough to extract the data in real-time for patient decision-making would be a tremendous addition to the current research applications and could create the next generation in medical information systems and decision support.