Variation in Resource Use And Life Saving Skills Among ICU Specialists

Here I present results of an analysis of both resource use and patient outcomes amongst ICU specialists done using data mining software within my former hospital's Electronic Medical Record.

Please read my first post to understand the background, context, and methods that produced the results I will present below.

VARIATION IN ORDERING CAT SCANS AND ECHOCARDIOGRAMS

Recall from my first post the “costs” of a CT scan for an ICU patient: radiation, the risks of transport, and the radiologist’s fee. But CT scans also deliver real benefits — they’re the most accurate, efficient test for dozens of life-threatening conditions.

So you could argue the more CTs you order, the better a diagnostician you are, and the more appropriate your treatment. Or the opposite: that the doctors who order the most CTs are the most uncertain — the least sure what’s actually wrong — making heavy CT use a marker for inexpert care.

Each doctor on the ICU service I formerly ran was given a letter code for anonymity. As before, the data covers 18 months and an average of 130 patients per doctor (fewer for those who rotated in less often). Here are the sample sizes:

In my first draft, I kept myself anonymous, but it's more interesting to tell you which doctor I was — it lets me show whether my original hypothesis held up. As an ultrasound expert who could routinely image the lungs, heart, brain (eyes), kidneys, liver, and abdomen at the bedside, did I order fewer tests? I'm Doctor E. (Lower rank = fewer tests ordered.)

As you can see above, my hypothesis that I would be the lowest-ordering doctor for CT scans proved correct, except for CT of the brain (lower right quadrant). That is because the information I can derive about the brain using ultrasound is very limited; thus, my ordering of brain CT’s was right in the middle of the pack. Note that this is the opposite of the other organ CTs, which lend themselves very well to ultrasound diagnostics, and I was the doctor who ordered the least “official” CT studies as a result. You can also see a couple of other doctors who were consistently ranked almost as low as me (Doctor K and Doctor I). Both knew ultrasound as well.

Another striking result was the variability in using these tests. Lets start with chest CT’s with contrast ( upper right quadrant table). That exam is typically ordered to rule out pulmonary embolism. The highest ordering doctor (Doctor G) ordered 19x as many as the lowest? Whoa. That shows some serious discomfort with “diagnostic uncertainty” and/or is someone who cannot rely on the tools I did at the bedside; thus this forced that doctor to expose a lot of patients to both the risks of transport as well as the risk of IV contrast (while also unintentionally helping the radiologist put their kids through college).

Note that in the rankings above, “Doctor G” was consistently one of the top ordering physicians. Without revealing too much, Doctor G was one of the most senior and respected doctors on the service. At the risk of foreshadowing, in terms of their ability to save lives, they rank 4th from the bottom (data at end of post). This preliminarily suggests that ordering a ton of CT scans does not necessarily correlate with being a “better” ICU doctor.

Also, if you look a little more carefully at that data above, you find that the doc who ordered the most CT chest scans (Doctor “I”) ordered almost twice as many CT’s of the chest as the next ranked doc, making him a serious outlier in use of this test. Again foreshadowing, I will show you at the end of this post that Doctor I is the lowest-ranked doctor in terms of their ability to discharge a patient from the ICU alive. Another data point suggesting overuse of imaging is not correlated with better outcomes.

Now let’s look at some other metrics of resource use:

The one that jumps out is the variation in calling cardiology consults - a tenfold variation? Doctor E (yours truly) again near the bottom. However, this is unsurprising if you know that I actually passed the adult cardiology echocardiography Boards, something only a handful of pulmonologists in the country had done at the time (at the risk of bragging, only 65% of cardiologists themselves are able to pass!

Now what makes an ICU specialist somewhat unique is that they have to master critical illness states across a number of different specialties, like heart, liver, kidney, brain, gastrointestinal etc. Still, one might argue that asking for sub-specialty “help” would help improve patient outcomes. So perhaps those who call for more sub-specialty input are better at saving lives? Although we are getting ahead of ourselves here, I will say that the two docs who ordered the most consults are also ranked at the top in terms of ICU survival of their patients; however, beyond that relationship, I did not see a strong correlation amongst the other doctors.

Note that I am 3rd in terms of ordering ID consults, but that one is weird because.. I recall asking for very few personally. You need to realize that I worked in a teaching hospital and my fellows ran the ICU at night (with me covering remotely). Ordering ID consults is something my trainees could do (and did) all the time. Many times they were forced to do so by hospital policies supporting “antibiotic stewardship - we simply couldn’t get pharmacy to give us the antibiotic we wanted unless “ID” was consulted or approved it. So maybe I was just good at choosing antibiotics (or bad?)

Otherwise, if you look again at the above, I was AGAIN one of the lowest ordering providers of blood transfusions, arterial blood gases, etc. I think this was because I was obsessively conservative with blood transfusions, so that is not a surprise, and another thought is that some ICU doctors order more blood tests than others, leading to something I call “iatrogenic anemia” (i.e. caused by the doctor). So, if you are an aggressive “lab tester,” you probably have to give more blood back to the patient, no?

Also, in terms of arterial blood gases (ABG’s), I routinely relied on other data instead. Keep in mind, though, that most “ABG’s” in the ICU are ordered by my trainees at the request of respiratory therapists or nurses. Still, it looks like I influenced the amount they ordered.

Anyway, below is the final “rank order list” of the 13 ICU specialists I studied. I gave the doctor 13 points if they were ranked first in that metric and 1 point if they ranked last (first was assigned to the doctor who ordered “the least” amount of each intervention). I totaled up their ranking points across 15 ICU-based metrics.

One reason for this approach is that I strongly believe, from experience, that except in certain situations or conditions, more medical care and testing is NOT better. Thus in my rating of ICU doctors, “minimalists” were given more points than “maximalists.” Here is how they varied:

As you can see, the most resource-intensive doctor (Doctor C) used approximately double the amount as the least resource-intensive. Although you may think of me as being egotistical in identifying myself as the least resource intensive in my delivery of ICU care, this also should be unsurprising as it was my inspiration for the analysis - I suspected this was the case and wanted to see just how much less diagnostic tests and interventions I used (reminding myself that although it seems like I used less testing, in reality, I may have used just as much or more given I literally walked around with my ultrasound machine at my side, constantly “peeking under the hood” into the internal organs).

More important than the above is… how good was this ultrasound-centric approach at keeping people alive? My hypothesis is that, given the often absurd efficiency with which I could identify the underlying cause of illness at the bedside, I could initiate “correct” therapies earlier in the disease (which is literally one of the most important factors in surviving critical illness). In this next section, we will test that hypothesis.