Variation in Resource Use And Life Saving Skills Among ICU Specialists

Here I present results of an analysis of both resource use and patient outcomes amongst ICU specialists done using data mining software within my former hospital's Electronic Medical Record.

Please read my first post to understand the background, context, and methods that produced the results I will present below.

VARIATION IN ORDERING CAT SCANS AND ECHOCARDIOGRAMS

Recall my discussion of the “costs” of ordering a CT scan for an ICU patient in my first post. Beyond the radiation exposure and risks of transport there is also the radiologists fee for interpreting the scan. But also know that CT scans can provide great benefits given they are literally the most accurate and efficient diagnostic test for many dozens of life-threatening conditions.

So, one might argue that the more CT scans you order the better a “diagnostician” you are and thus your treatments would be more effective and appropriate. A contrasting argument might be that those who order the most CT scans are the “most uncertain” or least knowledgeable about what is actually wrong with each patient and thus, those who rely more on CT scans might be a marker for inexpert medical care?

In the below results, each doctor on the ICU service that I was formerly a Chief of was assigned a letter code for anonymity. Again, data for each doctor covers a period of 18 months and was based on an average of 130 patients (some doctors had a smaller sample size if they rotated into the ICU less frequently). Here are the sample sizes for each doctor:

In my first draft of this post, I left myself anonymous but then decided it would be more interesting to tell you which doctor I was because that allows me to show you whether my initial motivation (hypothesis) for the study was validated. For instance, did I, as an ultrasound expert with skills that routinely allowed me to image the lungs, heart, brain (eyes), kidneys, liver, and abdomen order less tests? Lets look at the rankings in terms of who ordered the least amount of the below tests (note that the lower ranked you are on the list, the less tests you ordered). Know that I am Doctor E in the below analyses:

As you can see above, my hypothesis that I would be the lowest ordering doctor of CT scans proved correct, except for CT of the brain (lower right quadrant). That is because the information I can derive about the brain using ultrasound is very limited, thus my ordering of brain CT’s was right in the middle of the pack. Note that is the opposite of the other organ CT’s which lend themselves very well to ultrasound diagnostics and I was the doctor who ordered the least “official” CT studies as a result. You can also see a couple of other doctors who were consistently ranked almost as low as me (Doctor K and Doctor I). Both knew ultrasound as well.

Another striking result was the variability in the use of such tests. Lets start with chest CT’s with contrast ( upper right quadrant table). That exam is typically ordered to rule out the presence of pulmonary embolism. The highest ordering doctor (Doctor G) ordered 19x as many as the lowest? Whoa. That shows some serious discomfort with “diagnostic uncertainty”and/or is someone who cannot rely on the tools I did at the bedside, thus this forced that doctor to expose a lot of patients to both the risks of transport as well as the risk of IV contrast (while also unintentionally helping the radiologist put their kids though college).

Note that in the rankings above, “Doctor G” was consistently one of the top ordering physicians. Without revealing too much, Doctor G was one of the most senior and respected doctors on the service. At the risk of foreshadowing, in terms of their ability to save lives, they rank 4th from the bottom (data at end of post). This preliminarily suggests that ordering a ton of CT scans does not necessarily correlate with being a “better” ICU doctor.

Also, if you look a little more carefully at that data above, you find that the doc who ordered the most CT chest scans (Doctor “I”) ordered almost twice as many CT’s of the chest as the next ranked doc, making him a serious outlier in use of this test. Again foreshadowing, I will show you at the end of this post that Doctor I is the lowest ranked doctor in terms of their ability to discharge a patient from the ICU alive. Another data point suggesting overuse of imaging is not correlated with better outcomes.

Now lets look at some other metrics of resource use:

The one that jumps above is the variation in calling cardiology consults - a tenfold variation? Doctor E (yours truly) again near the bottom. However, this is unsurprising if you know that I actually passed the adult cardiology echocardiography Boards, something only a handful of pulmonologists in the country had done at the time (at the risk of bragging, only 65% of cardiologist themselves are able to pass!

Now what makes an ICU specialist somewhat unique is that they have to master critical illness states across a number of different specialties, like heart, liver, kidney, brain, gastrointestinal etc. Still, one might argue that asking for sub-specialty “help” would help improve patient outcomes. So perhaps those who call for more sub-specialty input are better at saving lives?Although we are getting ahead of ourselves here, I will say that the two docs who ordered the most consults are also ranked at the top in terms of ICU survival of their patients, however beyond that relationship, I did not see a strong correlation amongst the other doctors.

Note that I am 3rd in terms of ordering ID consults but that one is weird because.. I recall asking for very few personally. You need to realize that I worked in a teaching hospital and my fellows ran the ICU at night (with me covering remotely). Ordering ID consults is something my trainees could do (and did) all the time. Many times they were forced to do so by hospital policies supporting “antibiotic stewardship - we simply couldn’t get pharmacy to give us the antibiotic we wanted unless “ID” was consulted or approved it. So maybe I was just good at choosing antibiotics (or bad?)

Otherwise, if you look again at the above, I was AGAIN one of the lowest ordering providers of blood transfusions, arterial blood gases etc. I think this was because I was obsessively conservative with blood transfusions so that is not a surprise, and another thought is that some ICU doctors order more blood tests than others, leading to something I call “iatrogenic anemia” (i.e. caused by the doctor). So, if you are an aggressive “lab tester” you probably have to give more blood back to the patient no?

Also, in terms of arterial blood gases (ABG’s) I routinely relied on other data instead. Keep in mind though that most “ABG’s” in the ICU are ordered by my trainees at the request of respiratory therapists or nurses). Still, it looks like I influenced the amount they ordered.

Anyway, below is the final “rank order list” of the 13 ICU specialists I studied. I gave the doctor 13 points if they were ranked first in that metric and 1 point if they ranked last (first was assigned to the doctor who ordered “the least” amount of each intervention). I totaled up their ranking points across 15 ICU based metrics.

One reason for this approach to ranking is that I strongly believe, from experience, that except for certain situations or conditions, more medical care and testing is NOT better. Thus in my rating of ICU doctors, “minimalists” were given more points than “maximalists.” Here is how they varied:

As you can see, the most resource intensive doctor (Doctor C) used approximately double the amount as the least resource intensive. Although you may think of me as being egotistical in identifying myself as the least resource intensive in my delivery of ICU care, this also should be unsurprising as it was my inspiration for the analysis - I suspected this was the case and wanted to see just how much less diagnostic tests and interventions I used (reminding myself that although it seems like I used less testing, in reality, I may have used just as much or more given I literally walked around with my ultrasound machine at my side, constantly “peeking under the hood” into the internal organs).

More important than the above is… how good was this ultrasound-centric approach at keeping people alive? My hypothesis here is that, given the often absurd efficiency at which I could identify the underlying cause of illness at the bedside, that would mean I could initiate “correct” therapies earlier in the disease (which is literally one of the most important factors in surviving critical illness). In this next section, we will test that hypothesis.