Of course everyone wants performance to continually improve. If they didn’t then there would be no need for research projects, CME, conferences or even reviewing tough cases with your colleagues. As a group, we tend to be intrinsically motivated and work hard to continually improve the outcomes of our patients. So if that is the case, why wouldn’t all of us be completely supportive of “pay for performance” and the ubiquitous patient satisfaction surveys (HCAHPS, OAS CAHPS, CG CAHPS…etc)?
Measuring physicians’ performance
The first challenge in pay for performance is defining “performance”. How do you objectively measure the “performance” of a physician? Administrators assume that this is pretty straightforward. Can’t you just get patient satisfaction scores and outcome measures and maybe some administrative data like a number of patients seen per day and be done with it? Well, let’s begin with the patient outcome: How do you value the objective outcome of the patient as compared to the subjective outcome of the patient? And how does this relate to the patient’s satisfaction? The relationship between these three things are often independent of each other.
As an example, imagine the patient post-op from a PIP fracture-dislocation with no pain and near full strength and ROM but the patient is unhappy with the residual swelling…in addition, they are very upset because of rude nursing staff and difficulty getting a hold of the physician’s staff when the pharmacy wouldn’t fill their prescription.
The Challenge
The next obstacle is the consideration of the factors that affect patient outcomes. Of course, the surgeon is a component of this. However, the nursing staff, office staff, therapists, scrub technician, central processing staff and every other staff member that the patient either directly or indirectly interacts with has an impact on their outcome. Even the facility itself can have an effect on the outcome.
What about the patient-specific factors? It is logical that patient-specific factors such as socioeconomic status, family support, employment status, compliance, health literacy and a multitude of other factors all contribute in some way or another to an outcome. To express it mathematically: Performance = physician + hospital + health care staff + patient physical health + patient psyche + medical device company + weather conditions affecting patient + …….+, + all the hundreds and thousands of interactions between the components of the health care system.
In Systems Thinking we learn that it is a combination and relationship of all of these factors (The System) that produces the outcome. The surgeon is an important part, but just a part of The System. So, when you measure physician performance what you are actually measuring is The System in which the physician works. Therefore, how can you accurately compare one physician to another physician when they are taking care of a different set of patients and work in different clinics with different staff on different days…etc? Well…the simple answer is that you can’t. Your measurements are of the two different Systems. Therefore, you are not comparing the performance of one physician with another physician but rather the one System with the other System.
So far we have seen that: 1) measuring performance is much harder than administrators like to believe and 2) physician specific performance can not feasibly be measured. This is a concept that will take time to fully grasp and for some administrators will never be understood. Does this mean that we should not bother measuring the performance of physicians? No, data is very important. However, it is the analyses and the conclusions that we make where we must be very careful. If we can’t practically measure the performance of the physician independent of the System then we cannot reliably rank or rate physicians. That means, ranking a group of physicians in the top 5% or another group of physicians in the 30th percentile is fallacious.
Final Thoughts
If you can’t feasibly measure individual performance, then you can’t award an individual for their specific performance. This will undoubtedly unsettle many managers who rely on carrots and sticks to “motivate” employees. Performance data is important, but we need to know how to collect, quantify and analyze the data. Each data value is not an independent measure of performance (remember that performance comes from the System!). To collect and analyze performance data scientifically, and determine whether there is a positive or negative outcome related to a physician(s) requires application of the Shewhart theory and use of control charts for data analysis.
This sounds complicated right? Isn’t it just easier to carry on with the way that we’re doing it now? When the laboratory measures a patient’s HIV viral load, you better believe that they employ great precision and scientific analysis to make sure that the result is accurate and reflective of the true health state of the patient. Why wouldn’t we employ the same amount of science and knowledge when we rank and rate people?
Even if you could reliably measure the physician’s specific performance (independent of the multitude of other factors that affect performance) and could then pay the physician according to his/her “performance”, what is the point? How will this change behavior – short term and long term? Also, what is the Shewhart theory and control charts? Will this put me to sleep minutes after I start reading about it? How do we use this knowledge to make meaningful improvements and improve performance throughout all spectrums of the health care system? All of this will be covered in my next JHS Blog article(s).
