Would you consult a Michelin Guide if you were looking for the closest pizza place? No. But for people who are seeking a “once in a lifetime” dining experience on their special vacation, finding the right 3-star Michelin restaurant may be just what they want.
Michelin has a storied history and promotes their strict system for evaluating restaurants via their publishing & promotional efforts. Still, most consumers who are not familiar with Michelin’s methodology would probably guess that a 3-star rating isn’t so wonderful, compared to 4 and 5-star ratings doled out by so many other restaurant reviewers.[1]
My point is: when it comes to ratings, it is critical to know what universe is being rated and the methodology used to calculate the ratings.
On that score, the CMS Quality Star Ratings for hospitals offer an incremental step forward in improving the value of the Hospital Compare data to consumers. If nothing else, the Quality Star Ratings generate attention, which can lead to further research that uncovers richer information on which to base decisions.
Ratings and rankings of products and services will always be imperfect. So, why are ratings so popular? In part, because they fill an information gap for data that either aren’t available or aren’t easy to summarize because of their complexity. In essence, ratings are a signal of comparative quality and often a proxy for missing data.
Measuring and comparing quality among healthcare provider organizations presents an especially thorny problem as described by Andy Oram in a 2-part series on measuring quality[2]. One pertinent extract from part 2 for which I provided some input:
We are still searching for measures that we can rely on to prove quality–and as I have already indicated, there may be too many different “qualities” to find ironclad measures. McCallum offers the optimistic view that the US is just beginning to collect the outcomes data that will hopefully give us robust quality measures.
What is the overall objective of the CMS Star ratings? In essence, the star ratings serve to condense the information in the Hospital Compare database and improve the usefulness of that data by making it faster and easier for consumers to assess quality of hospitals in a comparable manner. The hospital star ratings also complement other Star Rating initiatives from CMS that cover nursing homes, dialysis facilities, home health care ratings and health plan finder ratings.
The composite star rating for hospitals is based on 7 quality categories:
|
1. Mortality |
22% |
|
2. Safety of Care |
22% |
|
3. Readmission |
22% |
|
4. Patient Experience |
22% |
|
5. Effectiveness of Care |
4% |
|
6. Timeliness of Care |
4% |
|
7. Efficient Use of Medical Imaging |
4% |
Categories were chosen to align with CMS Hospital Value-Based Purchasing (HBVP) program. The right-hand column lists the weights that were assigned each category in calculating the composite rating. Note, the methodology document describes how the weightings were calculated in more detail and should be consulted by those who really want to dig into the details. (See the Fact Sheet from CMS for a description of the ratings and a link to the methodology report: https://www.cms.gov/Newsroom/MediaReleaseDatabase/Fact-sheets/2016-Fact-sheets-items/2016-07-27.html).
The composite ratings and the underlying measures remain limited to values that are currently measured by HEDIS, HCAHPS surveys and other quality initiatives. As I mentioned in Andy Oram’s article, measuring outcomes in a meaningful and comparable way is still in early stage. In the future, clinical outcomes measures will improve and the ability to measure “effectiveness of care”, for instance, will improve and that category will likely be weighted more heavily (it is currently weighted at only 4% of the composite rating).
Back in 2011, after a lively presentation by then US CTO Todd Park (all of his presentations are lively!) on the topic of Data Liberación, I wrote:
Park spent some time describing Healthcare.gov and HealthData.gov and how they can act as a resource for entrepreneurs. I loved his analogy between HealthData.gov and NOAA data. He told an anecdote of how someone once told him that NOAA is unnecessary because one can find the same data in a more user-friendly application on Weather.com. What the commenter didn’t realize is that NOAA data form the backbone of Weather.com. The federal government provides the data gathering, normalizing, and updating functions and then makes the data available to others who can overlay, combine, segment, analyze, integrate and distribute the data in any variety of mashed-up and improved formats.
The tradition of building data businesses on the foundation of federal, state, and local government data is strong. Savvy data publishing entrepreneurs have been digging deeply into government sources of data and providing new applications based on the data for centuries and new data products and services continue to emerge.[3]
The door remains open to enterprising data companies that are willing to do the hard work of aggregating and integrating data from multiple sources and presenting the aggregated data clearly for consumers. In most cases, a single source of information isn’t sufficient to guide consumer decisions. Proximity and referrals will remain key determinants of choice of hospitals for consumers. Furthermore, choice is restricted by whether providers are included a patient’s health plan network.
So, from my perspective, the new CMS Hospital Quality Star Ratings represent a step forward in the supply of source data and ratings methodologies that healthcare data publishers can leverage to publish and promote value-added quality guides and other transparency tools for healthcare consumers.
To close, I’ll quote the final paragraph from Oram’s article:
When organizations claim to use quality measures for accountable care, ratings, or other purposes, they should have their eyes open about the validity of the validation measures, and how applicable they are. Better data collection and analysis over time should allow more refined and useful quality measures. We can celebrate each advance in the choices we have for measures and their meanings.
[1] Priceonomics recently posted a good article on the Michelin Guides: http://priceonomics.com/why-does-a-tire-company-publish-the-michelin-guide/. Accessed 8/2/2016.
[2] Part II of Oram’s 2-part series focuses on assessing and measuring healthcare quality: http://www.emrandehr.com/2016/02/10/what-is-quality-in-health-care-part-2-of-2/, accessed 8/2/16.
[3] Leveraging the Liberated Data, Health Content Advisors blog, http://www.healthcontentadvisors.com/blog/2011/9/25/leveraging-the-liberated-data.html. Accessed 8/2/16.