Original Article
Measuring Hospital-Wide Mortality—Pitfalls and Potential
Simon J. Mackenzie,
Don A. Goldmann,
Rocco J. Perla,
Gareth J. Parry,
Simon J. Mackenzie
Search for more papers by this authorDon A. Goldmann
Search for more papers by this authorRocco J. Perla
Search for more papers by this authorCorresponding Author
Gareth J. Parry
For more information on this article, contact Gareth Parry at gparry@ihi.orgSearch for more papers by this authorSimon J. Mackenzie,
Don A. Goldmann,
Rocco J. Perla,
Gareth J. Parry,
Simon J. Mackenzie
Search for more papers by this authorDon A. Goldmann
Search for more papers by this authorRocco J. Perla
Search for more papers by this authorCorresponding Author
Gareth J. Parry
For more information on this article, contact Gareth Parry at gparry@ihi.orgSearch for more papers by this authorFirst published: 07 August 2014
Abstract
Risk-adjusted hospital-wide mortality has been proposed as a key indicator of system-level quality. Several risk-adjusted measures are available, and one—the hospital standardized mortality ratio (HSMR)—is publicly reported in a number of countries, but not in the United States. This paper reviews potential uses of such measures. We conclude that available methods are not suitable for interhospital comparisons or rankings and should not be used for pay-for-performance or value-based purchasing/payment. Hospital-wide mortality is a relatively imprecise, crude measure of quality, but disaggregation into condition- and service-line–specific mortality can facilitate targeted improvement efforts. If tracked over time, both observed and expected mortality rates should be monitored to ensure that apparent improvement is not due to increasing expected mortality, which could reflect changes in case mix or coding. Risk-adjusted mortality can be used as an initial signal that a hospital's mortality rate is significantly higher than statistically expected, prompting further inquiry.
References
- Advancing Quality Alliance. (2011). North West Mortality Reduction Collaborative. Phase 1 Summary Report 2011. Retrieved February 25, 2014, from http://217.33.237.40/document_uploads/Rachel-documents/AQuA%20NW%20Reducing%20Mortality%20Report%20Phase%201%20output%20report%20June%202011_9b964.pdf.
- Agency for Healthcare Research and Quality. (2009). Presentations from a November 2008 meeting to discuss issues related to mortality measures. Retrieved February 25, 2014, from www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/mortality/.
- Black, N. (2010). Assessing the quality of hospitals. British Medical Journal, 340, 933–934. c2066.
- Campbell, M. J., Jacques, R. M., Fotheringham, J., Maheswaran, R., & Nicholl, J. (2012). Developing a summary hospital mortality index: Retrospective analysis in English hospitals over five years. BMJ, 1, 1–11. e1001. doi:10.1136/bmj.e1001
10.1136/bmj.e1001 Google Scholar
- Downar, J., Sibbald, R., & Lazar, N. M. (2010). Ethical considerations for classifying patients as “palliative” when calculating hospital standardised mortality ratios. Journal of Medical Ethics, 36, 387–390. doi:10.1136/jme.2010.035451
- Escobar, G. J., Greene, J. D., Scheirer, P., Gardner, M., Draper, D., & Kipnis, P. (2008). Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Medical Care, 46, 232–239.
- Francis, R. (2013). Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. London: House of Commons. Retrieved May 16, 2014, from www.midstaffspublicinquiry.com/report.
- Freemantle, N., Richardson, M., Wood, J., Ray, D., Khosla, S., Sun,P., et al. (2013). Can we update the Summary Hospital Mortality Index (SHMI) to make a useful measure of the quality of hospital care? An observational study. BMJ Open, 3, 1–8. e002018.
- Grigg, O. A., Farewell, V. T., & Spiegelhalter, D. J. (2003). Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts. Statistical Methods in Medical Research, 12, 147–170.
- Hawkes, N. (2010). Patient coding and the ratings game. British Medical Journal, 340, c2153.
- Healthcare Commission. (2009). Following up mortality “outliers”: A review of the programme for taking action where data suggest there may be serious concerns about the safety of patients. London: Healthcare Commission. Retrieved February 25, 2014, from www.cqc.org.uk/_db/_documents/Following_up_mortality_outliers_200906054425.pdf.
- Healthcare Improvement Scotland. (2013). Rapid review of the safety and quality of care for acute adult patients in NHS Lanarkshire. Retrieved February 25, 2014, from www.healthcareimprovementscotland.org/our_work/governance_and_assurance/programme_resources/nhs_lanarkshire__review.aspx.
- Hogan, H., Healey, F., Neale, G., Thomson, R., Vincent, C., & Black, N. (2012). Preventable deaths due to problems in care in English acute hospitals: A retrospective case record review study. BMJ Quality and Safety, 21, 737–745.
- Holloway, R. G., & Quill, T. E. (2007). Mortality as a measure of quality: Implications for palliative and end-of-life care. Journal of the American Medical Association, 298, 802–804.
- Jacques, R. M., Fotheringham, J., Campbell, M. J., & Nicholl, J. (2013). Did hospital mortality in England change from 2005 to 2010? A retrospective cohort analysis. BMC Health Services Research, 13, 216.
- Jarman, B., Bottle, A., Aylin, P., & Brown, M. (2005). Monitoring changes in hospital standardised mortality ratios. British Medical Journal, 330, 329.
- Jarman, B., Pieter, D., van der Veen, A. A., Kool, R. B., Aylin, P., Bottle, A., et al. (2010). The hospital standardised mortality ratio: A powerful tool for Dutch hospitals to assess their quality of care? Quality and Safety in Health Care, 19, 9–13. doi:10.1136/qshc.2009.032953
- Jha, A. K., Zhonghe, L., Orav, E. J., & Epstein, A. M. (2005). Care in US Hospitals—The hospital quality alliance program. New England Journal of Medicine, 353, 265–274.
- Keogh, B. (2013). Review into the quality of care and treatment provided by 14 hospital trusts in England: Overview report. England: National Health Service. Retrieved February 25, 2014, from www.nhs.uk/NHSEngland/bruce-keogh-review/Documents/outcomes/keogh-review-final-report.pdf.
- Klugman, R., Allen, L., Benjamin, E. M., Fitzgerald, J., & Ettinger, W., Jr. (2010). Mortality rates as a measure of quality and safety, “caveat emptor.” American Journal of Medical Quality, 25, 197–201.
- Krumholz, H. M., Brindis, R. G., Brush, J. E., Cohen, D. J., Epstein, A. J., Furie, K., et al. (2006). Standards for statistical models used for reporting of health outcomes: An American Heart Association scientific statement from the Quality of Care and Outcomes Research Interdisciplinary Writing Group. Circulation, 113, 456–462.
- Lilford, R., & Pronovost, P. (2010). Using hospital mortality rates to judge hospital performance: A bad idea that just won't go away. British Medical Journal, 340, 955–957. c2016.
- Mid Staffordshire NHS Foundation Trust Inquiry. (2010). Independent inquiry into care provided by Mid Staffordshire NHS Foundation Trust, January 2005—March 2009. Retrieved February 25, 2014, from www.midstaffsinquiry.com/assets/docs/Inquiry_Report-Vol1.pdf.
- Mohammed, M. A., Lilford, R., Rudge, G., Holder, R., & Stevens, A. (2013). The findings of the Mid Staffordshire Inquiry do not uphold the use of hospital standardized mortality ratios as a screening test for “bad” hospitals. Quarterly Journal of Medicine, 106, 849–854. doi:10.1093/qjmed/hct101
- Move Your Dot™: Measuring, Evaluating, and Reducing Hospital Mortality Rates (Part 1). (2003). IHI Innovation Series white paper. Boston, MA: Institute for Healthcare Improvement. Retrieved February 25, 2014, from www.ihi.org/resources/Pages/IHIWhitePapers/MoveYourDotMeasuringEvaluatingandReducingHospitalMortalityRates.aspx.
- Parry, G. J., Gould, C. R., McCabe, C. J., & Tarnow-Mordi, W. O. (1998). Annual league tables of hospital mortality in neonatal intensive care: A longitudinal study. British Medical Journal, 316, 1931–1935.
- Richardson, D., Tarnow-Mordi, W. O., & Lee, S. K. (1999). Risk-adjustment for quality improvement. Pediatrics, 103(Suppl), 255–265.
- Schulman, J., Spiegelhalter, D. J., & Parry, G. (2008). How to interpret your dot: Decoding the message of clinical performance indicators. Journal of Perinatology, 28, 588–596.
- Scott, I. A., Brand, C. A., Phelps, G. E., Barker, A. L., & Cameron, P. A. (2011). Using hospital standardised mortality ratios to assess quality of care–proceed with extreme caution. Medical Journal of Australia, 194, 645–648.
- Shahian, D. M., Iezzoni, L. I., Meyer, G. S., Kirle, L., & Normand, S. L. (2012). Hospital-wide mortality as a quality metric: Conceptual and methodological challenges. American Journal of Medical Quality, 27, 112–123.
- Shahian, D. M., Wolf, R. E., Iezzoni, L., Kirle, L., & Normand, S. L. (2010). Variability in the measurement of hospital-wide mortality rates. New England Journal of Medicine, 363, 2530–2539.
- Solberg, L., Mosser, G., & McDonald, S. (1997). The three faces of performance improvement: Improvement, accountability and research. Joint Commission Journal on Quality Improvement, 23, 135–147.
- Song, Y., Skinner, J., Bynum, J., Sutherland, J., Wennberg, J. E., & Fisher, E. S. (2010). Regional variations in diagnostic practices. New England Journal of Medicine, 363, 45–53.
- Spiegelhalter, D., Sherlaw-Johnson, C., Bardsley, M., Blunt, I., Wood, C., & Grigg, O. (2012). Statistical methods for healthcare regulation: Rating, screening and surveillance. Journal of the Royal Statistical Society: Series A (Statistics in Society), 175, 1–47. doi:10.1111/j.1467-985X.2011.01010.x
- van den Bosch, W. F., Kelder, J. C., & Wagner, C. (2011). Predicting hospital mortality among frequently readmitted patients: HSMR biased by readmission. BMC Health Services Research, 14, 57. doi:10.1186/1472-6963-11-57.
- van Gestel, Y. R., Lemmens, V. E., Lingsma, H. F., de Hingh, I. H., Rutten, H. J., & Coebergh, J. W. (2012). The hospital standardized mortality ratio fallacy: A narrative review. Medical Care, 50, 662–667.
- Welch, H. G., Sharp, S. M., Gottlieb, D. J., Skinner, J. S., & Wennberg, J. E. (2011). Geographic variation in diagnosis frequency and risk of death among Medicare beneficiaries. Journal of the American Medical Association, 305, 1113–1118.
- Wennberg, J. E., Staiger, D. O., Sharp, S. M., Gottlieb, D. J., Bevan, G., McPherson, K., et al. (2013). Observational intensity bias associated with illness adjustment: Cross-sectional analysis of insurance claims. British Medical Journal, 346, 1–11. f549. doi:10.1136/bmj.f549
