Risk Prediction in Transition: MAGGIC Score Performance at Discharge and Incremental Utility of Natriuretic Peptides

doi:10.1016/j.cardfail.2019.11.016

Journal of Cardiac Failure

Volume 26, Issue 1, January 2020, Pages 52-60

https://doi.org/10.1016/j.cardfail.2019.11.016 Get rights and content

ABSTRACT

Background

Risk stratification for hospitalized patients with heart failure (HF) remains a critical need. The Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) score is a robust model derived from patients with ambulatory HF. Its validity at the time of discharge and the incremental value of natriuretic peptides (NPs) in this setting is unclear.

Methods

This was a single-center study examining a total of 4138 patients with HF from 2 groups; hospital discharge patients from administrative data (n = 2503, 60.5%) and a prospective registry of patients with ambulatory HF (n = 1635, 39.5%). The ambulatory registry patients underwent N-terminal pro–B-type NP (BNP) measurement at enrollment, and in the hospitalize discharge cohort clinical BNP levels were abstracted. The primary endpoint was all-cause mortality within 1 year. MAGGIC score performance was compared between cohorts utilizing Cox regression and calibration plots. The incremental value of NPs was assessed using calculated area under the curve and net reclassification improvement (NRI).

Results

The hospitalized and ambulatory cohorts differed with respect to primary outcome (777 and 100 deaths, respectively), sex (52.1% vs 41.7% female) and race (35% vs 49.5% African American). The MAGGIC score showed poor discrimination of mortality risk in the hospital discharge (C statistic: 0.668, hazard ratio [HR]: 1.1 per point, 95% confidence interval [CI]: 0.652, 0.684) but fair discrimination in the ambulatory cohorts (C statistic: 0.784, HR: 1.16 per point, 95% CI: 0.74, 0.83), respectively, a difference that was statistically significant (P = .001 for C statistic, 0.002 for HR). Calibration assessment indicated that the slope and intercept (of MAGGIC-predicted to observed mortality) did not statistically differ from ideal in either cohort and did not differ between the cohorts (all P > .1). NP levels did not significantly improve prediction in the hospitalized cohort (P = .127) but did in the ambulatory cohort (C statistic: 0.784 [95% CI: 0.74, 0.83] vs 0.82 [95% CI: 0.78, 0.85]; P = .018) with a favorable NRI of 0.354 (95% CI: 0.202–0.469; P = .002).

Conclusion

The MAGGIC score showed poor discrimination when used in patients with HF at hospital discharge, which was inferior to its performance in patients with ambulatory HF. Discrimination within the hospital discharge group was not improved by including hospital NP levels.

Section snippets

Study Population

Research participants were identified from patients receiving care through the Henry Ford Health System (HFHS), a vertically integrated, health system serving the primary and specialty health-care needs of individuals in southeastern Michigan, which includes several hospitals, a multispecialty physician group of ∼1900 physicians and researchers (the Henry Ford Medical Group [HFMG]), and a system-owned health insurance product (Health Alliance Plan [HAP]). The study protocol was approved by the

Results

A total of 4138 patients were included in the study (hospital: n = 2503, 60.5%; ambulatory: n = 1635, 39.5%). There were 100 deaths (6.1%) within 1 year of registry enrollment for the ambulatory cohort and 777 deaths (31.0 %) within 1 year of hospital discharge that were included in the analyses. Baseline characteristics of both groups are shown in Table 1. Patients in the hospital cohort were on average slightly older (72.2 ± 15.3 vs 69.4 ± 11.9 years, P= .001), were less often self-identified

Discussion

Given the highly variable prognosis among the HF population,³⁸^,³⁹ risk stratification is critical for optimal management in order to inform patient expectations and effectively target interventions. Hospitalization is a common event in the HF patient experience and a key moment of clinical opportunity, but it also carries highly variable prognoses across patients²⁴ and lacks a convenient and valid risk-stratification method. The MAGGIC score, originally derived from ambulatory patients, is a

Conclusions

These data indicate that the MAGGIC score has poor discrimination when used to risk stratify patients at the time of hospital discharge. The risk discrimination in this setting was inferior to that among ambulatory patients and was not improved by the addition of hospital acquired (usually admission) BNP levels. Better risk stratification tools are needed for use in the post-hospitalization setting.

References (52)

C.M. O'Connor et al.
Predictors of mortality after discharge in patients hospitalized with heart failure: an analysis from the Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients with Heart Failure (OPTIMIZE-HF)
Am Heart J
(2008)
J. Zhang et al.
Identifying patients at risk of death or hospitalisation due to worsening heart failure using decision tree analysis: evidence from the Trans-European Network-Home-Care Management System (TEN-HMS) study
Int J Cardiol
(2013)
D.J. Whellan et al.
Development of a method to risk stratify patients with heart failure for 30-day readmission using implantable device diagnostics
Am J Cardiol
(2013)
H. Wang et al.
Using the LACE index to predict hospital readmissions in congestive heart failure patients
BMC Cardiovasc Disord
(2014)
P. Thavendiranathan et al.
Prediction of 30-day heart failure-specific readmission risk by echocardiographic parameters
Am J Cardiol
(2014)
M. Naffaa et al.
Brain natriuretic peptide at discharge as a predictor of 6-month mortality in acute decompensated heart failure
American J Emerg Med
(2014)
W.C. Levy et al.
Can the Seattle heart failure model be used to risk-stratify heart failure patients for potential left ventricular assist device therapy?
J Heart Lung Transplant
(2009)
Q.L. Huynh et al.
Roles of nonclinical and clinical data in prediction of 30-day rehospitalization or death among heart failure patients
J Card Fail
(2015)
G.C. Fonarow
Epidemiology and risk stratification in acute heart failure
Am Heart J
(2008)
L.M. Fleming et al.
Derivation and validation of a 30-day heart failure readmission model
Am J Cardiol
(2014)

Z.J. Eapen et al.

Validated, electronic health record deployable prediction models for assessing patient risk of 30-day rehospitalization and mortality in older heart failure patients

JACC Heart Fail

(2013)

A.G. Au et al.

Predicting the risk of unplanned readmission or death within 30 days of discharge after a heart failure hospitalization

Am Heart J

(2012)

O.F. AbouEzzeddine et al.

From statistical significance to clinical relevance: a simple algorithm to integrate brain natriuretic peptide and the Seattle Heart Failure Model for risk stratification in heart failure

J Heart Lung Transplant

(2016)

W. Ouwerkerk et al.

Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure

JACC Heart Fail

(2014)

C.M. O'Connor et al.

Triage after hospitalization with advanced heart failure: the ESCAPE (evaluation study of congestive heart failure and pulmonary artery catheterization effectiveness) risk model and discharge score

J Am Coll Cardiol

(2010)

R. Latini et al.

Incremental prognostic value of changes in B-type natriuretic peptide in heart failure

Am J Med

(2006)

W.C. Levy et al.

Heart failure risk prediction models: what have we learned?

JACC Heart Fail

(2014)

D. Scrutinio et al.

Clinical utility of N-terminal pro-B-type natriuretic peptide for risk stratification of patients with acute decompensated heart failure. Derivation and validation of the ADHF/NT-proBNP risk score

Int J Cardiol

(2013)

C. Madamanchi et al.

Obesity and natriuretic peptides, BNP and NT-proBNP: mechanisms and diagnostic implications for heart failure

Int J Cardiol

(2014)

C.W. Yancy et al.

2017 ACC/AHA/HFSA focused update of the 2013 ACCF/AHA Guideline for the Management of Heart Failure: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Failure Society of America

J Am Coll Cardiol.

(2017)

W.Y. Lee et al.

Gender and risk of adverse outcomes in heart failure

Am J Cardiol

(2004)

A. Schulz et al.

Social inequalities, stressors and self reported health status among African American and white women in the Detroit metropolitan area

Soc Sci Med

(2000)

P.K. Lindenauer et al.

Public reporting and pay for performance in hospital quality improvement

N Engl J Med

(2007)

W. Rosamond et al.

Heart disease and stroke statistics–2008 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee

Circulation

(2008)

J.G. Cleland et al.

Predictors of postdischarge outcomes from information acquired shortly after admission for acute heart failure: a report from the Placebo-Controlled Randomized Study of the Selective A1 Adenosine Receptor Antagonist Rolofylline for Patients Hospitalized With Acute Decompensated Heart Failure and Volume Overload to Assess Treatment Effect on Congestion and Renal Function (PROTECT) Study

Circ Heart Fail

(2014)

J.S. Ross et al.

Recent national trends in readmission rates after heart failure hospitalization

Circ Heart Fail

(2010)

Cited by (10)

The MAGGIC risk score in the prediction of death or hospitalization in patients with heart failure: Comparison with natriuretic peptides
2022, Revista Portuguesa de Cardiologia
The MAGGIC risk score has been validated to predict mortality in patients with heart failure (HF).
To assess the score ability to predict hospitalization and death and to compare with natriuretic peptides.
Ninety-three consecutive patients (mean age 62±10 years) with chronic HF and left ventricular ejection fraction (EF) <50% were studied. The MAGGIC score was applied at baseline and the patients were followed for 219±86 days. MAGGIC score was compared with NT-proBNP in the prediction of events. The primary end point was the time to the first event, which was defined as cardiovascular death or hospitalization for HF.
There were 23 (24.7%) events (3 deaths and 20 hospitalizations). The median score in patients with and without events was, respectively, 20 [interquartile range 14.2–22] vs. 15.5 [11/21], p=0.16. A ROC curve was performed and a cutoff point of 12 points showed a sensitivity of 87% and specificity of 37% with an area under the curve of 0.59 (95% CI 0.48–0.69) which was lower than that of NT-proBNP (AUC 0.67; 95% CI 0.56–0.76). The mean event-free survival time for patients above and below this cutpoint was 248.8±13 vs. 290±13.7 days (log rank test with p=0.044). Using the COX proportional hazard model, age (p=0.004), NT-proBNP >1000 pg/mL (p=0.014) and the MAGGIC score (p=0.025) were independently associated with the primary outcome.
The MAGGIC risk score was an independent predictor of events, including heart failure hospitalization. The addition of biomarkers improved the accuracy of the score.
O score de risco MAGGIC foi validado para predizer mortalidade em pacientes com insuficiência cardíaca (IC).
Avaliar a capacidade do score de predizer hospitalização e óbito e comparar com peptídeos natriuréticos.
Foram estudados 93 pacientes consecutivos (idade média 62 ± 10 anos) com IC crónica e fração de ejeção do ventrículo esquerdo (FE) <50%. O score MAGGIC foi aplicado no início do estudo e os pacientes foram acompanhados por 219 ± 86 dias. O score MAGGIC foi comparado com o NT-proBNP na previsão de eventos. O desfecho primário foi o tempo até o primeiro evento, que foi definido como morte cardiovascular ou hospitalização por IC.
Ocorreram 23 (24,7%) eventos (3 óbitos e 20 hospitalizações). A pontuação média em pacientes com e sem eventos foi, respetivamente, 20 (intervalo interquartil 14,2-22) versus 15,5 (11/21), p=0,16. Curva ROC foi realizada e um ponto de corte de 12 pontos mostrou uma sensibilidade de 87% e especificidade de 37% com uma área sob a curva de 0,59 (IC 95% 0,48-0,69) que foi menor do que a do NT-proBNP (AUC 0,67; IC 95% 0,56-0,76). O tempo médio de sobrevida livre de eventos para pacientes acima e abaixo desse ponto de corte foi de 248,8 ± 13 versus 290 ± 13,7 dias (teste de log rank com p=0,044). Usando o modelo de risco proporcional COX, idade (p=0,004), NT-proBNP >1000 pg/mL (p=0,014) e o score MAGGIC (p=0,025) foram independentemente associados ao desfecho primário.
O score de risco MAGGIC foi um preditor independente de eventos, incluindo hospitalização por insuficiência cardíaca. A adição de biomarcadores melhorou sua acurácia.
Suppression tumorigenicity 2 (ST2) turbidimetric immunoassay compared to enzyme-linked immunosorbent assay in predicting survival in heart failure patients with reduced ejection fraction
2020, Clinica Chimica Acta
Citation Excerpt :
This was calculated using the integer tabulation as originally published. We chose the MAGGIC score as the clinical risk adjuster because it was derived from a cohort of 39,372 HF patients from 30 studies [18], has been additionally widely validated [19,20], and is tabulated using commonly available clinical data. The N-terminal pro b-type NP (NTproBNP) values were measured in plasma from the same participants collected at baseline and stored at −70 °C until use.
Suppressor of tumorigenicity 2 (ST2) is a powerful marker of prognosis and treatment response in heart failure (HF), however, it is an enzyme-linked immunosorbent assay (ELISA) which may be cumbersome and costly. A turbidimetric immunoassay (TIA) that can run on common chemistry analyzers could overcome this. We studied a novel TIA for ST2, comparing it to commercial ST2 (ELISA).
Patients age ≥ 18 years meeting Framingham definition for HF were enrolled in a prospective registry (Oct 2007 – March 2015) at Henry Ford Hospital and donated blood samples. Participants with reduced ejection fraction (<50%) and available plasma samples were included and valid ST2 measurements were obtained on the same sample using both TIA and ELISA (N = 721). The primary endpoint was all cause death. Correlation between the methods was quantified. The association with survival was tested using unadjusted and adjusted (for MAGGIC score and NTproBNP) Cox models and comparing the Area Under the Curve (AUC).
The inter-assay Spearman correlation coefficient was 0.77. Nonparametric regression showed no significant proportional difference (slope = 0.97) and a very small systematic difference (3.2 ng/mL). In univariate analyses, both TIA and ELISA ST2 were significant associates of survival with similar effect sizes (HR 4.46 and 3.50, respectively, both p < 0.001). In models adjusted for MAGGIC score, both ST2 remained significant in Cox models and incrementally improved AUC vs. MAGGIC alone (MAGGIC AUC = 0.757; TIA + MAGGIC AUC = 0.786, p = 0.025; ELISA + MAGGIC AUC = 0.793, p = 0.033). In models with both MAGGIC and NTproBNP included, both ST2 still remained significant but did not improve AUC.
A novel TIA method for ST2 quantification correlates highly with ELISA and offers similarly powerful risk-stratification.
Development and Validation of a Protein Risk Score for Mortality in Heart Failure A Community Cohort Study
2024, Annals of Internal Medicine
An update on utilising brain natriuretic peptide for risk stratification, monitoring and guiding therapy in heart failure
2023, Expert Review of Molecular Diagnostics
Validation of the Meta-Analysis Global Group in Chronic Heart Failure risk score for the prediction of 1-year mortality in a Chinese cohort
2022, Chinese Medical Journal
Performance of current risk stratification models for predicting mortality in patients with heart failure: A systematic review and meta-analysis
2022, European Journal of Preventive Cardiology

View all citing articles on Scopus

: Funding: This research was supported by the National Heart, Lung, and Blood Institute (Lanfear R01HL103871, R01HL132154). K.W. is supported by NHLBI (R01HL118267), the National Institute of Allergy and Infectious Diseases (R01AI079139), and the National Institute of Diabetes and Digestive and Kidney Diseases (R01DK064695). H.N.S. is supported by the National Heart, Lung, and Blood Institute (PO1HL074237, R01HL132154).

View full text

Risk Prediction in Transition: MAGGIC Score Performance at Discharge and Incremental Utility of Natriuretic Peptides

ABSTRACT

Background

Methods

Results

Conclusion

Section snippets

Study Population

Results

Discussion

Conclusions

Am Heart J

Int J Cardiol

Am J Cardiol

BMC Cardiovasc Disord

Am J Cardiol

American J Emerg Med

J Heart Lung Transplant

J Card Fail

Am Heart J

Am J Cardiol

JACC Heart Fail

Am Heart J

J Heart Lung Transplant

JACC Heart Fail

J Am Coll Cardiol

Am J Med

JACC Heart Fail

Int J Cardiol

Int J Cardiol

J Am Coll Cardiol.

Am J Cardiol

Soc Sci Med

Public reporting and pay for performance in hospital quality improvement

N Engl J Med

Heart disease and stroke statistics–2008 update: a report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee

Circulation

Circ Heart Fail

Recent national trends in readmission rates after heart failure hospitalization

Circ Heart Fail