Journal Information
Vol. 36. Issue 9.
Pages 681-682 (September 2017)
Download PDF
More article options
Vol. 36. Issue 9.
Pages 681-682 (September 2017)
Letter to the Editor
Open Access
Response to Letter to the Editor “Risk stratification in acute coronary syndromes: Graced by a new score?”
Resposta à Carta ao Editor «Estratificação de risco nas síndromas coronárias agudas: como poderá o Grace ser destronado?»
Ana Timóteo
Serviço de Cardiologia, CHLC, Hospital de Santa Marta, Lisboa, Portugal
Related content
Rev Port Cardiol. 2017;36:677-910.1016/j.repce.2017.03.008
Luís Paiva, Sérgio Barra, Lino Gonçalves
This item has received

Under a Creative Commons license
Article information
Full Text
Download PDF
Full Text

We read with great interest the Letter to the Editor by Paiva et al. We appreciate their interest and their constructive comments on our article recently published in the Journal in which we develop and validate a simple risk stratification score (ProACS) for patients with acute coronary syndromes from a large nationwide registry.1 In our paper, the ProACS risk score showed a significantly lower discriminative power compared to the GRACE score in the external validation cohort, but slightly better compared to the development and internal validation cohorts. The Canada Acute Coronary Syndrome (C-ACS) risk score,5 developed with the same principles and objectives, yields similar discriminative power to our own score but is still considered adequate. Paiva et al. performed an additional external validation (highly recommended for any prediction model) in an independent contemporary cohort of 1000 consecutive myocardial infarction (MI) patients (43.5% with ST-elevation MI [STEMI]), fairly similar to the external validation cohort from our paper.2,3 Their results showed that 98% of in-hospital deaths were accurately classified in the intermediate- or high-risk groups; however, the ProACS score's discriminative power was significantly lower than that of the GRACE score (and slightly lower than our results), which might compromise the accuracy of risk stratification. They also performed a risk reclassification study, which showed that ProACS is better at identifying low-risk patients, particularly in the non-ST-elevation MI (NSTEMI) cohort. GRACE is superior in identifying high-risk patients. We obviously agree with the authors’ concluding remark that it remains to be determined whether the simplicity of this new score is offset by its inferior prognostic power compared to the gold standard GRACE risk score.

Our group also performed an external validation in patients from a single center, although with some different baseline characteristics, particularly a predominance of STEMI (62%) and with more cardiovascular risk factors.4 Compared to GRACE, we also obtained a lower discriminative power, with an area under the curve (AUC) of 0.769 for in-hospital mortality, similar to that reported for C-ACS, and slightly superior in STEMI (0.77 vs. 0.74), albeit non-significant.5 Also in an earlier study from our group in a STEMI cohort, we showed that a simple (reduced) score (not yet the ProACS score) led to significant improvement when reclassification was analyzed, particularly in patients without events, as was also demonstrated by Paiva et al.2,6

From a statistical point of view, dissatisfaction with AUC analysis has prompted proposals for new statistical metrics based on risk categories and reclassification.7 Reclassification analysis with the use of net reclassification improvement (NRI) and integrated discrimination improvement (IDI) are particularly useful because they can help to clarify and quantify the degree of correct reclassification of predicted probabilities. However, for some authors NRI has many of the same problems as the AUC.8,9 It is sometimes difficult to be sure if the measure is clinically meaningful due to lack of experience with the index. Some authors recommend the use of category-free (or continuous) NRI, avoiding predefined risk categories, but this can also mislead investigators by overstating the incremental value of an additional biomarker. Furthermore, without proper attention to model fit, NRI can mislead researchers and it is recommended to use bootstrap methods for estimating the variance of NRI and constructing confidence intervals. For those reasons, some caution is advised when interpreting reclassification analysis. Also, in the presence of a fairly robust risk score, such as GRACE, the quantitative improvement in model performance is expected to be small or even negative.

Finally, from a clinical point of view, the fact that ProACS's predicted ability is lower than GRACE does not hinder its application, because an AUC ≥0.75 means that it is still valid. The ProACS risk score better identifies those who do not have events. This is clinically important, because identification of these “truly low-risk patients” may enable better patient selection, avoiding unnecessary interventions that can increase costs as well as the risk of intervention-related adverse events, and may help in the selection of patients for early discharge. As we stated in our paper, risk stratification is a dynamic process that requires risk recalculation after admission. ProACS can be used at the first medical contact, when it is important to decide whether to refer the patient directly to a tertiary hospital, and due to its simplicity, even healthcare professionals without advanced medical or cardiological training (in a pre-hospital setting or in emergency department triage) can use this simple score. However, when full clinical and laboratory data are available, clinicians should calculate the GRACE score, because it provides more accurate risk stratification, which is crucial to patient management decisions.

Conflicts of interest

The author has no conflicts of interest to declare.

A.T. Timoteo, S. Aguiar Rosa, M. Afonso Nogueira, et al.
ProACS risk score: an early and simple score for risk stratification of patients with acute coronary syndromes.
Rev Port Cardiol, 36 (2017), pp. 77-83
L. Paiva, S. Barra, L. Gonçalves.
Risk stratification in acute coronary syndromes. Graced by a new score?.
Rev Port Cardiol, 36 (2017), pp. 677-679
G.S. Collins, J.B. Reitsma, D.G. Altman, et al.
Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD). The TRIPOD statement.
Circulation, 131 (2015), pp. 211-219
A.T. Timoteo, S. Aguiar Rosa, M.A. Nogueira, et al.
External validation of the ProACS score for risk stratification of patients with acute coronary syndromes.
Rev Port Cardiol, 35 (2016), pp. 323-328
T. Huynh, S. Krouz, A. Yan, et al.
Canada Acute Coronary Syndrome risk score: a new risk score for early prognostication in acute coronary syndromes.
Am Heart J, 166 (2013), pp. 58-63
A.T. Timoteo, A.L. Papoila, J.P. Lopes, et al.
Is it possible to simplify risk stratification scores for patients with ST-segment elevation myocardial infarction undergoing primary angioplasty?.
Rev Port Cardiol, 32 (2013), pp. 967-973
M.J. Pencina, R.B. D’Agostino Sr, R.B. D’Agostino Jr., et al.
Evaluating the added predictive ability of a new marker: from the area under the ROC curve to reclassification and beyond.
Stat Med, 27 (2008), pp. 157-172
M.S. Pepe.
Problems with risk reclassification methods for evaluating prediction models.
Am J Epidemiol, 173 (2011), pp. 1327-1335
K.F. Kerr, Z. Wang, H. Janes, et al.
Net reclassification indices for evaluating risk prediction instruments: a critical review.
Epidemiology, 25 (2014), pp. 114-121
Copyright © 2017. Sociedade Portuguesa de Cardiologia
Revista Portuguesa de Cardiologia (English edition)
Article options
en pt

Are you a health professional able to prescribe or dispense drugs?

Você é um profissional de saúde habilitado a prescrever ou dispensar medicamentos

By checking that you are a health professional, you are stating that you are aware and accept that the Portuguese Journal of Cardiology (RPC) is the Data Controller that processes the personal information of users of its website, with its registered office at Campo Grande, n.º 28, 13.º, 1700-093 Lisbon, telephone 217 970 685 and 217 817 630, fax 217 931 095, and email I declare for all purposes that the information provided herein is accurate and correct.