The ProACS risk score is an early and simple risk stratification score developed for all-cause in-hospital mortality in acute coronary syndromes (ACS) from a Portuguese nationwide ACS registry. Our center only recently participated in the registry and was not included in the cohort used for developing the score. Our objective was to perform an external validation of this risk score for short- and long-term follow-up.
MethodsConsecutive patients admitted to our center with ACS were included. Demographic and admission characteristics, as well as treatment and outcome data were collected. The ProACS risk score variables are age (≥72 years), systolic blood pressure (≤116 mmHg), Killip class (2/3 or 4) and ST-segment elevation. We calculated ProACS, Global Registry of Acute Coronary Events (GRACE) and Canada Acute Coronary Syndrome risk score (C-ACS) risk scores for each patient.
ResultsA total of 3170 patients were included, with a mean age of 64±13 years, 62% with ST-segment elevation myocardial infarction. All-cause in-hospital mortality was 5.7% and 10.3% at one-year follow-up. The ProACS risk score showed good discriminative ability for all considered outcomes (area under the receiver operating characteristic curve >0.75) and a good fit, similar to C-ACS, but lower than the GRACE risk score and slightly lower than in the original development cohort. The ProACS risk score provided good differentiation between patients at low, intermediate and high mortality risk in both short- and long-term follow-up (p<0.001 for all comparisons).
ConclusionsThe ProACS score is valid in external cohorts for risk stratification for ACS. It can be applied very early, at the first medical contact, but should subsequently be complemented by the GRACE risk score.
O score ProACS é um score simples e precoce desenvolvido para estratificação de risco de mortalidade hospitalar nas síndromes coronárias agudas (SCA), a partir de um registo nacional de SCA. O nosso centro só participou mais recentemente, pelo que os nossos doentes não foram incluídos na coorte de desenvolvimento do score. O nosso objetivo é o de validar externamente o score ProACS para mortalidade a curto e longo prazo.
MétodosForam incluídos doentes consecutivos admitidos no nosso centro por SCA. Obtiveram-se as características demográficas e da admissão, bem como o tratamento e seguimento. O score ProACS inclui as seguintes variáveis: idade ≥72 anos, pressão arterial sistólica ≤116mmHg, classe Killip na admissão e elevação do segmento ST. Para cada doente foi calculado o score ProACS, Global Registry of Acute Coronary Events (GRACE) e o C-ACS.
ResultadosIncluíram-se 3170 doentes, idade média de 64±13 anos, 62% com enfarte com elevação de ST. A mortalidade total hospitalar foi de 5,7 e 10,3% a um ano de seguimento. O score ProACS mostrou uma boa capacidade discriminativa (AUC>0,75) e boa calibração, semelhante ao C-ACS, mas inferior quando comparado com o score GRACE e ligeiramente inferior quando comparado com a coorte de desenvolvimento original. Permite uma boa diferenciação entre doentes com risco baixo, intermédio e alto quer para mortalidade a curto quer a longo prazo (p<0,001 para todas as comparações).
ConclusõesO score ProACS é um score válido em coortes externas. Pode ser aplicado muito precocemente no primeiro contacto médico, mas posteriormente deverá ser complementados pelo score GRACE.
The approach to acute coronary syndrome (ACS) has undergone many changes in recent years, particularly regarding invasive treatments, not only for ST-segment elevation myocardial infarction (STEMI) but also other ACS and in patients at high risk for cardiovascular events.1–4 Various scores have been developed over the last 20 years to stratify this risk. The first were developed on the basis of large multicenter clinical trials, such as the TIMI score; the disadvantage was they were not always sufficiently representative of real-world populations.5,6 More recently, the most widely used score has been the Global Registry of Acute Coronary Events (GRACE) score, which was based on a multinational ACS registry and hence is more representative. It also has better discriminative ability for both STEMI and other ACS.7,8 However, such scores are not always used in clinical practice, partly due to their complexity.9,10
Our group has previously demonstrated that risk scores can be simplified, albeit with a slight reduction in discriminative ability.11 The ProACS score was accordingly developed on the basis of the Portuguese Registry on Acute Coronary Syndromes (ProACS).12 The aim of the present study was to validate this score in a Portuguese population and to determine its short- and long-term predictive value.
MethodsThe ProACS score was developed from the population of the ProACS registry, and included all patients enrolled between January 1, 2002 and October 31, 2014. ProACS is a continuous, prospective observational registry with 33 participating cardiology departments in Portugal. The inclusion criteria and variables have been published previously.13 The ProACS score was based on 37460 records of all-cause in-hospital mortality. Patients enrolled between January 1, 2002 and June 30, 2011 were randomly separated into the development (60%) and internal validation (40%) cohorts, while the 8586 patients enrolled from after July 1, 2011 made up the external validation cohort. Since our center only recently began to participate in the ProACS registry, our records were not included in the development cohort, and we therefore decided to perform an external validation of the ProACS score in a Portuguese population.
The score uses the following variables: age ≥72 years (2 points), systolic blood pressure ≤116 mmHg, Killip class at admission 2 or 3 and ST-segment elevation (1 point each), and Killip class 4 (3 points). Individuals who score 0 are at low risk of in-hospital mortality, 1 or 2 represents intermediate risk, and ≥3 points indicates high risk. The inclusion criteria and variables of our center's registry are similar to those of ProACS and also include demographic characteristics, risk factors, previous heart disease, characteristics at admission and treatment. We assessed all-cause in-hospital and one-year mortality.
All patients gave written informed consent for inclusion in the registry. The study is in accordance with the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) recommendations for the reporting of studies validating prediction scores.14
Statistical analysisContinuous variables were expressed as means and standard deviation and were compared using the Student's t test. Normal distribution was tested with the Kolmogorov-Smirnov test. Categorical variables were expressed as percentages and differences between groups were analyzed by the chi-square test or Fisher's test, as appropriate.
The ProACS score was calculated for each patient. For purposes of comparison the GRACE score and the Canada Acute Coronary Syndrome Risk Score (C-ACS) were also calculated.8,15 The latter was developed on the basis of the same assumptions as the ProACS score but in a Canadian population.
The predictive value of the ProACS score was analyzed by means of the area under the receiver operating characteristic (ROC) curve (AUC), and the model's goodness of fit was assessed by the Hosmer-Lemeshow test, in which a model has a good fit if its p value is not significant. AUCs for dependent and independent samples were compared using Henley and MacNeil's method.16,17
The statistical analysis was performed using IBM SPSS Statistics (version 19.0.0.2). A p value <0.05 was considered statistically significant.
ResultsThe external validation cohort of our center included 3170 patients with a mean age of 64±13 years, 71% male, and 62% with STEMI. Table 1 compares this cohort with the development cohort of the ProACS score. In general, patients in the validation cohort were younger and had more cardiovascular risk factors (except diabetes), fewer had a history of myocardial infarction and stroke, but more had undergone coronary angioplasty. STEMI was the most frequent form of ACS in the validation cohort, more often treated invasively and with a better profile at admission. In-hospital mortality was similar to the development cohort. All-cause mortality in the validation cohort was 7.2% at 30 days and 10.3% at one year.
Clinical characteristics of the development cohort (from the ProACS registry) and the validation cohort.
Development cohort (n=17380) | External validation cohort (n=3170) | p | |
---|---|---|---|
Age (years) | 66±13 | 64±13 | <0.001 |
Male (%) | 69.9 | 70.6 | 0.438 |
BMI (kg/m2) | 27±4 | 27±4 | 1.000 |
Previous history (%) | |||
Hypertension | 62.2 | 65.9 | <0.001 |
Smoking | 24.0 | 37.1 | <0.001 |
Diabetes | 27.0 | 26.1 | 0.300 |
Dyslipidemia | 45.2 | 50.7 | <0.001 |
MI | 18.4 | 15.1 | <0.001 |
PCI | 8.0 | 10.7 | <0.001 |
CABG | 4.1 | 4.0 | 0.830 |
Stroke/TIA | 7.0 | 5.8 | 0.015 |
PAD | 3.1 | 3.5 | 0.255 |
Admission | |||
STEMI (%) | 43.6 | 62.2 | <0.001 |
HR (bpm) | 79±20 | 78±20 | <0.001 |
SBP (mmHg) | 141±30 | 136±29 | <0.001 |
Killip class ≥2 (%) | 18.9 | 12.6 | <0.001 |
CA (%) | 68.9 | 87.5 | <0.001 |
PCI (%) | 43.8 | 63.5 | <0.001 |
CABG (%) | 1.4 | 1.6 | 0.425 |
In-hospital mortality (%) | 5.4 | 5.7 | 0.517 |
BMI: body mass index; CA: coronary angiography; CABG: coronary artery bypass grafting; HR: heart rate; MI: myocardial infarction; PAD: peripheral arterial disease; PCI: percutaneous coronary intervention; SBP: systolic blood pressure; STEMI: ST-elevation myocardial infarction; TIA: transient ischemic attack.
In our population, the ProACS score showed good discriminative ability for short-, medium- and long-term mortality, with a good fit (Table 2). Its discriminative ability was similar to that of the C-ACS score but lower than the GRACE score. Applying the risk thresholds described also correctly identified individuals at low, intermediate and high risk for in-hospital mortality (Figure 1). The ProACS score had a similar predictive ability for STEMI and other ACS, in all cases comparable to that of the C-ACS score but lower than the GRACE score (Table 3).
Predictive ability and goodness of fit of the ProACS, GRACE and C-ACS risk scores for short-, medium- and long-term mortality.
Score | AUC (95% CI) | pa | p (HL) |
---|---|---|---|
In-hospital mortality | |||
ProACS | 0.769 (0.732-0.805) | – | 0.280 |
GRACE | 0.857 (0.829-0.886) | <0.001 | 0.263 |
C-ACS | 0.769 (0.731-0.806) | 1.000 | 0.056 |
30-day mortality | |||
ProACS | 0.755 (0.720-0.789) | – | 0.389 |
GRACE | 0.829 (0.802-0.857) | <0.001 | 0.098 |
C-ACS | 0.748 (0.713-0.784) | 0.670 | 0.136 |
One-year mortality | |||
ProACS | 0.748 (0.719-0.777) | – | 0.623 |
GRACE | 0.804 (0.779-0.829) | <0.001 | 0.633 |
C-ACS | 0.752 (0.722-0.782) | 0.781 | 0.018 |
AUC: area under the curve; C-ACS: Canada Acute Coronary Syndrome Risk Score; CI: confidence interval; GRACE: Global Registry of Acute Coronary Events; HL: Hosmer-Lemeshow test.
Comparison of the ProACS, GRACE and C-ACS risk scores for ST-elevation myocardial infarction and other acute coronary syndromes in terms of short-term (in-hospital) and long-term (one year) mortality.
ProACS | GRACE | pa | C-ACS | pa | ||
---|---|---|---|---|---|---|
In-hospital | ||||||
STEMI | AUC | 0.77 (0.73-0.82) | 0.87 (0.84-0.90) | 0.002 | 0.77 (0.72-0.81) | 0.816 |
HL | 0.247 | 0.540 | 0.800 | |||
NSTE-ACS | AUC | 0.74 (0.67-0.80) | 0.82 (0.75-0.88) | 0.149 | 0.79 (0.73-0.84) | 0.391 |
HL | 0.113 | 0.785 | <0.001 | |||
pb | 0.435 | 0.208 | 0.669 | |||
One year | ||||||
STEMI | AUC | 0.76 (0.72-0.79) | 0.82 (0.79-0.85) | 0.009 | 0.74 (0.71-0.78) | 0.694 |
HL | 0.231 | 0.289 | 0.885 | |||
NSTE-ACS | AUC | 0.74 (0.69-0.79) | 0.79 (0.74-0.83) | 0.221 | 0.77 (0.72-0.81) | 0.504 |
HL | 0.200 | 0.486 | <0.001 | |||
pb | 0.661 | 0.244 | 0.513 |
AUC: area under the curve with 95% confidence interval; C-ACS: Canada Acute Coronary Syndrome Risk Score; GRACE: Global Registry of Acute Coronary Events; HL: Hosmer-Lemeshow test; NSTE-ACS: non-ST-elevation acute coronary syndrome; STEMI: ST-elevation myocardial infarction.
In recent years, demonstration of the advantages of early invasive treatment of ACS has made effective risk stratification essential in order to identify individuals at high or intermediate risk who will benefit most from an invasive approach.1–4 While this stratification is less important in cases of STEMI, since primary revascularization is the first-line treatment for the majority of these patients, in non-ST-elevation ACS – a much more heterogeneous group in terms of risk – timely risk stratification is vital.1–4 Even in STEMI patients, transfer of those at higher risk to specialized centers is also desirable.
The TIMI score was one of the first to be developed and rapidly gained popularity due to its clear benefits.5,6 However, the early scores were based on multicenter randomized clinical trials with highly specific inclusion criteria and therefore did not truly represent real-world ACS populations; for example, very elderly patients and those with severe renal failure are usually excluded from such trials and were thus not considered in the development of the risk scores. The GRACE score,7,8 developed some years later, was based on an ACS registry and included new variables with a strong impact on prognosis, including renal function; it was shown to have better discriminative ability, and for this reason it came to replace the TIMI score. However, the improved discriminative ability of more recent scores came at the cost of a significant increase in complexity, which partly explains physicians’ reluctance to use them.9,10 Computer software has been developed to facilitate calculation of the scores, but even so, the fact that they rely on laboratory test results that may only be available some time after admission can delay their application. This is important, since the results of these scores can be decisive in the management of urgent patients.
Our group has previously demonstrated that risk scores can be simplified, albeit with a slight reduction in discriminative ability.11 However, in view of the limitations of our sample, particularly its small size, we did not propose a new score. A Canadian group recently developed a new score, the C-ACS, based on large Canadian registries,15 in which they identified age, Killip class, systolic blood pressure and heart rate at admission as the main variables. Although the predictive ability of this score was lower than the GRACE score, it was considered acceptable and close to that of the TIMI score.5,6 Given the differences between North American and European (particularly southern European) populations, we decided to develop a score based on a representative sample of Portuguese ACS patients, and the ProACS registry was an obvious source of data for this purpose. The result was the ProACS score, which includes very simple variables (age, Killip class on admission, systolic blood pressure and ST elevation).12 The continuous variables were dichotomized to simplify calculation. The score enables rapid stratification of ACS patients, even by health care professionals without advanced medical training, in pre-hospital settings or emergency departments. The ProACS score showed similar discriminative ability to the C-ACS score, but lower than the GRACE score, and with an AUC of ≥0.75.
In our external validation cohort, the ProACS score showed reasonably good predictive ability, not only in the short term as demonstrated by the original validation, but also in the medium and long term. The fact that its predictive ability is lower than that of the GRACE score does not hinder its application, since an AUC of ≥0.75 means it is valid for use at the first medical contact, when it may be important to decide whether to refer the patient to a center able to perform interventions 24/7 or to a less specialized unit. It would also be valuable in the context of STEMI, enabling some patients to be transferred directly to specialist tertiary centers. However, when full clinical and laboratory data are available the GRACE score should be applied, since it provides a complementary stratification that may be useful for the management of these patients, particularly in terms of medication or the possibility of early discharge.
LimitationsOur external validation cohort differs significantly from the population used to develop the score and also from the populations of other hospitals, particularly in the high proportion of patients with STEMI (>60%). For this reason our results may not apply to other populations, and further validation is required.
ConclusionsThe ProACS score, developed on the basis of a representative Portuguese population, is a simple and easily applied risk score for stratification of ACS patients, and has been validated in external cohorts. It can be applied very early, at the first medical contact, but should subsequently be complemented by more effective scores, particularly the GRACE risk score.
Ethical disclosuresProtection of human and animal subjectsThe authors declare that no experiments were performed on humans or animals for this study.
Confidentiality of dataThe authors declare that they have followed the protocols of their work center on the publication of patient data.
Right to privacy and informed consentThe authors declare that no patient data appear in this article.
Conflicts of interestThe authors have no conflicts of interest to declare.
Please cite this article as: Timóteo A, Aguiar Rosa S, Nogueira MA, et al. Validação externa do score de risco ProACS para estratificação de risco de doentes com síndrome coronária aguda. Rev Port Cardiol. 2016;35:323–328.