This study was approved by the NOVA Medical School/Faculdade de Ciências Médicas and Centro Hospitalar Universitário de Lisboa Central (CHULC) prior to initiation. We conducted a prospective observational study of consecutive elective cardiac surgery patients between July 2016 and November 2018, who were followed up for one year, and will be the subject of a different manuscript. To calculate the larger study sample size, an α=0.05, an error of 0.04 and a frailty prevalence of 15% were considered, achieving a sample size of 306 patients.11,12 At our institution, all elective cardiac surgery patients are observed in preoperative anesthesia consultations. Patient recruitment took place once the nature and goals of the study had been explained and consent for data collection obtained. A convenience sample of consecutive patients 65 years old and above, admitted for elective valve or myocardial revascularization surgery and observed by the principal investigator in preoperative anesthesia consultation, was selected. For the purpose of validity and reproducibility studies, we estimated subsamples of 139 and 64 patients, respectively (detailed ahead). Exclusion criteria were complex aortic arch procedures, urgent or redo surgery, and the presence of sensory or cognitive deprivation that prevented communication between patients and investigators. Figure 1 depicts the flowchart from enrolled patients for the prospective longitudinal, validity and reproducibility studies.

Figure 1.

Flowchart of enrolled patients.

(0.29MB).

Prior to reproducibility testing and after obtaining permission from the EFS authors, the original English version of the EFS was translated to Portuguese by two independent translators. A group of eight doctors from different specialties (anesthesiologists, cardiac surgeons, and family physicians) with knowledge of the frailty construct met in a focus group to build a final consensus Portuguese version. Both the final Portuguese version and its English backtranslation were sent to the original authors for approval. The original English and Portuguese versions, licensed by the University of Alberta for use in this study, are depicted in Figures 2 and 3.

Figure 2.

Original English version from the University of Alberta.

(0.4MB).

Figure 3.

Adapted and licensed Portuguese version of the Edmonton Frail Scale.

(0.43MB).

Prior to the start of the study, the principal investigator administered the Portuguese EFS adapted version to ten eligible patients to identify possible difficulties with its use and contacted the original authors to clarify any doubts that arose. Subsequently, another two collaborators were trained by the principal investigator to administer the scale.

The maximum scoring for EFS is 17 and cut-points proposed in the official 2019 EFS webpage were considered in frailty diagnosis (1–3 fit; 4–5 vulnerable; 6–7 mild frailty; 8–9 moderate frailty; 10 or more severe frailty).

Other pre-defined variables were collected as part of the larger study, including demographic and clinical variables, such as number of school years, marital status, main surgical diagnosis, and scores in the Portuguese adapted versions of the Mini-Mental State Examination (MMSE)13,14 and Geriatric Depression Scale (GDS),15 as well as Katz16 scale and Clinical Frailty Scale (CFS)17 scores. In addition, the prognostic scores EuroSCORE II18 (European System for Cardiac Operation Risk Evaluation) and STS19,20 were collected.

To test for reproducibility of the P-EFS adapted version and assuming that good agreement would be a Kappa coefficient >0.60 or an intraclass correlation coefficient >0.8,21 an α=0.05 and a β=0.10 (power of 0.90), we estimated a sample size of 30 subjects. To avoid exposure to the P-EFS related bias, we more than doubled the sample size and performed 34 inter-observer and 33 intra-observer agreement tests in 64 patients. P-EFS was re-applied within a minimum interval of 2h and a maximum of 2 months. A substantial part of the enrolled patients lived more than 50km away from the hospital and/or were not totally autonomous in traveling to the hospital. Therefore, the research team agreed to administer P-EFS at patients’ scheduled appointments for routine and essential examinations for preoperative assessment, and no patient returned exclusively for the purpose of P-EFS re-administration.

To test for construct and criterion validity, 139 patients provided at least 0.80 power (effect size=0.4 and α=0.05, two-sided test) to detect differences among groups.

To assess construct validity, we evaluated the agreement of Katz and MMSE measurements with P-EFS total scores, and regarding criterion validity we compared P-EFS and Clinical Frailty Scales scores. The Clinical Frailty Scale has been widely used both in outpatient and acute care settings. It is an ordinal scale that incorporates clinician judgment and employs a 1–9 scoring system that includes the evaluation of many frailty domains. It stratifies frailty into fit (1–3), vulnerable (4), frail (5–8) and terminally ill (9) (Figure 4). For P-EFS specific domains of cognition and mood, we assessed criterion validity by comparing the Clock Drawing Test with MMSE and Mood questions in P-EFS with GDS30 and GDS 15.

Figure 4.

Clinical Frailty Scale.

(0.7MB).

Regarding MMSE normative data, we used both Morgado et al.’s13 cut-off points for cognitive deficit, as they are the most widely accepted by the Portuguese medical community, and the more recent Freitas et al.14 normative data.

Content validity assessment was not performed as it would require the CGA of study participants to establish a frailty diagnosis, and this is not part of a standard preoperative evaluation in clinical practice. On the other hand, the content validity for the EFS was established by the authors of the original version when it was first described.

The characteristics of study participants were described with frequencies (percentages) and with median and range (min–max), as appropriate. The Mann–Whitney test, Spearman's correlation coefficient, marginal homogeneity test, and Kappa coefficient were used as needed to assess validity.

For reproducibility assessment, the Kappa coefficient was estimated for both frailty diagnosis and the 11 EFS items in inter- and intra-observer analyses. To assess the intra-observer repeatability and inter-observer reproducibility, intraclass correlation coefficients (ICCs) and corresponding 95% confidence intervals were estimated using linear mixed-effects models.

The level of significance was set at α=0.05. Data were analyzed using the Statistical Package for the Social Sciences for Windows (version 25.0; IBM Corp. Released 2017. IBM SPSS Statistics for Windows, version 25.0. Armonk, NY: IBM Corp.).

In the 139 patients sample used for validity study, mean age was 73.55 years (5.84), with 57.6% male and 18.0% widowers, and a mean years of schooling of 5.49 (4.23). In the surgery group, 47.5% underwent aortic valve replacement, 30.2% to coronary artery bypass graft, 2.2% to mitral valve replacement, and the remaining patients underwent more than one surgery (Table 1). The median raw EFS score was 6, ranging from 1 to 13. Considering the EFS score in categories, 15 (10.8%) were fit (1–3), 35 (25.2%) were vulnerable (4–5), 49 (35.3%) exhibited mild frailty (6–7), 25 (18%) exhibited moderate frailty (8–9), and 15 (10.8%) had severe frailty (≥10).

The EuroSCORE II, STS Mortality, and STS Morbi-mortality, GDS30 and GDS15, MMSE, Katz, and CFS descriptive data are presented in Table 2.

Analysis of GDS30 results revealed that 63.3% of the patients had no depressive symptoms, 31.7% had mild depression, and 5% had severe depression. When GDS15 was considered, 33.1% had depressive symptoms. Regarding P-EFS mood questions, 40.3% answered “yes” when asked about sadness or depression.

Regarding criteria validity for the mood domain, the mood question in P-EFS showed good agreement (k=0.651, p<0.001) with P-GDS30 and moderate agreement with P-GDS15 (k=0.569, p<0.001).

For the screening of cognitive deficits using MMSE, we considered both the 2009 and 2014 normative data for the Portuguese population. According to the 2009 criteria, only 12.2% of patients scored for cognitive deficit, but this percentage increased to 19.4% when the 2014 criteria were applied. In the validity sample and analysis of the P-EFS cognition domain, 26.6% scored for major errors in the Clock Drawing Test (CDT), and the remaining had either no errors (29.5%) or minor errors (43.9%).

Regarding criterion validity testing for the cognition domain, two categories in the CDT were considered: one composed of “no errors + minor errors” and the other “major errors”, as only these are valued as cognitive deficit in the CDT. Clock Drawing Test in the P-EFS showed fair to moderate agreement with P-MMSE either considering 2014 criteria (k=0.356, p<0.001) or 2009 criteria (k=0.422, p<0.001).

Regarding criterion validity, only a fair agreement between P-EFS and CFS scores was found (k=0.224, p<0.001) and as such we could not demonstrate criterion validity for P-EFS using CFS as another accepted measure of frailty.

To test for construct validity, the P-EFS total score was compared with the Katz and MMSE scores. P-EFS total scores were significantly different between patients with a Katz score of 6 (independent on all BADL) and Katz scores <6 (dependent on at least one BADL) (p<0.001).

The P-EFS total score was negatively correlated with the MMSE total score (Spearman rho=−0.357, p<0.001) with increasing P-EFS scores associated with decreasing MMSE scores. When the 2014 and 2009 criteria for cognitive deficit were considered, a statistically significant association existed between the P-EFS total score and MMSE screening cognitive deficit (p=0.002 and p=0.001, respectively).

In the reproducibility study, the mean age was 74.85 (SD=6.32) years and 74.91 (SD=6.50) years in the inter-observer and intra-observer study sample, respectively, with 52.9% (inter-observer) and 48.5% (intra-observer) male, 26.5% (inter-observer) and 21.2% (intra-observer) widowers, and 84.8% (inter-observer) and 57.6% (intra-observer) with no schooling or less than 4 schooling years, respectively (Table 3).

Frailty raw scores and classified into five categories of descriptive results for the inter- and intra-observer study samples are shown in Table 4. The two measurements were obtained by two different investigator in the inter-observer study and by the same investigator at two distinct time points in the intra-observer study.

Regarding re-application of the scale, for the inter-observer test, median was 0 (P25=0; P75=1) days and mean was 3.32 (SD=10.57) days. For the intra-observer test, median was 21 (P25=7; P75=36) days and mean was 30.51 (SD=52.26).

Intraclass correlation coefficient estimates obtained with the inter- and intra-observer study samples were 0.94 (95% CI 0.89–0.97) and 0.85 (95% CI 0.72–0.92), respectively.

The kappa coefficient estimates and percentage of agreement regarding the 11 items of P-EFS and the different P-EFS categories are depicted in Table 5.