Pulmonary embolism (PE) is a life-threatening condition that presents most of the time with non-specific signs and symptoms.1 It represents 2–9% of all causes of out-of-hospital cardiac arrest and it can be associated with an extremely unfavorable prognosis.2

In the absence of contraindications, systemic thrombolysis is the standard treatment for hemodynamically unstable PE patients, as it has been shown to improve survival.1 As the adverse effects of thrombolysis-related bleeding can be devastating, it should only be performed in patients with a high suspicion of or confirmed PE, whose risk/benefit profile favors thrombolytic therapy.1 However, given the lack of specificity in the clinical presentation of PE, the decision to perform fibrinolysis is challenging. Most clinicians may be highly resistant to recommending empirical fibrinolysis due to the uncertainty of the diagnosis.

Computed tomography pulmonary angiography (CTPA) is often required to establish the definitive diagnosis of acute PE.1 However, CTPA cannot be performed in hemodynamically unstable patients and it is not available in out-of-hospital settings and even in a hospital setting, it is not often readily available. Although echocardiography may be helpful in establishing the diagnosis, it is generally unavailable in an out-of-hospital setting, and only a few clinicians can perform a reliable examination to indicate empirical fibrinolysis. These constraints may explain why only a minority of hemodynamically unstable PE patients receive thrombolytic therapy.3 Thus, safely identifying patients with high-risk PE who may benefit from empirical thrombolysis is challenging but critical to delivering proper treatment, thus avoiding exposure to unnecessary bleeding risk.

An electrocardiogram (ECG) is a low-cost, rapid, and widely available test that cardiologists and non-cardiologists have used for decades. Previous studies that applied ECG scoring systems for acute PE diagnosis demonstrated a reasonable performance, but it is a time consuming process, requires expertise for ECG interpretation and is not routinely implemented in clinical practice.4,5

Artificial intelligence (AI) is increasingly being applied in various fields of cardiology, including ECG analysis.6–8 Several studies reported using AI to analyze ECG signals in order to detect cardiovascular conditions, such as arrhythmias, left ventricular dysfunction and hypertrophic cardiomyopathy.7–10 In this study, we aimed to develop and validate a deep learning-based AI model for detecting PE using a conventional 12-lead ECG and to compare the results of this model with the ones obtained with the traditional clinical prediction rules.

In this study we included 1014 patients admitted to an ED who underwent CTPA due to suspected PE. Of those, data from 911 patients were used for the development of the deep learning model (derivation cohort) and 103 patients were used for testing and validation (validation cohort). As the development of the AI model was previously described, in the results section we focus on the performance of the AI model in the validation cohort.

A total of 103 patients were included in the validation cohort: mean age 69 (IQR 52–81) years, 53% male. Of those, PE was confirmed by CTPA in 38 patients (37%). Regarding patients with PE, the vascular allocation of emboli showed a predominantly central distribution (n=34, 89%), affecting main and lobar arteries (47% and 42%, respectively). Bilateral involvement was documented in 16 patients (42%), and 22 patients (58%) had evidence of right heart strain. The prevalence of classical risk factors for venous thromboembolism in patients with and without PE is represented in Table 1.

There were no differences in pretest clinical probability evaluated by Wells and revised Geneva scores between both groups. Although elevated in all patients, those with PE had significantly higher D-dimer levels compared to those without PE (3495 (IQR 1743–9640) vs. 1804 (IQR 730–3335), p=0.003).

In addition, regarding typical electrocardiographic findings for PE, such as tachycardia, right bundle branch block, S1Q3T3, and T wave inversion or ST depression in V1 to V4 leads, no difference was found between patients with and without PE (p>0.05 for all). No difference was found in Daniels’ ECG score between patients with and without PE (median 2 (IQR 0–3) and median 2 (IQR 0–2), p=0.584).

Table 2 shows the diagnostic performance of the different prediction rules for PE. In our cohort, we found that the use of a fixed D-dimer threshold of 500 ng/mL or an age-adjusted cut-off was associated with similar sensitivity, although the use of an age-adjusted cut-off was associated with non-statistically significant higher specificity for both Wells and Geneva scores (p=0.125). However, even considering the age-adjusted cut-off, the specificity remains low (18%). Globally, the AUC for both Wells and Geneva score combined with a cut-off of 500 ng/mL and an age-adjusted cut-off suggests nearly no discriminative power. The YEARS and PEGeD algorithms were both associated with a non-significant decrease in sensitivity compared to Wells and Geneva algorithms, with a significant increase in specificity (p=0.001 for both). However, even with the use of this algorithm, the specificity remains above 31%. The AI model was associated with a significantly decrease in sensitivity compared to the other models (p=0.003 for YEARS and PEGeD algorithms; p=0.001 for Wells and Geneva algorithms combined with fixed or an age-adjusted cut-off). The documented specificity for the diagnosis of acute PE of the AI model was 100%, which is significantly higher compared to the other models (p<0.002 for all of them). Use of the new deep learning model for the identification of acute PE among ED patients achieved an AUC of 0.75 (95% CI 0.66–0.82; p<0.001), performing significantly better than all other models (p<0.05 for Wells and Geneva score combined with fixed and age-adjusted cut-off, YEARS and PEGeD algorithms). A graphical representation of the ROC curves is shown in Figure 3.

Figure 3.

Receiver operating characteristics (ROC) curve demonstrating the diagnostic performance of different decision rules to predict pulmonary embolism.

(0.17MB).

In this study, we developed and validated a deep learning-based model for PE detection using a 12-lead ECG, which has the major advantage of 100% specificity. To our knowledge, this study is the first to develop a deep learning model for detecting acute PE based solely on ECG.

The management of PE in the ED is of critical importance to decrease mortality rate. Given the potential benefit of anticoagulation or systemic thrombolytic therapy, every effort should be made to diagnose acute PE quickly and accurately. However, clear medical history is often difficult to obtain in most patients, particularly those with hemodynamic instability, who benefit the most from thrombolytic therapy.

There are many diagnostic tools with high sensitivity for diagnosing PE.1 In our study, the combination of clinical prediction scores, such as the Geneva and Wells scores, with D-dimer measurement was associated with around 90% of sensitivity, similar to the previously documented.20,21 However, these strategies were associated with a lack of specificity,20,21 which can be as low as 12% in our study. Thus, besides being optimal for a triage test, no confident decision regarding reperfusion therapy or anticoagulation should be made based on these tests. Furthermore, CTPA availability is often delayed due to high demand in very busy EDs, absence of equipment in some hospitals (or out-of-hospital presentation) or hemodynamic instability. An empirical decision for anticoagulation or systemic thrombolysis is often necessary in all such cases.22,23 In a systematic review and meta-analysis, transthoracic echocardiography was demonstrated to have a specificity of 83% (95% CI, 74–90%) for the diagnosis of acute PE, making it potentially adequate as a rule-in test in patients unable to get confirmatory studies.24 However, considering the potential harm of antithrombotic therapy, a higher level of specificity would be desirable. In addition, several Portuguese hospitals do not have a 24-hour on-site attending cardiologist, which would enable the immediate performance of a comprehensive echocardiogram. Focused cardiac ultrasounds performed by emergency physicians have demonstrated good agreement for identifying right ventricular enlargement compared to those performed by a cardiologist (96%, k=0.86, 95% CI 0.82–0.90).25 However, less experienced operators may have a high level of uncertainty in indicating systemic thrombolysis.

Although several ECG abnormalities are commonly associated with PE, all of them are associated with low sensitivity and specificity.4,5 ECG scores combining several ECG criteria demonstrate better performance compared to single ECG criteria for the diagnosis of PE.4,5 The Daniel score comprises several ECG variables and demonstrated a specificity of 98% (95% CI 96–99%) but low sensitivity (24%; 95% CI 16–31%).4 In addition, ECG scores are time consuming, poorly validated and not routinely used in clinical practice.

Artificial intelligence has been applied in several healthcare applications in recent years. Considering the lack of specificity in clinical presentation and available non-imaging diagnostic tests, our focus in this study was to develop a deep learning model centered on high specificity for the diagnosis of acute PE based on a readily available 12-lead ECG. This is relevant considering that early identification of patients with acute PE is essential, particularly where confirmatory CTPA cannot be immediately performed. In these patients, early decision to initiate anticoagulation or systemic thrombolysis may be lifesaving.

In this study, we developed a deep learning model based only on ECG signals capable of detecting PE with high specificity, which is substantially higher compared to the commonly used clinical prediction scores combined with D-dimer measurement. Although the imaging tests, such as CTPA, are still the gold standard for the diagnosis of PE, a non-image-based easily available test with 100% specificity may support the decision for anticoagulation or systemic thrombolysis in patients with PE.

We demonstrated that the incidence of typical ECG features for PE (such as tachycardia, S1Q3T3, right bundle branch block and V1-V3 T wave inversion and ST depression) was similar between patients with and without PE. In our study, we found no difference in Daniels’ ECG score between patients with and without PE (median 2 (IQR 0–3) and median 2 (IQR 0–2), p=0.584). Thus, the deep learning model provides information beyond these findings and can improve PE prediction.

We found that embeddings from raw ECG data significantly contributed to model prediction and are acquired through multiple convolution operations over the raw data ECG values, representing nonobvious features. The most relevant characteristics of the signal are selected by the model using attention mechanisms, which provide information beyond what is contained in traditional ECG measurements. This led to an improvement compared to the human capacity for PE prediction based on ECG analysis. Although the deep learning model was associated with lower sensitivity compared to the alternative approaches, the primary purpose of its development was to promote a high specificity, as we considered that the greatest gain a new diagnostic tool for PE diagnosis is for patients with suspected PE who cannot immediately undergo a CTPA where a decision about empirical anticoagulation or fibrinolysis should be made. This is especially true in high risk cases. Thus, in the future we aimed to externally validate our model and to test its performance in patients with high risk PE suspicion in which CTPA was not performed before the fibrinolysis decision.

Our study has major strengths. First, our model uses only ECGs as input and does not require additional clinical data, simplifying real world applicability. In addition, we did not exclude ECGs based on cardiac rhythm, meaning that our model is applicable regardless of cardiac conditions, including pacing rhythms, enhancing its generalizability. The retrospective study design and the relatively small sample limit the model's strength, a limitation that can be overcome with more data. In addition, it must be safeguarded that beyond being a promising diagnostic tool, this work represents our first results. Further application in larger cohorts and external validation of the deep learning model are essential to fully validate its performance. These will be conducted prior to pursuing implementation in clinical practice. Lastly, despite the potential usefulness of this technology in patients with hemodynamic instability or out-of-hospital cardiac arrest, our cohort did not focus on these patients, but rather on all patients with suspected PE admitted to an ED. As a result, it should be clear that the application of this tool for this subgroup is still exploratory and requires specific validation testing.

We aim to validate our results in an external sample from other centers in the future, as well as in high risk patient subgroups, for example, patients with hemodynamic instability. From a technical perspective, our model is simplistic enough to be easily integrated into existing information technologies structures and smartphones, either via existing apps or in standalone software. As a result, we expect deployment in clinical practice may occur within a relatively short time period.

In this study, we developed and validated a deep learning model with very high specificity for acute PE diagnosis. Our study provides evidence to support the real world application of AI in routine clinical practice, improving the management of patients with PE.

The authors have no conflicts of interest to declare.