Simulation has been used for training in professions that require precise cognitive and physical tasks in high-risk environments, with potentially fatal complications. Currently, many non-medical professions require simulation as part of routine training or maintenance of competency and annual skills assessment.1 Multiple high-fidelity medical simulators have been developed, over the past decade, to address the 21st century challenges of rapid expanding new technologies, restrictions in working hours and a demand by regulatory bodies for simulation to be implemented in the medical field.2

Interventional cardiology (IC) is a fertile field in which simulation can blossom, because of its highly complex procedures with a long learning curve and that involve life-threatening complications that are prone to a variety of medical errors.3 By using simulation in IC, the traditional approach “see one, do one, teach one” can be replaced with “learn the operation before the operation room”.4 However, an international survey of 172 cardiologists showed that only 48% had already participated in simulation training, even though 91% considered it to be “necessary” in cardiology.5 The prevalence of medical errors has been evident since the publication of the Institute of Medicine's report “To Err Is Human”.3 Patient safety and prevention of medical errors define one of the rationales for simulation training.

Three-dimensional model simulation is a growing novel tool that can be used for educational purposes, training or individualized medicine, and even patient empowerment. Its versatility enables several shortcomings of clinical simulation to be solved, enabling the standardization of a simulation platform with educational cases that are cheaper and more practical than traditional or cadaveric training.6 Simulators based on 3D printing offer an alternative way of learning in an immersive reality with real materials where trainees can make mistakes, repeat, and learn percutaneous intervention skills in a controlled, safe, and realistic environment.

Many studies have shown improvement in operator skills using simulations over traditional mentor-based training in specific IC skill sets.7–10 Simulator-based training in coronary angiography improved operator skills compared with traditional mentor-based training – including, a shorter procedure time, lower radiation dose used, and a higher global procedure skill score.7

The Education and Training European Association of Percutaneous Cardiovascular Interventions (EAPCI) published a recommendation for simulation sessions to be incorporated in training centers of IC11 and the Accreditation Council for Graduate Medical education requires cardiovascular fellowship training programs to include some component of simulation as part of fellow training.2 Therefore, in the future, it is expected that simulations will be incorporated into training programs and certification examinations for an interventional cardiologist.

To ensure that simulators provide a realistic comparison to real-life environment they must undergo scientific validation, according to different levels of evidence, following the Kirkpatrick model for evaluating the effectiveness of training.12

This study aims to evaluate a novel 3D-printed simulator (SimulHeart®) for face and content validity in IC simulation.

Fifty-six participants completed the study: 16 “experts”, 26 “novices” and 14 NAP from sixteen hospitals across Portugal. No participant had prior experience with the SimulHeart® simulator. Sociodemographic characteristics are explained in Table 2.

The questionnaire showed good values of internal consistency, with a global reliability of 0.93, measured by Cronbach's alpha. The items of face, content, and satisfaction showed an internal consistency of 0.85, 0.81, and 0.87, respectively.

The overall mean score of face validity was 4.38 ([SD 0.35]) (with classifications varying from 2 to 5). The individual frequencies of the items that evaluated face validity are described in Figure 3.

Figure 3.

Mean face validity of individual items of the post-simulation training questionnaire.

(0.32MB).

The overall mean score of content validity was 4.69 ([SD 0.32]) (only varying between 4 and 5 classifications) – Figure 4.

Figure 4.

Mean content validity of individual items of the post-simulation training questionnaire.

(0.21MB).

In both face and content validity, there was no statistically significant difference in the scores of “experts” and “novices”.

Optional written narrative qualitative assessment by participants showed several common themes. Participants found the simulator to be “realist”, have “good fidelity” and be an “enriching formative experience”. Some participants, however, considered that the course should have “more hours of individual training”.

In the questionnaire, satisfaction and self-efficacy were also measured (as shown in Figure 5), 80.4% strongly agreed that the course was well executed and interactive, 75.0% strongly considered the course improved their theoretical knowledge, 60.7% also considered it strongly improved their technique. 67.9% strongly agreed that after the simulation training, they felt confident to explain the procedure to a patient. 82.1% agreed or strongly agreed that after the simulation they felt confident to perform the procedure on a patient. Most participants (85.7%) strongly agreed they would recommend the course to colleagues.

Figure 5.

Percentage agreement on satisfaction and self-efficacy of individual items of the post-simulation training questionnaire.

(0.26MB).

The mean score (on a ten-point Likert scale, with 1 being not relevant and 10 very relevant) in general terms for the relevance of using this model in training was 9.41 (SD 0.80, Range [7–10]).

This study showed that the SimulHeart® simulator for percutaneous interventions met the criteria for both face and content validity at a “good” level, based on the predefined definitions of validity. The simulator presents excellent realism and simulates all the procedure steps. Also, there was a high level of satisfaction and self-efficacy with the training in this model, with increased confidence.

Simulation is breaking barriers in the modern era of medical and surgical education. With ever increasing pressures on surgical performance, the profession is eagerly looking for training systems that are novel, reproducible, and validated. The reality is trainees are operating less than ever before21 due to shortened training programs, reduced working hours,22 and advancement of medical and minimally invasive techniques, therefore the application of 3D simulation technology is evident. This is particularly important in training for minimally invasive percutaneous procedures, which are complex and leave little space for error. In this area a valid 3D simulator may reduce the learning curve and improve patient safety. This modality of simulation affords a unique opportunity for trainees to practice reality-based surgical skills without any risk to the patients.

While there are studies on 3D simulators for patient-specific percutaneous interventions,13,23–30 there is minimal data in the literature on the validation of 3D model simulators for training in cardiology. Common benchmarks on which simulators are judged include reliability, face, content, construct, and predictive validities.31 Even though there are some validated training devices and protocols for coronary angiography, this area has few well-established or validated tools.

A 2019 study with Simbionix Angio-Mentor (Simbionix USA, Cleveland, Ohio) documented a significant improvement of skills in real-world practice (coronary angiography in patients) after simulator-based training.7 According to Popovic et al., the simulation group showed significant improvement in respect to contrast use, procedural time, fluoroscopic time, and global performance score in coronary catheterization in patients in the cardiac catheterization lab after four hours of high-fidelity simulation training in comparison to a control group. These findings documented an improved intra-operator performance in the clinical setting before and after simulation training demonstrating the impact of simulation on the transference of skills to real-life practice. They concluded that “simulation can be used as an assessment tool by defining a mastery threshold ensuring all individuals have reached a predefined level of proficiency to enable safe patient care”.

However, there is room for improvement. The criteria with the lowest scores were “resembles living human tissue”, with a mean score of 4.00, and “simulates the fluoroscopic appearance (X-ray)”, with a mean score of 4.08. These items would score higher if the simulation took place in an actual catheterization laboratory, with radiation and sterile drapes. That environment undoubtedly lacks practicality for widespread practical courses.

Despite the possible limitations, most participants (96.40%) “agreed” or “strongly agreed” that the training in the simulator should be integrated into the cardiology residency curriculum. Furthermore, the improved subjective confidence of “novices” participants suggests a benefit from training in the simulator and supports its use as an educational tool.

The categorization of participants as “experts” or “novices” was based on the definition of the EAPCI curriculum and was self-reported in the questionnaire by the participants. One of the limitations of our study was overall participant numbers were low (n=56), a more significant sample could have more statistical power to detect differences between groups and to evaluate construct validity. Expanding the study to a larger number of participants would be reasonable based on our results.

For future investigation construct validity should be performed on the model. Further research is required to correlate simulator performance to clinical performance in a real patient.

The SimulHeart® simulator built by 3D CardioSolutions showed a good level of face and content validity. There was a high level of satisfaction and efficacy, as well as improved confidence. Both “experts” and “novices” participants agreed that training in the simulator should be incorporated into the cardiology residency curriculum.

Based on our study, we suggest training in this simulator should be used in interventional cardiology for medical residents, fellows, and allied professionals to gain experience and skill in a safe environment.

The authors have no conflicts of interest to declare.