An innovative and safe way to train novice Ear Nose and Throat residents through simulation: the SimORL experience| ACTA Otorhinolaryngologica Italica

Abstract

Medical simulation enables trainees to learn procedural skills in a tailored, non-threatening, controlled environment that can provide feedback and educational experiences. The goals of this study are to describe the set-up and execution of an educational intervention (SimORL) in Ear Nose and Throat (ENT) simulation, to report confidence in performing basic ENT procedures before and after the event and investigate whether participants would find it useful and educationally effective. SimORL was a two-day formative event held at SIMNOVA - Eastern Piedmont Simulation Centre, Italy. The event was open to ENT trainees from any Italian ENT training program; participants were divided into 5 teams and rotated around 10 different simulation stations over two days. Stations included: high-fidelity, skill trainer, computer based, wet lab and dissection. Stations were: virtual otoscopy (OtoSim®), simulated clinical cases with high-fidelity mannequin (e.g. epistaxis) or standardised patients (e.g. vestibular neuronitis), robotic surgery (Da Vinci®), human anatomy (zSPACE AIO®), surgical tracheostomy (wet model), cadaveric sino-nasal endoscopy (wet model), crisis resource management (team exercise), surgical sutures (Limbs&Things SkinPad®), surgical set station and team building exercises. Participants were asked to complete a pre- and post-test that queried previous experience and confidence using 10-item unanchored semantic scales. Results are presented as median (25-75 percentile). Satisfaction was assessed by a validated 5-item Likert Simulation Experience Scale (SSES). Twenty-three ENT trainees attended SimORL 2018. Only 3 participants reported limited previous simulation experience. Pre-post confidence significantly improved between before and after the event. Overall satisfaction with Simulation Experience Scale (SSES) was very high with a median of 4.5 of 5. Regarding simulation evaluation, the most appreciated station was nasal endoscopy (10/10), while the least appreciated was otoscopy (6/10). SimORL proved to be a highly rated and useful educational tool to improve junior ENT trainees’ confidence in performing basic ENT procedures.

Introduction

Post-graduate specialist education is a progressive process. Studies show a variable range of competences as junior doctors enter their specialist clinical training ^1,2. Many junior residents are initially not comfortable with the clinical setting as it is perceived as a stressful environment, something that is only experienced to a degree during medical school ³.

It would be advisable, however, that junior trainees entering the clinics are equipped with a homogenous skill-set and clinical competences that allow them to safely approach the first on-calls.

In Italy, most Ear Nose and Throat (ENT) training programs use apprenticeship as their main teaching model (time-based education), where junior trainees learn from direct observation and trial from senior residents and attending physicians. This is a very common model in the medical sciences, and is particularly rooted in surgical disciplines ⁴. This model has some limitations, however, as case and procedure exposure for observation and practice will be dependent on a specific institution’s characteristics, mentors personal subspecialty or preference and case-mix available at any given time. An alternative or integration to time-based education is competency based medical education. Competency-based medical education allows trainees to progress through their training based on competency, rather than fulfillment of a pre-defined amount of time ^5,6.

Medical simulation offers an excellent tool to supplement this model by offering a safe and repeatable learning environment to integrate specific cases and procedures that can be tailored to the resident’s competency needs. Simulation allows the creation of custom-made learning programs that are able to provide junior trainees with basic tools to approach safely, among others, the first year of training and associated on-call duties.

The field of otorhinolaryngology is not new to the use of simulation and much literature is available, especially regarding simulation of specific surgical approaches ^7-11. Recently, some centres have launched short educational events aimed at introducing junior residents to the duties of being a new resident, the so-called boot camp. Almost all published boot-camps in ENT are from North America, are one day long, use simulation as a main teaching tool and focus predominantly on training about common ENT emergencies ^12-17. These events are designed with the goal of developing the cognitive, communication and procedural skills necessary for otolaryngology residency ^18.

The aim of this work is to present and describe the model of SimORL – a two-day simulation event in ENT dedicated to junior residents – and investigate whether participants found the event beneficial to their professional development. We hypothesised that SimORL would increase participants confidence and be perceived as useful and educationally effective in teaching basic ENT topics of interest to the junior trainee.

Methods

SimORL (Simulation in Otolaryngology) is a two-day intensive, focused, training course that covers multiple topics of interest to the junior trainee. SimORL uses simulation as the main educational strategy, including different modalities ranging from cadaver models to high fidelity mannequins.

The first SimORL took place in 2018 at the SIMNOVA Healthcare Simulation Centre, Università del Piemonte Orientale, Novara, Italy.

Participants

SimORL was open to any junior ENT resident across the country. Invitees included novice otolaryngology - head and neck surgery trainees (postgraduate year one and/or two). The event was also advertised through social media channels.

Participants were divided into 5-person teams leveled out by training year. Participants signed an on-line written informed consent as part of their registration process for the event.

Curriculum design and content

The core curriculum of the event was developed to meet the following educational goals: 1) recognise and treat common ENT presentations; 2) perform simple instrumental diagnostics; 3) perform basic surgical interventions (tracheostomy was chosen as the surgical intervention); and 4) communicate effectively with teams and relatives. Over the two days a total of 9 + 1 educational stations were delivered (five per day) in three teaching categories. More information about the stations are reported in Table I. Each group rotated through all of the stations. All stations included debriefing and no assessment was carried out to grade participants in their actions.

Program evaluation

Self-assessment data was collected from participants using anonymous pre-course and post-course surveys. Prior to attending the event, participants were asked to complete a survey on demographics and pre-course simulation experience, as well as confidence levels in performing the skills involved in the event. Post-event surveys assessed participants’ post-training confidence level with each of the skills and abilities presented. Moreover, participants completed a satisfaction questionnaire and expressed an educational effectiveness rating for each of the event stations. The previously validated Satisfaction with Simulation Experience Scale (SSES, a 5-item Likert scale) was used to assess satisfaction ¹⁹. For the other sections, responders were asked to reply using an unanchored semantic scale ranging from “1 - strongly disagree/not confident at all” to “10 - strongly agree/very confident”. Residents were also given the opportunity to include free-text comment about the event, the stations and suggestions for improvement of the event.

Data analysis

Data was collected on paper-based forms and later transformed into digital spreadsheets. Descriptive statistics were summarised using median and percentiles. Pre-post-event differences in self-perception regarding confidence were tested using the Wilcoxon signed-rank test.

Results

Twenty-three ENT residents from eight different training programs around the country participated in the first SimORL; all completed the pre- and post-event questionnaires. The post graduate year (PGY) levels of the 23 participants were: 11 (48%) PGY-1, 6 (26%) PGY-2, 5 (22%) PGY-3, and 1 (4%) PGY-4. 15 (65%) were female. Only 3 (13%) individuals reported limited previous experience with simulation and the remaining 20 (87%) reported no experience with simulation. No individuals were excluded from the analysis.

Confidence

Post-event confidence levels demonstrated significant increases in all six stations. Confidence in performing diagnostic otoscopy improved from 6 (5-7) to 7 (6-8) (p = 0.021); in performing diagnostic nasal endoscopy from 6 (5-7) to 8 (6-8) (p = 0.0003); confidence with surgical anatomy from 4 (3-6) to 7 (6-8) (p < 0.0001); confidence managing common ENT conditions from 6 (5-7) to 7 (6-8) (p = 0.036); confidence performing a surgical tracheostomy from 5 (2-6) to 7 (5-8) (p < 0.0001) and finally confidence in suturing from 6 (5-7) to 8 (7-8) (p = 0.0011). Variations of confidence pre-post including min-max scores per station are presented in Figure 1.

Participant’s satisfaction with the event and the simulation experience

Overall participants found SimORL a useful learning experience, rating it 10 (9-10) of 10 for global satisfaction and educational value. The majority of participants highly rated each of the learning stations for educational effectiveness (Tab. II). The most appreciated station was diagnostic nasal endoscopy, while the least appreciated was the otoscopy station.

Overall, the global SSES rating was very high with a median of 4.5 of 5 (4-5). All satisfaction measures are reported in Table III.

Discussion

The results of our study suggest that SimORL can be a valuable tool to provide didactic, hands-on and confidence building experiences for junior otorhinolaryngology trainees. After completing the SimORL curriculum, participants referred significant improvement in self-confidence in several areas, including improvements in the diagnostics process (otoscopy and nasal endoscopy), general knowledge (anatomy), surgical procedures (suturing and tracheostomy) and complex clinical reasoning (management of common ENT emergencies).

Previous studies suggest a correlation between actual proficiency and self-perceived competence ^20-22.

The high rating in the SSES likely reflects a high-quality formative process including well-delivered expert debriefing with non-judgmental constructive approaches throughout the entire event and the application of theoretical reinforcements of key teaching points by tutors.

The entire program was designed with the concepts of deliberate practice in mind. Based on the Ericsson research on skill acquisition, significant improvements in performance can be realised when individuals are: a) given a task with a well defined goal; b) motivated to improve; c) provided with feedback; and d) provided with opportunities for repetition ²³.

SimORL allowed residents to increase their confidence by practice, trial-and-error and help from supervisors and teachers. A key aspect of the practice of surgical disciplines is the ability to perform practical procedures efficiently and safely. Patient safety, malpractice and increased mortality rates caused by human errors have increased the importance of medical simulation sessions for undergraduate and postgraduate training programs ²⁴. Simulation-based training offers well-described advantages as an educational tool. It provides many opportunities to practice both technical and non-technical skills in a safe environment, learning from achievements and errors without the consequences that may result from mistakes ²⁵.

According to Italian training standards for the ENT residency program, at the end of the training period, residents becoming specialists should have proficiently mastered all the different regions and systems pertaining to the head and neck, including surgery. Other residency programs (such as Anesthesia or Emergency Medicine) include the option to use simulation as a complementary simulation tool in their national training requirements. SimORL could offer the opportunity to safely improve fields of training.

There are previously published studies with similar methodologies. Our results are in line with that of other similar events both in the field of otorhinolaryngology as well as in other medical specialties ^26,27.

Recently, Dean et al. conducted a cross-sectional survey of all otolaryngology residency program directors in the United States and Puerto Rico: based on the participants’ answers, the authors concluded that simulation-based boot camps can be a significant and valuable component of residency training ²⁸.

International data have shown that current training paradigms may fail to adequately prepare surgeons for independent practice ²⁹. Restrictions on trainee working hours, new patient safety initiatives and pressures for increased operating room efficiency have significantly reduced the number and quality of learning opportunities for surgical trainees ^5,30.

This is, to our best knowledge, the first nationwide simulation event in the field of otorhinolaryngology in Italy. SimORL aims to be a stimulus for the integration of training with simulation, an approach to competency-based post-graduate medical education and, hopefully, provides a small contribution to the improvement of patient care.

There are several limitations to this study. The sample size was small and may not be representative of the overall trainee population. There was no summative assessment of skills and performance, as this was excluded to preserve the psychological safety of trainees who are new to the concept of simulation and debriefing, which, however, limits the ability to document improvement objectively.

There was no follow-up over time: this study presents immediate results, when the signals are stronger and no long-term follow-up of confidence self-perception has been performed nor did we assess for knowledge retention over time. Although longer than most ENT simulation events reported in literature, the length is still significantly shorter than other specialty boot camps than can span from one week to one month. The ideal program length is yet to be clarified.

Conclusions

The SimORL experience demonstrated that simulation for ENT residents can be a valid educational tool to improve confidence in performing specific ENT procedures, showing a very high overall global satisfaction rate. Further studies are needed to confirm these encouraging results.

Figures and tables

Figure 1.Radar chart presenting median self-confidence perception level before (dotted white) and after (grey) the event. Apexes of polygon represent each simulation station (1: diagnostic otoscopy; 2: diagnostic nasal endoscopy; 3: surgical anatomy; 4: common ENT management scenarios; 5: surgical tracheostomy; and finally 6: suturing). Axes in each radar chart represent the 10-points semantic unanchored scale.

References

1. Burch VC, Nash RC, Zabow T. A structured assessment of newly qualified medical graduates. Med Educ. 2005; 39:723-31.
2. Sachdeva AK, Loiacono LA, Amiel GE. Variability in the clinical skills of residents entering training programs in surgery. Surgery. 1995; 118:300-8.
3. Moercke AM, Eika B. What are the clinical skills levels of newly graduated physicians? Self-assessment study of an intended curriculum identified by a Delphi process. Med Educ. 2002; 36:472-8.
4. Walter AJ. Surgical education for the twenty-first century: beyond the apprentice model. Obstet Gynecol Clin North Am. 2006; 33:233-6. DOI
5. Wagner N, Fahim C, Dunn K. Otolaryngology residency education: a scoping review on the shift towards competency-based medical education. Clin Otolaryngol. 2017; 42:564-72. DOI
6. Reznick RK, MacRae H. Teaching surgical skills - changes in the wind. N Engl J Med. 2006; 355:2664-9. DOI
7. Arora A, Lau LY, Awad Z. Virtual reality simulation training in Otolaryngology. Int J Surg. 2014; 12:87-94. DOI
8. Clifton N, Klingmann C, Khalil H. Teaching otolaryngology skills through simulation. Eur Arch Otorhinolaryngol. 2011; 268:949-53. DOI
9. Javia L, Deutsch ES. A systematic review of simulators in otolaryngology. Otolaryngol Head Neck Surg. 2012; 147:999-1011. DOI
10. Musbahi O, Aydin A, Al Omran Y. Current status of simulation in otolaryngology: a systematic review. J Surg Educ. 2017; 74:203-15. DOI
11. Stew B, Kao SS, Dharmawardana N. A systematic review of validated sinus surgery simulators. Clin Otolaryngol. 2018;812-22. DOI
12. Chin CJ, Chin CA, Roth K. Simulation-based otolaryngology - head and neck surgery boot camp: ‘how I do it’. J Laryngol Otol. 2016; 130:284-90. DOI
13. Deutsch ES, Malloy KM, Malekzadeh S. Simulation-based otorhinolaryngology emergencies boot camp. Part 3: Complex teamwork scenarios and conclusions. Laryngoscope. 2014; 124:1570-2. DOI
14. Malekzadeh S, Deutsch ES, Malloy KM. Simulation-based otorhinolaryngology emergencies boot camp. Part 2: Special skills using task trainers. Laryngoscope. 2014; 124:1566-9. DOI
15. Malekzadeh S, Malloy KM, Chu EE. ORL emergencies boot camp: using simulation to onboard residents. Laryngoscope. 2011; 121:2114-21. DOI
16. Malloy KM, Malekzadeh S, Deutsch ES. Simulation-based otorhinolaryngology emergencies boot camp. Part 1: Curriculum design and airway skills. Laryngoscope. 2014; 124:1562-5. DOI
17. Smith ME, Trinidade A, Tysome JR. The ENT boot camp: an effective training method for ENT induction. Clin Otolaryngol. 2016; 41:421-4. DOI
18. Yeh DH, Fung K, Malekzadeh S. Boot Camps: preparing for residency. Otolaryngol Clin North Am. 2017; 50:1003-13. DOI
19. Levett-Jones T, McCoy M, Lapkin S. The development and psychometric testing of the satisfaction with Simulation Experience Scale. Nurse Educ Today. 2011; 31:705-10. DOI
20. Ingrassia PL, Franc JM, Carenzo L. A novel simulation competition format as an effective instructional tool in post-graduate medical education. Adv Simul (Lond). 2018; 3:17. DOI
21. Fincher RM, Lewis LA. Learning, experience, and self-assessment of competence of third-year medical students in performing bedside procedures. Acad Med. 1994; 69:291-5.
22. Hicks CM, Gonzalez R, Morton MT. Procedural experience and comfort level in internal medicine trainees. J Gen Intern Med. 2000; 15:716-22.
23. Ericsson KA, Krampe RT, Tesch-Römer C. The role of deliberate practice in the acquisition of expert performance. Psychological Review. 1993; 100:363-406.
24. McLaughlin S, Fitch MT, Goyal DG. Simulation in graduate medical education 2008: a review for emergency medicine. Acad Emerg Med. 2008; 15:1117-29. DOI
25. Berkenstadt H, Erez D, Munz Y. Training and assessment of trauma management: the role of simulation-based medical education. Anesthesiol Clin. 2007; 25:65-74.
26. Barra FL, Carenzo L, Franc J. Anesthesiology Resident Induction Month: a pilot study showing an effective and safe way to train novice residents through simulation. Minerva Anestesiol. 2018; 84:1377-86. DOI
27. Ceresnak SR, Axelrod DM, Motonaga KS. Pediatric cardiology boot camp: description and evaluation of a novel intensive training program for pediatric cardiology trainees. Pediatr Cardiol. 2016; 37:834-44. DOI
28. Dean KM, DeMason CE, Choi SS. Otolaryngology boot camps: current landscape and future directions. Laryngoscope. 2019; 129:2707-12. DOI
29. Bell RH, Biester TW, Tabuenca A. Operative experience of residents in US general surgery programs: a gap between expectation and experience. Ann Surg. 2009; 249:719-24. DOI
30. Temple J. Time for training: a review of the impact of the European Working Time Directive on the quality of training. Medical Education: London; 2010.