May 22, 2020By Dr. Kim Baily

Using Healthcare Simulation to Evaluate Checklists: Part 1

Although checklists are widely used in high reliability organizations such as aircraft carriers and electrical power grids, they are not so frequently used in medicine, despite evidence that they can reduce medical errors and mortality. Today’s article by guest author Dr. Kim Baily PhD, MSN, RN, CNE, previous Simulation Coordinator for Los Angeles Harbor College and Director of Nursing for El Camino College, explores how medical simulation can be used to evaluate the potential of patient safety and procedure checklists.

Hayes et al., followed eight hospitals in eight cities (Toronto, Canada; New Delhi, India; Amman, Jordan; Auckland, New Zealand; Manila, Philippines; Ifakara, Tanzania; London, England; and Seattle, WA) before and after the introduction of surgical checklists. The hospitals participated in the World Health Organization’s Safe Surgery Saves Lives program. Data related to surgical complications including death of patients was collected from 3733 consecutively enrolled patients 16 years of age or older who were undergoing noncardiac surgery prior to the introduction of a checklist.

Following the introduction of the Surgical Safety Checklist, 3395 patients were evaluated for the same data. The results of the study found the rate of death was 1.5% before the checklist was introduced and declined to 0.8% afterward (P=0.003). Inpatient complications occurred in 11.0% of patients at baseline and in 7.0% after introduction of the checklist (P<0.001). This represents a significant reduction (Hayes et al., A Surgical Safety Checklist to Reduce Morbidity and Mortality in a Global Population, New England Journal of Medicine 360 (2009): 491-499.


Sponsored Content:


Medical simulation can play a critical role in the development, implementation and evaluation of checklists. For example, Arriaga et al. used high-fidelity healthcare simulation to evaluate the effectiveness of a surgical crisis checklist (Arriaga et al., Simulation-Based Trial of Surgical-Crisis Checklists. N Engl J Med 2013; 368:246-25. DOI:10.1056/NEJMsa1204720). Operating room crises maybe common events in large hospitals but can be rare for individual clinicians. The primary outcome measure was failure to adhere to critical processes of care.

Participants were also surveyed regarding their perceptions of the usefulness and clinical relevance of the checklists. A total of 17 operating-room teams participated in 106 simulated surgical-crisis scenarios. Teams consisting of anesthesia staff (attending physicians, residents, and certified registered nurse anesthetists), operating-room nurses, surgical technologists, and a mock surgeon participant were randomly divided into two groups.

Both groups participated in a series of surgical-crisis simulation scenarios however one group used set crisis checklists and the other had to respond to the crisis based on memory alone. The study found that failure to adhere to lifesaving processes of care was less common during simulations when checklists were available (6% of steps missed when checklists were available vs. 23% when they were unavailable, P<0.001). Every team performed better when the crisis checklists were available than when they were not. A total of 97% of the participants reported that if one of these crises occurred while they were undergoing an operation, they would want the checklist used.



In another study involving medical simulation, Sevilla-Berrios et al., developed a novel electronic tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN) to facilitate standardized evaluation and treatment approach for acutely decompensating patients. This study included a “prompter” who was a team member separate from the leader who records and reviews pertinent CERTAIN algorithms and verbalizes these to the team. Volunteer clinicians with valid adult cardiac life support (ACLS) certification were invited to test the CERTAIN model in a high-fidelity simulation center.


Sponsored Content:


The first session was used to establish a baseline evaluation in a standard clinical resuscitation scenario. Each subject then underwent online training before returning to a simulation center for a live didactic lecture, software knowledge assessment, and practice scenarios. Each subject was then evaluated on a scenario with a similar content to the baseline.

All subjects took a post-experience satisfaction survey. Eighteen clinicians completed baseline and post-education sessions. CERTAIN prompting was associated with reduced omissions of critical tasks (46 to 32%, p < 0.01) and 12 out of 14 general assessment tasks were completed in a more timely manner. The post-test survey indicated that 72% subjects felt better prepared during an emergency scenario using the CERTAIN model and 85% would want to be treated with the CERTAIN if they were critically ill. Prompting with electronic checklist improves clinicians’ performance and satisfaction when dealing with medical emergencies in high-fidelity simulation environment. The authors note that this study did not compare the effectiveness of CERTAIN prompting vs training.

Get Your Copy of The Checklist Manifesto Here!


Have a story to share with the global healthcare simulation community? Submit your simulation news and resources here!


Sponsored Content: