Actions, Communication, and Teaching in Simulation Tool (ACTS)

As part of the Evaluating Healthcare Simulation tools, the Actions, Communication, & Teaching in Simulation Tool (ACTS) was developed by Jill S. Sanko, PhD, MS, ARNP, CHSE-A, FSSH, Ilya Shekhter, MS, MBA, CHSE, David Birnbach, MD, MPH, and Karina Gattamorta, PhD, EdS, to provide an objective way to evaluate confederates’ contributions to simulation encounters. Given the need to be able to measure all aspects of simulation to make improvements and noting that confederates make errors that impact educational opportunities, Sanko and colleagues (2016) embarked on a quest to design and develop a tool that could measure confederates’ “performance” for the purposes of quality improvement. The ACTS tool is a single-factor, five-item measure using a seven-point behaviorally anchored scale scoring schema designed to objectively measure the performances and portrayal accuracy of confederates playing support roles in simulation scenarios.

Download the Actions, Communication, and Teaching in Simulation (ACTS) tool. 

Permission to Use FREELY: General use is already permitted by posting the statement: I understand that I have been granted permission by the creators of the requested evaluation instrument to use it for academic, clinical and/or research purposes. I agree that I will use the evaluation instrument only for its intended use, and will not alter it in any way. I will share findings as well as publication references with the instrument creator(s). I am allowed to place the evaluation instrument into electronic format for data collection. If official ‘Permission to Use’ letter is required, please contact the primary author. Include the purpose of the official request (research, grant), the intended use of the tool and with what population.

Why the Actions, Communication, and Teaching in Simulation Tool was Developed: The ACTS tool was developed to provide an objective way to evaluate confederates’ contributions to simulation encounters. There were no validated tools published in the literature designed to objectively measure confederates. Given the need to be able to measure something to make improvements and noting that often confederates make errors that impact educational opportunities, we embarked on a quest to design and develop a tool that could measure confederates’ “performance” for the purposes of quality improvement.

How the ACTS Tool was Developed: The ACTS tool was developed using a process of face and construct validity, surveying experts, and psychometric evaluations through piloting. Nine experts in the fields of simulation and theatre were asked to rate confederates in a series of video recordings of four scenarios.

Reliability and Validity of the ACTS Tool: Initial evaluation demonstrated excellent reliability with a Cronbach’s alpha of 0.93, and good consistency with item total correlations above 0.52 for all items in the measure.

Using the ACTS Tool: The ACTS tool is a single-factor, five item measure using a seven point behaviorally anchored scale scoring schema. The findings of initial usage and testing of the tool provide evidence to support the reliability and validity of this tool designed to objectively measure the performances and portrayal accuracy of confederates playing support roles in simulation scenarios.

Inter-rater Reliability: Inter-rater reliability overall had good agreement with a Pearson’s r value of 0.91.It is vital that inter-rater reliability be established prior to using the ACTS tool to evaluate confederates to ensure that observers are interpreting the tool in the same way.

Evaluation: The information gained from use of the ACTS tool can be used to individually evaluate confederates as well as to identify development needs and prioritize confederate development activities.

Scoring: Each of the 5 topic points are scored on a 0 (inadequate) to 6 (outstanding) scale. Total scale scores range from 0 to 30 possible.

Download the Actions, Communication, and Teaching in Simulation (ACTS) tool.

References: For additional detail regarding the development and psychometric analysis of the ACTS:

• Sanko, J. S., Shekhter, I., Gattamorta, K. A., & Birnbach, D. J. (2016). Development and psychometric analysis of a tool to evaluate confederates. Clinical Simulation in Nursing, 12(11), 475-481. http://dx.doi.org/10.1016/j.ecns.2016.07.006
• Webpage suggested citation: Sanko, J. S., Shekhter, I., Gattamorta, K. A., & Birnbach, D. J. (2018). Actions, Communication, and Teaching in Simulation. Retrieved from: https://www.healthysimulation.com/tools/evaluating-healthcare-simulation/actions-communication-and-teaching-in-simulation-tool
• If you need an official ‘Permission to Use’ letter, or have questions, please contact Dr. Jill Sanko.

—

Return to the Evaluating Healthcare Simulation tools webpage.