Clinical Simulation Assessment: Time & Performance Measures to Design Scenarios

Today, many medical facilities and institutions employ an educational or training program that incorporates some form of healthcare simulation to achieve learning outcomes. Further, the notion of a standardized methodology for incorporating difficulty into a healthcare simulation scenario often appeals to the desire for reliability, or validity, or possibly both. To enhance these outcomes and increase reliability and validity, there is now a student simulation assessment tool available to help professionals design difficulty into their clinical simulation scenarios, as well as gauge student performance. In this HealthySimulation.com article, guest author Lorenzo Saenz RN, simulation coordinator at Western New Mexico University School of Nursing and Kinesiology, highlights the importance of clinical simulation assessment and explains how time and performance measures can be used to design scenarios. Lorenzo writes…

For many years I had assumed that the medical condition of the simulated patient was the sole driving force behind the learning outcomes and expected actions of our nursing students in simulations. This patient was a post OP hip, that patient an obstetric patient who was about to give birth, and so on. I incorrectly tried to apply the patient’s condition to what I wanted the student to accomplish. When the student was inevitably unable to see the course of treatment to fruition I was left frustrated and questioning my approach. In her book “Simulation In Nursing Education From Conceptualization To Evaluation,” Dr. Pamela Jeffries states that problem-solving is related to the level of complexity of a simulation (Jeffries, 2020). When I read those words, I had an epiphany.

There are problems to be solved within the framework of every simulation. Those problems can be placed on a linear axis and more importantly, as a nurse educator, those problems could be designed and placed to reflect the nursing process! They could just as easily be applied to EMS, law enforcement, or anyone who builds scenarios and uses simulations to gauge student understanding.

Performance Reference Points

Any of these problems can be placed at the start of the clinical simulation.  They can also be programmed to happen as the clinical simulation progresses. Each of these problems should have similar aspects to their nature. There should be a cue for the participant to know there is a problem. The cue can be anything from physiological signs such as cyanosis to stated complaints of pain by a confederate to alarms on a simulated monitor.

Whatever the nature of the cue is, it should equate to a successful recognition of that problem by the participant or in some instances a failure by the participant to recognize the problem. Recognition of the problem should lead to an assessment to determine severity, and cause.

The assessment leads to an intervention. The intervention requires the reassessment to establish the efficacy of the intervention. Due to the fact that I place these “problems” on a timeline the problems become points of reference when evaluating student performance. I have replaced the term problem with a performance reference point as I believe the terminology required a minor shift in how it impacts scenario design. Since I am placing the performance reference points on a timeline I can also begin to establish performance norms and baselines as they pertain to time within the scenario or clinical simulation.

Using Performance Reference Points to Design Difficulty

Now that we have established that performance reference points or PRPs are essentially problems with an associated cue, assessment, intervention, and reassessment that can be placed on a timeline, let’s explore how we can use PRPs to design difficulty. Let’s say for instance that I am working with first-semester nursing students. They have only recently learned how to apply oxygen to a patient via nasal cannula. If I am using a high-fidelity simulator I would set the monitor to read an SPO2 to 85 percent. This would be the cue for the participant.

The student would recognize the alarm, assess the reading of low oxygen via the monitor, and intervene by raising the head of the bed and applying oxygen. The reassessment of the patient would occur when the percentage of oxygen begins to rise and the participant acknowledges the change in the “patient” condition. At this point, we could end the simulation. If however, we wanted to reinforce the concept of documentation at a later date we would incorporate an electronic healthcare record with an order for that oxygen.

This would force participants to first ensure that an order for oxygen exists and would add complexity to the scenario by incorporating an entirely new PRP. We now have objective data in the way of time between the components of the PRP. So, from cue to recognition, from recognition to assessment, from assessment to intervention, and from intervention to reassessment there will be timestamps that as the students become more proficient should decrease. We also have objective performance data in the time it takes for students to successfully complete multiple performance reference points.

This is where the majority of meaningful data will come from in intermediate and advanced students. So assume for a second that the patient scenario is now an asthma patient. The performance reference points could include elevating the head of the bed, administration of oxygen, a nebulizer or metered-dose inhaler, and perhaps a steroid. We have now greatly increased the complexity of the scenario by instituting a wide range of performance reference points.

What is X without Y?

The use of a timeline is a great tool to measure how fast a student performs a task. It can measure if they are faster or slower than they were last semester or last week. Rapidity, however, is only half of the story. The other half is the slightly more subjective assessment of performance. In this area, there are multiple rubrics and tools which measure whether a skill was performed correctly or not. Whether the performance by the participant is an assessment or a treatment I have an affinity for yes or no questions. As such the Y-axis of this tool has five yes or no questions for every performance reference point. Where yes is equal to a number greater than zero and no is equal to zero. They are as follows:

1. Was the PRP done at all? (this would require the full gamut of recognition assessment intervention and reassessment)
2. Was it done safely?
3. Was the action on the part of the participant appropriate with regards to the condition of the patient?
4. Was the process of completing the PRP communicated to team members?
5. Upon completion was the task or assessment documented?

Together time and performance measures can be used to design scenarios, help to ascertain participant performance, and hopefully gives a starting point for your simulations to be more standardized and reliable.

Reference:

• Jeffries, P. R. (2020). Simulation in nursing education from conceptualization to evaluation. Wolters Kluwer.

Learn more about this Student Simulation Assessment Tool

Today’s article was guest authored by Lorenzo Saenz RN, Simulation Coordinator at Western New Mexico University School of Nursing and Kinesiology.

Have a story to share with the global healthcare simulation community? Submit your simulation news and resources here!