Is Virtual Reality the Next Wave for Clinical Assessments?

Oxford Medical Simulation (OMS) designs, builds, and delivers immersive clinical simulated experiential learning opportunities for healthcare students and practitioners. OMS offers learners access to over 240 clinical healthcare simulations scenarios any time, anywhere. These scenarios have been created by an in-house team of experts with experience in direct patient care to develop realistic, clinically-accurate scenarios. This HealthySimulation.com article will explore Oxford Medical Simulation’s virtual reality answer for clinical assessment of learners.

The Challenge With Assessing Clinical Competencies

OMS Create, a no-code authoring platform allows customization of existing scenarios or the option to build a unique simulation from scratch. Versatile by design, OMS empowers learners to practice, independently or as a team, to develop psychomotor skills for procedures, hone communication skills by talking with AI-driven patients and colleagues, or prepare for practice by managing a multi-patient caseload. Combined with evidence-based feedback and precision analytics, learners and educators can gain valuable insights into clinical performance using the robust OMS platform to deliver reliable, standardized clinical training for any learner level.

Today, competency-based education and patient safety are two of the most discussed and highly prioritized components of clinical training. Demonstration of competency is imperative for healthcare professionals related to patient safety and quality of care. The difficulty with clinical competence has largely been around the establishment of a standardized method for competency assessment. Testing is often a measure of knowledge rather than application. While knowledge-testing is necessary, written tests are only one piece of the larger puzzle when determining how well a person will perform clinically.

In short, a learner can know the material but not demonstrate application of the knowledge. WIth a written test, all valuable information must be provided to the learner. However, in simulation, the learner must determine what is valuable and then apply the information to make a clinical decision. There are tests, such as the NCLEX, that acknowledge current limitations and are making strides to include components of competency, like clinical judgment. The OSCEs have limitations, as well, with studies demonstrating limited evidence supporting the reliability of scoring. The rise and continued development of technologies presents another means for effectively measuring clinical performance using VR in a standardized and objective way.

Why And How to Use Virtual Reality For Assessment?

One exciting key feature of the OMS platform is its inherent standardization, which logs objective metrics throughout a scenario and compiles data for review by a facilitator.

Employing a standardized, unbiased assessment process is ideal for ensuring that learners are held to the same standards and receive the same opportunity to excel and demonstrate their clinical skills. However, the incorporation of a new technology into an existing curriculum can feel daunting, but there are several ways to use OMS to support assessments.

Virtual reality has long been used as a platform for high-stakes assessment in emergency medicine (McGrath et al, 2017), and the University of Oxford has been expanding use of OMS for OSCEs since 2021, highlighting the success of virtual simulation for assessing higher-level cognitive skills including clinical decision-making, prioritization, and multi-patient management. Specialized areas of care, like mental health nursing (Kim et al., 2024), have adopted VR as a tool to provide a safe space for practice and assessment. A study investigating interdisciplinary teamwork in the recognition and treatment of pediatric sepsis also demonstrated valid evidence supporting VR in clinical training and assessment (Zackoff, 2023). Simply put, there is evidence that supports the use of VR for assessment, and there are many ways to use this technology for assessments.

Can a VR Scenario be an Assessment Tool

Yes, OMS scenarios can be used as summative assessment in a couple of ways:

1. Use OMS scenarios with platform scoring & analytics: One way VR scenarios can be used to assess learners is simply through one of the most frequently used tools – observational analysis. The OMS platform tracks and logs actions taken throughout the scenario, freeing up facilitators to observe as learners evaluate a case and make clinical decisions in real time. Following the scenario, learners and faculty will each receive a clinically-weighted score, taking into account the learner’s clinical performance, timing, and prioritization. As all learners are measured by the same rubric, the score generated contributes to a more standardized assessment process, using objective metrics and timestamps to support scoring. Through the OMS platform’s objective metrics are used for scoring, feedback, and data to inform observations for a comprehensive picture of clinical performance.
2. Use OMS Create to upload a clinical educator created scoring rubric. Most clinical educators already have existing measurement tools to assess learners’ skills. In this case, clinical educators can use our authoring platform, OMS Create, to upload their own scoring rubric to match OMS scenarios, as well as tracking and assessing specific competencies using the Competency Mapping and Tracking feature. Providing learners with immediate access to evidence-based feedback reinforces reflective thinking and deliberate practice following the simulation, and can be used later to facilitate a more robust debrief, supporting the educators and trainers who will be leading those discussions. Experienced facilitators can then focus elsewhere, resting assured that evaluation is taking place in the background in exactly the way intended.

View the LEARN CE/CME Platform Webinar Immersive Learning: Integrating VR Simulation and Continuity of Care in Nursing Education to learn more!

What to Consider When Developing a VR-Based Assessment

Considering the vast array of VR applications for assessment, selecting the most suitable approach requires careful consideration:

1. Assessment objectives – what are the learner outcomes to be achieved? What skills are learners expected to be able to demonstrate during the patient assessment? If the objectives include communication skills, like history taking, the clinical educator might want a VR simulation that allows learners to speak directly with virtual patients. To expose learners to more specialized areas of care that are not easily replicated by traditional methods, like mental health or pediatrics, OMS scenarios can cover these areas while at the same time exposing learners to a diversity of virtual patients, just as in real world practice. Knowing what to expect from learners can help inform what type of simulation to use. Then the final aspect to consider is the tests and measures to use to effectively determine clinical performance.
2. Tests and measures – how will the objectives be measured? Once measures have been decided on, a more specific determination needs to be made about the type of simulation that will best suit the learners. Will the clinical educator use the platform as the analysis tool, a created assessment metric or a combination? In addition to the clinical performance score and evidence-based feedback, the OMS platform houses robust data & analytics, which can be used to track progress, identify trends, and remediate knowledge gaps. With the Competency Mapping & Tracking feature, as an example, both a learner and an educator can view how the actions taken in the scenario align with the specific components of a competency framework. Using the data & analytics provided, educators can see, on an individual or cohort-level view, how learners are progressing towards their competencies. These metrics add yet another layer of standardization to the assessment process and can bolster the clinical educators’ decisions with objective data to support their curriculum choices.

Leverage OMS Virtual Reality to Support Clinical Assessments

References:

• McGrath JL, Taekman JM, Dev P, Danforth DR, Mohan D, Kman N, Crichlow A, Bond WF. Using Virtual Reality Simulation Environments to Assess Competence for Emergency Medicine Learners. Acad Emerg Med. 2018 Feb;25(2):186-195. doi: 10.1111/acem.13308. Epub 2017 Oct 11. PMID: 28888070.
• Kim GM, Lim JY, Kim EJ, Yeom M. Impact of Virtual Reality Mental Health Nursing Simulation on Nursing Students’ Competence. J Multidiscip Healthc. 2024 Jan 11;17:191-202. doi: 10.2147/JMDH.S435986. PMID: 38226028; PMCID: PMC10789577.
• Zackoff, Matthew W., Bradley Cruse, Rashmi D. Sahay, Bin Zhang, Tina Sosa, Jerome Schwartz, Holly Depinet, Daniel Schumacher, and Gary L. Geis. “Multiuser immersive virtual reality simulation for interprofessional sepsis recognition and management.” Journal of Hospital Medicine (2024).