Quantum Performance Assessment System From ObjectivityPlus Enhances Graduate Nurse Education
In a recent MTM article Donna Faye McHaney, DNP, MSN, MIS, APRN, FNP-BC, Mary Atkinson Smith, DNP, FNP-BC, FNAP, Nicole Peters Kroll, PhD, ANP-C and Stephen C. Hetherman, EdD shared how to “Use Technology and Simulation to Enhance Graduate Nurse Education” — which utilized the Quantum performance assessment tracking system from ObjectivityPlus. They found that “with the use of Quantum and evidence gathered to validate inferences, students can be assessed over time to evaluate progression of skills indicating knowledge acquisition and preparedness for practice throughout the program. As technology continues to grow and spread, it is important for educational institutions and faculty to contribute to technology utilization and transformational learning in the academic environment. To do so, requires faculty to use innovative approaches with the assistance of technological devices and psychometrics to help family nurse practitioner students develop competent physical assessment skills while further developing the nurse practitioners’ knowledge level of advanced health assessment and medical decision-making skills.”
About Quantum
Quantum is a tablet-based system that quantitatively measures performance in simulation using an instructor-mediated assessment, to measure a participant’s integrated knowledge, skills, and attitudes, and then provides detailed score reporting and customized remediation! Quantum can evaluate an instructors level of severity. Quantum features an exceptional administrative scheduling, mass communication tools, and robust reporting.
Sponsored Content:
Quantum For Academia
Measure student performance and validate competency with Quantum, the first and only instructor-mediated, performance assessment that has the capability to estimate and adjust for instructor and item variations to produce fair and objective student test scores. Quantum is comprised of several standardized clinical scenarios for use in the simulation laboratory that each student must successfully complete in order to demonstrate competence in key areas. Quantum incorporates educational learning and critical thinking theories, and measures all domains of learning. Psychometrically sound, customizable, legally defensible, touch-screen Tablet technology. Includes instructor and student scheduling, communicating, and reporting.
Quantum For Hospitals
For hospitals, Quantum operates in the HR vertical. Quantum, an instructor-mediated, performance assessment used in a simulation laboratory to assist instructors to objectively measure a learner’s performance and document his or her clinical competency. The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) noted that healthcare organizations have historically relied on education and experience to support competence, but an increasing number of healthcare organizations are seeking objective measures of a nurse’s knowledge that is required for safe practice. Whether it is to drive an educational intervention to improve safe practice or safely decrease the length of orientation for new employees using simulation for teaching and learning reflects a presumption that learning is taking place and the teaching is effective. How do you know if you don’t measure it objectively?
Sponsored Content:
What can Quantum be applied to?
If a performance can be observed against a known standard, Quantum can measure it! These are just a few of the many purposes Quantum serves:
- Hospitals: Operates in the HR and Risk Management vertical to validate clinical competency
- Education: Measures student performance in school sim labs
- Sports: Assists coaches with measuring player/athlete intelligence of recruits or current players
- Officiating: Assists sports league administrators to improve consistency in referees
Quantum’s Core Workflow
Pre-Assessment
- Administration and instructors create login accounts
- Lead instructor and simulation lab manager provide basic information to generate and approve test administration schedule
- Lead instructor drafts and transmits notification to all participants to create their account
- Participants create their accounts and select test administration date based upon the test administration schedule
- Participants obtains test administration receipt via confirmation text or email
Assessment
- Participant verifies their identity with a picture, signature, and voice pattern
- Participant prepares for performance assessment using an in-app summary
- Instructors assess the learner’s competence
- Participant completes the attitudinal survey
Post-Assessment
- Instructors review the Quantum Admin Report
- Learner reviews Quantum Report
- Participant reviews any pertinent remediation
How Does Quantum Provide Valid & Reliable Assessments?
Quantum’s validity evidence spans eight facets of construct validity: content, substantive, structural, generalizability, external, consequential, responsiveness and interpretability (Messick, 1989, 1995; Medical Outcomes Trust, 1995). Regarding reliability and standard error, Quantum provides a direct estimate of the modeled error variance for each estimate of a learner’s ability, an item’s difficulty, and an instructor’s severity (Wright & Masters, 1982; Wright & Stone, 1979). These standard errors provide a quantification of the precision of individual person measure, item difficulty, and instructor severity, and can be used to describe the range (i.e., confidence intervals) within which each learner’s true ability, item’s true difficulty or instructor’s true severity falls. Reference: Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd ed. pp.13-103), New York: American Council on Education and Macmillan. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. Medical Outcomes Trust Scientific Advisory Committee. (1995). Instrument Review Criteria. Medical Outcomes Trust Bulletin, 1-4. Wright, B.D. & Masters, G.N. (1982). Rating scale analysis. Chicago: MESA Press. Wright, B.D. & Stone, M.H. (1979). Best test design. Chicago: MESA Press.
For example, given a three dimensional matrix of data represented by: learners X items X instructors, the difficulty of item i is estimated based on the sum of all ratings on that item across all learners and all instructors. Likewise, the severity of instructors j is estimated based on the sum of all ratings comprising all items across all learners scored by that instructor. The person measure positions learners relative to each other after the additive effects of items and instructors specified in the model have been subtracted from the calculation. The resulting person measures are linear and also objective because they have been freed from the differences due to variations in items and instructors (Linacre, 1994). Reference: Linacre, J.M. (1994). Many-faceted Rasch measurement (2nd ed.). Chicago: MESA Press.
Learn more about the reliable Quantum performance assessment tracking system on the ObjectivityPlus Website!
Lance Baily, BA, EMT-B, is the Founder & CEO of HealthySimulation.com, which he started while serving as the Director of the Nevada System of Higher Education’s Clinical Simulation Center of Las Vegas back in 2010. Lance is also the Founder and acting Advisor to the Board of SimGHOSTS.org, the world’s only non-profit organization dedicated to supporting professionals operating healthcare simulation technologies. His co-edited Book: “Comprehensive Healthcare Simulation: Operations, Technology, and Innovative Practice” is cited as a key source for professional certification in the industry. Lance’s background also includes serving as a Simulation Technology Specialist for the LA Community College District, EMS fire fighting, Hollywood movie production, rescue diving, and global travel. He and his wife Abigail Baily, PhD live in Las Vegas, Nevada with their two amazing daughters.
Sponsored Content:
