Live Remote Proctoring Provides Comparable Outcomes to Traditional Test-Centre Proctoring Prometric-CARPE Study Finds 

Written by: Li-Ann Kuan (Ph.D. – VP, Assessment Services, Prometric) and Michael O’Leary (Professor – Director, CARPE/DCU)

When the onset of the COVID-19 pandemic forced the closure of test-centre networks around the globe in early 2020, programmes that rely on test-centre computer-based testing as their primary mode of administration quickly pivoted to live remote proctoring. With its accelerated implementation, the need to determine if live remote proctoring provides comparable outcomes to more traditional proctoring in test centres is more important than ever. Learn how Prometric, in collaboration with the Centre for Assessment Research, Policy, and Practice in Education (CARPE) at Dublin City University (DCU), used Prometric’s live remote assessment solution, ProProctor™ to explore this critical issue. 

The ability of test sponsors, working collaboratively with their test delivery partners, to provide timely, reliable access to their assessment programmes is imperative to the industry. When the onset of the COVID-19 pandemic forced the closure of test-centre networks around the globe in early 2020, programmes that rely on test-centre computer-based testing as their primary mode of administration quickly pivoted to live remote proctoring solutions. While the use of live remote proctoring has been growing for the last decade, the pandemic increased demand significantly and in a short period of time. This acceleration left many in the industry asking the question: are the outcomes from live remote proctoring and test-centre proctoring comparable?   

In December 2020, Prometric, in collaboration with the Centre for Assessment Research, Policy, and Practice in Education (CARPE) at Dublin City University (DCU), conducted a study that explored this important question by focusing on candidate outcomes and test psychometric properties across the modes. Using U.S. data from 16 different computer-based multiple-choice exams taken by over 15,000 candidates at either a test-centre or via Prometric’s ProProctor™ platform, the research team uncovered the following key findings:

1. Candidate outcomes across the two proctoring modes were almost identical.

When comparing the candidate outcomes, the average number of questions answered correctly by candidates who took their exams in-centre or via the ProProctor platform was nearly identical (69.97% and 70.00% respectively). Similarly, when averaged across all 16 tests, the difference in passing rates between those who choose to take their exam in test-centres (54.44%) and those who took the exam via the ProProctor platform (56.14%) was not significant.

2. The psychometric properties of tests proctored in test centres and via the ProProctor platform were equally satisfactory. 

The average internal consistency reliability measure for test-centre exams was 0.89 and 0.90 for the ProProctor platform, while the average decision consistency measure (Subkoviak’s Coefficient of Agreement) across both modes was 0.84.

3. Candidates taking their exam on the ProProctor platform took less time, on average, to complete testing.

When averaged across exams, ProProctor candidates took about four minutes less time to complete exams than their test-centre counterparts. While this finding needs further exploration, it may provide some comfort to those who worry that some remotely proctored candidates are spending time engaged in unethical behaviors, such as consulting sources for answers to test questions. 

While still in the early stages of research, these initial findings suggest no reason why live remote proctoring should not be used for high-stakes certification and licensure exam delivery.  The sense is that live remote proctoring solutions utilizing the same delivery platform as in-centre testing is secure, and will provide greater candidate access in the future. Live remote proctoring is here to stay and is shaping up to be an exciting next step for the testing industry. 

Learn more here about live remote proctoring

About Prometric

As a global leader in test development, testing delivery, and candidate services, Prometric partners with the world’s top credentialing and licensure organisations to design and deliver leading exam programs that help individuals advance their careers and serve their communities. Our integrated, end-to-end solutions provide exam development, management, and distribution that set the industry standard in quality, security, and service excellence. Prometric leverages our proprietary platform, advanced technologies, and vast operational experience to deliver an exceptional user experience on our world-class secure testing network. Today, we are paving the industry’s path forward with new solutions and innovation to ensure reliable access to secure assessments anytime, anywhere. For more information, visit prometric.com or follow us on Twitter at @PrometricGlobal and Linkedin.