What is online proctoring? A blog post by member Nathan Thompson

Reproduced with kind permission

Online proctoring has been around for over a decade by now. But given the recent outbreak of COVID-19, educational and workforce/certification institutions are scrambling to change their operations, and a huge part of this is an incredible surge in online proctoring. This blog post is intended to provide an overview of the online proctoring industry for someone that is new to the topic or or is starting to shop and is just overwhelmed by all the options out there!

Online Proctoring: Two Distinct Markets

First, I would describe the online proctoring industry as actually falling into two distinct markets, so the first step is to determine which of these fits your organization

1. Large scale, lower cost (when large scale), lower security systems designed to be used only as a plugin to major LMS platforms like Blackboard or Canvas. These online proctoring systems are therefore designed for medium-stakes exams like an Intro to Psychology midterm at a university.
2. Lower scale, higher cost, higher security systems designed to be used with standalone assessment platforms. These are generally for higher-stakes exams like certification or workforce, or perhaps special use at universities like Admissions and Placement exams.

How to tell the difference? The first type will advertise about easy integration with systems like Blackboard or Canvas as a key feature. They will also often focus on AI review of videos, rather than using real humans. Another key consideration is to look at the existing client base, which is often advertised.

Other ways that Online Proctoring systems can differ

AI vs humans: Some systems rely purely on artificial intelligence algorithms to flag video recordings of examinees. Other systems utilize real humans.

Record & Review vs. Real-Time Humans: If live humans are used, there are two ways. First, it can be live and real-time, meaning that there is a human on the other end of the video that can confirm identity before allowing the test to start, and stop the test if there is obviously illicit activity. Record & Review will record the audio and a human will check it within 24-48 hours. This is more scalable, but you can’t stop the test if someone is stealing the content – you probably won’t know until the next day.

Screencapture: Some online proctoring providers have an option to record/stream the screen as well as the webcam. Some also provide the option to only do this (no webcam) for lower stakes exams.

Mobile phone as third camera: Some newer platforms provide the option to easily integrate the examinee’s mobile phone as a third camera, which effectively operates as a human proctor. Examinees will be instructed to use the video to show under the table, behind the monitor, etc., before starting the exam. They then might be instructed to stand up the phone 2 meters away with a clear view of the entire room while the test is being delivered.

Using your own proctors: Some online proctoring systems allow you to utilize your own staff as proctors, which is especially useful if the test is delivered in a small time window. If continuously delivered 24×7 all year, you probably want to use the vendor’s highly trained staff.

API integrations: Some systems require software developers to set up an API integration with your LMS or assessment platform. Others are more flexible, and you can just log in yourself, upload a list of examinees, and you are all set.

On-Demand vs. Scheduled: Some platforms involve the examinee scheduling a time slot. Others are purely on-demand, and the examinee can show up whenever they are ready. MonitorEDU is a prime example of this: examinees show up at any time, present their ID to a live human, and are then started on the test immediately – no downloads/installs, no system checks, no API integrations, nothing.

More security: A better test delivery system

A good testing delivery platform will also come with its own functionality to enhance test security: randomization, automated item generation, computerized adaptive testing, linear-on-the-fly testing, professional item banking, item response theory scoring, scaled scoring, psychometric analytics, equating, lockdown delivery, and more. In the context of online proctoring, perhaps the most salient is the lockdown delivery. In this case, the test will completely take over the examinee’s computer and they can’t use it for anything else until the test is done.

LMS systems rarely include any of this functionality, because they are not needed for a midterm exam of Intro to Psychology. However, most assessments in the world that have real stakes – university admissions, certifications, workforce hiring, etc. – depend heavily on such functionality. It’s not just out of habit or tradition, either. Such methods are considered essential by international standards including AERA/APA/NCMA, ITC, and NCCA.

About Nathan Thompson

Nathan Thompson earned his PhD in Psychometrics from the University of Minnesota, with a focus on computerized adaptive testing. His undergraduate degree was from Luther College with a triple major of Mathematics, Psychology, and Latin. He is primarily interested in the use of AI and software automation to augment and replace the work done by psychometricians, which has provided extensive experience in software design and programming. Dr. Thompson has published over 100 journal articles and conference presentations.

Nathan Thonpson is VP of Psychometrics at Assessment Systems Corporation

Photo by Andrew Teoh on Unsplash