Online Proctoring Seminar: Your Questions Answered

In March 2023, we held an Online Proctoring seminar covering three areas:

• AI in Online Proctoring​
• Online Proctoring: What Happens Next?​
• Online Assessment in a Medical Context​

There were several questions asked during the seminar, so many in fact that we didn’t have time to answer them all. eAA Board member, Patrick Coates, kindly offered to review them and provide some answers and perspectives.

Question: “Up to what percentage we can trust AI, especially in terms of exams delivery? Do we solely rely on it or do we still require a manual monitoring?

There are solutions available in the market do include 100% AI but it is up to the exam owner as to what is an appropriate level of security for that exam. The key is understanding what the issues are if you get it wrong, i.e. you stop someone from completing an exam, but the AI got it wrong. Most of the solutions that are being used are AI-assisted, where the AI is flagging up for a person to review and make a decision.

Question: Any views on affect recognition – could this be introduced and misused?

There was a mention of biometrics (if I have understood this correctly) being introduced in the ‘Online Proctoring: What Happens Next?​ panel discussion, with the possible introduction of wearable tech. It is probably difficult to be able to use some data (exams are stressful anyway and the possibility for false positives may be high). However, anything that provides more data, in theory should be beneficial, if used together with range of data points.

 Question: Do you have any concerns that developing a good proctoring service may just become an ‘arms race’? Would a move away from the traditional ‘controlled conditioned exams’ towards a more ‘authentic assessment’ model impact the demand for proctoring in the future?

There are other assessments in place as we discussed in the Medical Setting’ panel session but these have challenges associated with them. No solution, even how we do things at the moment, is perfect. At the moment we are using technology to existing exam process, without changing it. There is work going on with Augmented Reality and simulations to provide alternatives, but these have challenges too.

Question: To what degree do you see human supervisors entirely trusting the AI rather than verifying the reports themselves?

I see it as the AI providing the data and it is up to the proctor to make the decision on how to interpret that data, based on their training. There is always evidence in terms of recording available and it is also a question of who is proctoring the proctors. I do know organisations do check that the correct decisions are made, not just as part of an appeals process, but also to check the integrity of their processes.

One audience member also replied to this question during the session: At my institution, we have an authentic assessment strategy but there are always some modules that lend themselves better to tests and exams

Question: In respect to security again, how do you see AI being used to prevent fraud/theft of content rather than candidate cheating to pass an exam?

The live proctoring provides this, as opposed to the record and review which works in an academic setting.

If you want to watch the online proctoring seminar again, or if you are one of the few who couldn’t make it, you can watch the event again here

Online Proctoring Webinar