Colluding with technology to tackle exam malpractice in assessment?

A blog by Gwyneth Toolan, Product Manager, RM

Moving more assessments online has meant that some traditional security measures are no longer as effective as they could be. The technology that enables us to continue working and learning remotely, now adds a new aspect to the widespread challenge of exam malpractice within the assessment community.

At RM, we are committed to maintaining secure tests while also meeting candidate expectations, and this year we have been breaking new ground in the battle against one of the most prevalent and fast-growing forms of exam malpractice – collusion.

By working with clients, we have been able to create a scalable tool that flags instances of collusion and other forms of malpractice, providing awarding bodies with a newfound confidence in their exam security. This enables more informed decisions about the delivery of exams and the way to protect the integrity of assessments going forwards.

A collaborative approach

Working closely with our partners enables us to easily identify areas where we can improve their assessment offering. We always strive to understand the challenge before working on a solution, which requires open and thorough communication so that we can address the challenge in the best way going forwards.  By identifying the core problem in this much detail, we can sketch out potential solutions with their support. In this case, we concluded that an out-of-the-box solution would only solve part of the problem.

Remote proctoring goes some way towards protecting against collusion, but candidates have many ways of avoiding detection through this security measure. Through our trial work with customers, we have been able to better understand the methods that some candidates use to collude and using data science techniques, develop a concept that can detect them even when they try to disguise them.

What is next? 

We are still developing our solution, but we have already checked over 200,000 exam responses for collusion. We have found over 2,700 pairs of candidates whose responses were suspicious. 62% of those were confirmed as suspicious when investigated further.

We do not want to stop there though, so we are looking to help more awarding organisations check for collusion and make improvements to our collusion detection model. We are offering a completely free health check on one of your previously completed exam sessions.

Our process is simple: We start with a discovery meeting, through which the team will get a better understanding of your exam programme and identify where our collusion service could work. After that, if the model fits, you will just need to securely send us some exam responses (do not worry, we will set this up for you). Once we have the responses, we will check them for collusion and then send the responses back to you.

As you might imagine, who we work with, and how the tool works is entirely confidential. But in its simplest terms, the process is straightforward. We start by checking each exam response against every other exam response in the session. When we identify suspiciously similar pairs of responses, we flag them, those responses are then sent to you so you can investigate them.

(838) RM Exam Malpractice Service Preview – YouTube

The obvious benefit is that if there is collusion taking place in your exams, we can help you find it, and the potential scale of it. Not only that, but the data from our detection service can also provide you with confidence in your exam security, or the knowledge to act to protect it. It may also provide additional insights which can help you make informed decisions about your exam programme going forward.

We can also run our collusion detection service on live exam series. For a fee, we can detect collusion and provide you with all the information you need to investigate potential cases of collusion and take action against them promptly, all before results are issued.

Upon experiencing a vast improvement in identifying suspicious pairs of responses, we are looking to further ways the solution can be used in exam sessions. Positive and constructive feedback provided on the collusion service will only further its development and value. Exam malpractice is a predominant challenge in the assessment space, and one which we are committed to act on.

In this industry, we all know how important exam integrity is, so if you are interested in RM’s exam collusion service, or just want to know a bit more about it, get in touch with Gwyneth at [email protected]!