New blog: Remote invigilation / Remote proctoring

Patrick Coates, Director of International Skills UK and Board Member of the eAA writes about remote invigilation / remote proctoring and invites you to get in touch if you are involved in using or trialling remote proctoring, particularly with a view to share the challenges, questions and answers across the e-assessment community.

Remote invigilation or proctoring₁ has been around for a while and I have seen evidence of its use in the mainstream for exams in the professional and academic world in the UK, US and Australia for a number of years. However, it has not taken off as quickly as would have been expected or hoped. It is fair to say that there have been a number of technology innovations over the years in e-assessment, from adaptive testing to on-demand testing or machine marking of free text responses, which have not quite made it in the mainstream as yet. Remote invigilation is one of these.  Why is that?

On the one hand, there could be perceived or actual deficiencies in using the technology for a particular assessment, or as is more likely, a lack of understanding of what can be achieved or a general willingness to commit to something new. Having seen it in action, I think the big challenge with remote invigilation is a lack of understanding of what is involved and perceived issues when compared to the traditional test centre model. It is this lack of understanding which leads to a slow uptake of the ‘new’ technology.

 https://en.wikipedia.org/wiki/Technology_adoption_life_cycle

Whilst this is not an academic piece, if we look at the technology adoption model (with the caveat that the early part of the bell curve should be more of an S-shape in my opinion, but is too in depth for discussion here) the real challenge is that on the X-axis there isn’t any indication of time.

So we don’t know whether the early majority are going to be using the technology in a matter of a few years, or decades.

A key thing that will accelerate that adoption is communication. If we can help explain what remote invigilation is, what the opportunity is, then this will hopefully expedite adoption. Hence this blog and if there’s interest perhaps a more in depth piece of research.

So probably, best to start from basics. What is remote invigilation/proctoring?

The truth is it can take on a number of forms. Just as there is still confusion over what is an online test versus and on-screen test, there are different ways in which technology can be involved in the process and the detail is absolutely key. For simplicity, I have categorised the different sorts of remote invigilation under three headings, really simply to show the variety of ways in which this can work. In reality each test sponsor will look for a slightly different blend of solutions.

Online Invigilation

This is where an invigilator is online looking at the candidate and/or their screen in real time, with them checking the process, listening and watching for any untoward activity. The invigilators are not necessarily looking at one candidate but could be looking at a number of candidates on a screen. It is likely that the number of screens being monitored at any one time impacts on the price of the service. The candidates could be based at home, in a formal test centre or any 3rd party venue. The only requirement from a venue perhaps being the meet and greet element. Then again, maybe we should be thinking of the passport style testing booth in a station or post office.

Offline Invigilation

In this instance, the test session is recorded, perhaps where there is intermittent or low bandwidth, or perhaps where all the tests have to take place on the same day and it is not feasible to have them all monitored simultaneously. These session recordings are then transferred to a location where they can be checked after the test has taken place. The results of the test result would only be provisional until they had been checked.

Hybrid model

Advances in technology will see an increase in hybrid models where computer algorithms flag up any untoward activity or issues, only referring to a human grader where need be. Whether running this technology in parallel with live invigilators or by recording all sessions, we will see increasing reach of technology in this sector.

I would certainly argue that any of these methods are in fact more secure than a traditional test centre model as there isn’t an opportunity for local fraud based on a candidate’s relationship with the test administrator – or statistically unlikely anyway.

Biometrics and identification

Whilst the delivery is theoretically straight forward the other key issue with remote invigilation is being as sure as you are in a traditional test centre model, that the candidate is who they say they are.

The easiest way is holding a photo ID up to a screen to have it checked against your image whilst capturing both images for future comparison. Plus, potentially embedding the candidate picture on any results. Of course now there are more sophisticated and rigorous methods with the use of biometrics with passport scanning equipment and the potential new means of retinal scan, face recognition or fingerprinting all of which can be automated and would work certainly in a test venue of some form, but in the future could be used at home. It is just a question of time.

Test Sponsors

So I have said that communication is key to adoption and the gatekeeper to adoption here is the test sponsors – those who actually own the test, exam or assessment, rather than the software companies developing the technology. They are the ones that need to be convinced.

Simply put, test sponsors need to become more informed. What is now possible? What point have we reached with the technology? And furthermore, how does this technology enable us to make a step change to how we test, rather than looking to replicate existing systems simply because we can. Why copy the inadequacies of the previous system when you can do things in different ways?

The problem with the classic test centre model

Remote invigilation opens a door to on-demand testing and more personalised assessment. I would urge current test sponsors to look at the costs, but also the flaws and the advantages of current systems and procedures, and to start by asking what would they like to improve, it needs to be viewed from a strategic perspective.  I am confident that there is technology out there to make the changes you seek.

1 “Proctor” and “Invigilate” are interchangeable words with identical meanings in both the Cambridge Dictionary and the Merriam Webster Dictionary. However, it is worth recognising that the former term is used extensively in the US, whereas the latter is the term more commonly used in the UK.