Going remote but staying close: keeping test takers at the heart of the British Council’s remotely proctored Aptis test.

Going remote but staying close: keeping test takers at the heart of the British Council’s remotely proctored Aptis test.

Although technology undoubtedly enables testing solutions to reach more people in more places, the British Council noted some of the assumptions implicit in its remotely proctored Aptis test processes, and adapted the customer journey to address these following initial sessions with test takers in Mexico. 

While Aptis is normally delivered by computer in a physical, invigilated environment, the remotely proctored delivery model of the test was rolled out following the outbreak of Covid-19. This all sounds well and good, but we were interested in how our test takers felt about the experience, and why in our earlier sessions there were higher rates of no-shows on test day than in standard sessions.

Testing at the push of a button? Not quite! 

The most challenging part of the remotely proctored process for Aptis, and one which appeared to impact test takers’ overall experience related to onboarding instructions, adherence to which is necessary for a smooth test day experience. In the build-up to remotely delivered testing sessions, there was often an expectation that this high-stakes, professionally proctored test would somehow happen ‘at the push of a button’ and that all candidates had to do was log into their computer on test day, push a button and start responding to the tasks.  The essential but downplayed step was onboarding, yet it was this preliminary step in the process that seemed to affect candidates’ affective filters and stress levels the most. This was partly due to the detached nature of the system-generated communications in English, which took for granted that recipients had a certain proficiency in English and were also tech-minded enough to be sure that their home set-up and equipment met the technical requirements described. 

In early sessions, up to 25% of test takers who had enrolled failed to show up on test day. We were intrigued to understand why and to adjust the customer journey to bring in aspects of the human support experienced in the in-centre testing process.

Closing the emotional gap between in-centre and at-home testing

We worked with our proctoring software provider to tailor their system-generated communications, and we also developed our own communications pack. As a priority, onboarding instructions were translated into Spanish and were modified to reflect the British Council voice as much as possible. A pre-onboarding test-taker training session was developed to bring back some of the personalised contact normally associated with test registration, and to increase candidate confidence at providing assurances about their technical set-up, a task traditionally carried out by a test day supervisor. The local team also set up a closed helpline function on test day for any candidates experiencing problems. Essentially this closed the gap with the F2F test day journey where test-day staff are on hand to help if there are technical glitches or simply to ensure an atmosphere of reassurance in the exam room until all tests are submitted.

As our experience increased, not only did our rate of no-shows reduce significantly, but other feedback from the test takers became increasingly positive with over 70% of candidates now assigning the top rating of ‘very positive’ to describe their remote testing experience, almost 90% now rating their perception of ‘friendliness’ of the experience as between 8 and 10 on a 1-10 scale, and over 80% describing their sense of feeling appropriately attended to within the same range. However, although no longer an obstacle, nearly 20% of candidates still describe the onboarding process as difficult, and we continue working to improve this part of the process.

Different challenges, same objective: giving test takers their best chance

Perhaps one element of the test-room testing process can never be replicated. This is the social aspect, the test day group dynamic, the sense of nerves and excitement transmitting through the cohort in their shared experience of going through what is usually a high-stakes process for each individual. Sometimes these are classmates and sometimes they are strangers but in the here-and-now of the physical test room there is often a sense of togetherness and equity in the experience. As Professor Barry O’Sullivan, assessment expert and designer of Aptis, commented in a recent conversation, the live environment may also positively impact on some candidates by raising their adrenalin prior to test performance, making them feel more ‘up’ for the test. The more solitary experience in the home environment will offer other advantages to some people, such as avoiding a stressful or costly journey into a test centre, or taking advantage of one’s own quiet home environment to perform at one’s best, where circumstances permit.  However, the fact remains that whereas in-centre testing serves to reduce differences affecting candidates’ test-day experience, remote testing implies a greater number of factors at play.

Our experience reminds us that while the technology behind remote proctoring is as important as it is exciting, the end -users we are serving are even more important and will not always act with machine-like predictability. The human touch becomes more important than ever when there are additional variables in the mix.  For such a high stakes experience as a test, and for candidates to have their best chance, we have to listen to them, their needs and their preferences, whatever the delivery model. Perhaps the most important factor in our story is that remotely proctored testing became the only option – as opposed to the chosen option – overnight. Whereas the test itself remained the same, the process and stressors associated with testing changed. When it comes to remote proctoring, it is helpful to examine and challenge assumptions about the easiness of remote solutions from the test taker perspective, and from both technical and emotional angles. 

The British Council hopes to continue working with test takers worldwide to ensure that the huge opportunities afforded by e-assessment methodologies reach our customers in the fairest way possible to support their ongoing personal, academic and professional ambitions.

Written by Fiona Mitchell and David Sorrentino