The e-Assesment Association Company Logo
the e-Assessment Association

New blog: Running trials in a pandemic

New blog: Running trials in a pandemic

By Andy Lewis, AlphaPlus

It has been almost a year since schools first shut to limit the spread of coronavirus. As well as all of the obvious disruptions, Covid-19 has had a massive impact on our trialling work, delaying or otherwise changing many of the trialling projects we had in the pipeline. As part of our work developing assessments, we undertake a lot of trialling – learners trying out assessments so we can calibrate the assessment content and obtain feedback from learners and teachers about how the tests work. Over the last three years, we’ve delivered 25 major trials (and many smaller ones) in the UK and overseas with over 135,000 learners, and 11 residential marking operations with over 250,000 scripts marked, with more than 75 test administrators and markers.

Despite the ongoing school closures and major disruption, we successfully delivered four major onscreen trials for over 20,000 students in schools during autumn 2020. All were non-statutory so required voluntary participation. Despite obvious challenges, we were able to recruit the required numbers and deliver the trials successfully. Here are some of things we have learnt over the past few months.

Schools are suffering from an information overload

One MAT we know well has 5 staff working full time to read, review and interpret the guidance and directives flowing out to schools. It is easy for an invitation to participate in a trial to be missed in the deluge of correspondence! 

We found a single-page infographic ‘Quick-start’ guide, with all the relevant details, popular with teachers needing to quickly understand all the features of a trial.

Schools are still keen to take part in trials

This is the most important factor, as it was one of the greatest unknowns before we resumed our trialling service. Initially we were concerned that schools would – understandably – be too busy to participate in trials after months of pupil absence.

However, we have instead found that schools want to learn how their students are doing after such a long time away from lessons, and many want to take advantage of free tests with feedback on how their pupils are performing.

This isn’t just limited to our trialling projects. In Wales, almost 20,000 national adaptive personalised assessments were scheduled in the first four weeks of the autumn 2020 term alone, as schools wanted to take advantage of the formative benefits of the assessments.

As always, and quite reasonably, schools require that the trials are as low hassle for them as they can be. If there’s technology involved (onscreen tests, for example) then it has to work, the test content has to be properly organised and the operational arrangements must be clear. Quick and helpful support over the phone has to be available on tap, too.

Schools may be forced to drop out

With whole classes needing to isolate if exposed to the virus, schools are unable to guarantee that they will be able to take part in trials. This is completely understandable, and our role involves doing everything we can to support schools to make taking part easier. This includes, for example, extending the length of the trialling window to increase the likelihood of schools participating, as well as overrecruiting to mitigate the impact if a school is forced to drop out.

Obviously if a national lockdown is called at short notice (as was the case in January 2021), all trials must be halted until schools can reopen, and schools may not be able to commit to the delayed timeline.

Schools have concerns

As well as their usual concerns (how much time the trial will take, whether it will be too large a distraction from teaching, what support they will receive, what feedback they will get from the assessments, etc.), schools want to know how COVID factors will impact the trial. We can help with these concerns by providing additional advice and support.

We can’t send our test administrators into schools

In the past, we used to often send a trained test administrator into schools to ensure that everything ran smoothly – this reduced the administrative burden on the school and also meant we could ensure the trial was delivered precisely according to the protocol. However, due to the pandemic, we were reluctant to send in test administrators to schools, and many schools are currently operating a ‘no-visitors’ policy for non-essential purposes. This has meant that we needed a new strategy for trials that would have previously been administered by one of our trained administrators, which has resulted in more trials being ‘self-administered’ by schools online.

Issues specific to delivering online assessments in schools:  

Trialling Image

Trials need more project management and more time spent speaking to schools

In order to run onscreen trials successfully, we need to provide clear and comprehensive guidance materials (user guides, videos) so that school staff are able to set up and run assessments via e-assessment platforms. 

For each trial we provided a designated helpdesk, with support available from the AlphaPlus trials team via email and telephone. Each of the four trials in November 2020 used a different e-assessment platform, so our team had to be conversant with a range of systems in order to provide support to school staff.

School restrictions and constraints

Onscreen trials are typically delivered in school IT suites, but due to COVID ‘corridors’ this has restricted the movement of pupils and staff within schools. It has resulted in several issues: 

  • Fewer classes have access to the IT suite to run trials, so it is more difficult to schedule sessions during the trialling window. This often limits the number of pupils per school that can take part.
  • Access to materials is now limited due to the requirement to clean everything between users, so for example calculators, pens/pencils and paper that are essential when taking many Maths tests (and wouldn’t be an issue in ‘normal’ times) are suddenly unavailable.
  • Staffing issues: the high level of staff absence due to COVID has meant that the member of staff running the trial on a given day often has no prior awareness of the trial, how to interact with the e-assessment platform and support pupils, all of which makes the helpdesk function extremely important. 
  • Schools are under incredible pressure and unprecedented constraints. It is vital to appreciate this and always be supportive and patient when interacting with teachers and support staff.

Despite all these challenges, our trialling projects were completed successfully

Our trialling team has responded incredibly well to the new challenges, and successfully recruited and ran four non-statutory trials with a combined total of over 20,000 learners. This is all down to the flexibility and decency of the schools we work with and to our in-house team, who are knowledgeable and committed to making trials a success, not just for our clients but also for the school communities we work with.

Finally, we would like to thank all schools that participated in the autumn trials, despite the many challenges they faced. 

Find out more about AlphaPlus’ trialling capabilities here.

News categories

Upcoming Events

Quicklinks