Bringing assessment into the digital age

eAA Article published in QA Education

By Patrick Coates, director at International Skills UK and board member of The e-Assessment Association

e-Assessment is increasingly the norm in the professional world, with e-Testing having been used for IT exams for over 30 years.

Although this was typically in objective questions format, driven by the ‘license to practice’ market in the USA where legal defensibility of exam results was and is still required, the system has become more sophisticated over the years; including simulation in the IT world or the use of video in the medical sectors.

Inevitably, the UK has followed suit, with a broad range of e-assessment being introduced increasingly to improve the speed and efficiency of the examination process. e-Portfolios have been used widely in the vocational qualification market for the last 15 years and e-marking has been used in the last 10 years for education exams with all the unitary awarding bodies having adopted e-marking; a process where scripts are scanned in and then human-graded using simple work-flow or complex tools to manage the marking and quality checking of exam scripts.

As a judge in the inaugural e-Assessment Awards, I had the opportunity to review and acknowledge those that had delivered excellence in e-assessment. A wide range of industries were represented, including: degree and masters-level business qualifications; IT qualifications in India, Oil and Gas Safety; several Accountancy exams; Corporate Treasurers, Journalism as well as Midwifery and Reproductive Health.

While e-assessment has been widely used for a long time in a whole range of professional areas, if we look at mainstream education specifically, then could it radically transform the sector? The increasing use of technology will certainly be seen in some areas, but how will e-assessment change the entire market?

Let’s start with MOOCs (Massive Open Online Courses) for their global impact. When MOOCs were first introduced, they didn’t have the expected revolutionary effect in education. For me, the challenge was that MOOCs were not that ground-breaking (we used to call lots of freely-accessible content in a single location a library) it was giving away your intellectual property that was revolutionary. If an academic institution is giving away their content for free, how do they make money? Well one way is in the use of assessment, but this poses a number of questions. How do you deliver assessment anywhere in the world? How do you deliver it securely? And how do you deliver it cost-effectively?

• Remote Invigilation – one of the challenges of not having a physical location where you study is that there isn’t anywhere to deliver an exam. High-stakes examinations are essential to the value of any qualification, and these must be moderated.  By having remote cameras using biometrics to ensure the candidate is who they say they are and they are not cheating solves that problem. There are several providers doing this now, but on a smaller scale, as exam boards are still reluctant to allow candidates to participate remotely.  Remote invigilation for e-assessment also means that institutions do not have to provide 1000s of PCs to attending candidates, as they can deliver the assessment anywhere.
• Adaptive Testing – This is the concept of asking the candidate a question, the response to which is subject to various statistical and psychometric analytics to determine the next question. This means the test is much shorter as you are only asking the candidate the questions you need to ask them to determine their grade and you also reduce their exposure to questions (or items). The challenge is that you need a large number of questions to achieve this (a difficult and expensive task) and there are still risks of candidates remembering items and publishing them online.
• Auto-marking – This is widely used for exams for UK awarding bodies and is increasingly being used around the world. This doesn’t fundamentally change the way that the exams are delivered, but it improves the efficiency, quality and accuracy of how they are marked.  The research I have seen suggests that there isn’t a difference between human and computer marking, but there are two main challenges. Firstly, the script needs to be in a digital format which either means you need lots of PCs for candidates, or you get the writing recognition software to convert written text accurately into digital text that can then be marked. Humans would be required to teach the computer what “good” looks like, and there would still be the need for the best and worst scripts (think of a bell-curve) to be human-graded. However, you could take out a huge chunk of the cost of marking scripts and potentially remove the problem of finding markers.

There are other technologies in development that could really change the way that assessment is undertaken, if we are able to move away from our current limitation to how we conduct exams and assessment, but that’s for another day.