E-assessment is getting easier and more diverse

I spoke recently at a conference organised by Rod Gordon of Gordon Associates (http://www.gordonassociates.co.uk/Default.aspx).  Essentially, the substance of what I was saying was that “e-assessment used to be expensive and technically challenging to implement, and there was not a great deal of diversity amongst the solutions on offer, but things are now getting easier, and more diverse”.  I thought I’d record these observations in a short blog post.

People in the UK and overseas have been predicting for over 10 years that technology will revolutionise assessment.  I still think it will, but the pace of change has been quite a lot slower than many of us one-time evangelists expected.  Assessment approaches (for example, once a year, everyone sits down to the exam at the same time) and assessment instruments (for example, longer essay style questions, written responses including diagrams, etc.) have stubbornly refused to bend to the opportunities that e-assessment offered.  Quite right too, in many cases, as technology should serve assessment rather than vice versa.  But the education system has stubbornly resisted much smaller changes too even where these would have had little real impact on the assessment other than to enable better use of technology.

Anyway, the point of all this is that I was recently set to looking at a range of more ingenious assessment solutions involving technology and I wanted to mention a few of them here:

Existing systems are getting less costly and difficult

The first thing to say is that mainstream e-assessment services, such as those provided by Calibrand, BTL, RM, Question Mark, etc., are not only getting less expensive to deploy, but the amount of skill and effort required to deploy them by practitioners and IT specialists  is much decreased.  At AlphaPlus (not an AO, by the way) we commissioned and deployed a live testing system in just a few weeks starting from scratch with new users trained from scratch, including bespoke item creation, and it worked largely without hassle in a variety of settings.  I can say with absolute confidence that this would not have been the case only a few years ago.

Both the existing suppliers and new entrants to the market are looking to the capabilities of cloud computing (typically multi-tenant systems) to greatly simplify the roll-out of e-assessment to new customers.  Using these new systems AOs can concentrate on qualification and assessment quality rather than having to be IT experts, although multi-tenant systems do present some operational challenges for AOs in the short term.

A simple solution to secure essay-based exam e-testing

Most of the current e-assessment systems cope poorly with supporting long essay questions (say 1500 words plus) – a type of response that is still common in higher level academic and professional examinations.  We recently helped an AO deploy a system designed specifically for essay writing questions.  Intriguingly, it required students to use their own laptop which was secured by the software to avoid other applications being accessed.  It worked very effectively (US Bar Examinations have been using the system for about a decade) and candidates liked being able to use their own machine for typing long essays.  Many preferred typing to writing due to being able to type faster, and make changes to their work as part of the editing process.

The software was also very inexpensive to setup and deploy.  The company that make the software (Exam4 – http://www.exam4.com/) describe it as an “armored word processor” and it fulfils that requirement (and no others!).  Incidentally, the markers of the scripts that the candidates produce greatly preferred typed scripts to avoid problems of legibility)

Alternatives to traditional assessment

Those of us working within traditional vocational and academic qualifications often regard exam-based assessment (objective, constructed response, etc.) as the “gold standard” in that it is secure and “sorts the academic sheep from the goats”.  I have no great concern about that, except to note that the reality of vocational and professional testing is that there are competing requirements, for example practicability, and making the results of the assessment contribute to other aspects of the business (for example, HR systems, organisation continuous improvement cycles, etc.).

Noting the enthusiasm in BIS (The Whitehead Report, etc.) for qualifications and assessment more focused on the needs of employers, I came across the following site: www.elephantsdontforget.com

This service provides an interesting alternative to compliance testing which could perhaps be used for vocational and professional certification, although the nature of the assessments look nothing like those that we who work in the regulated fields (i.e. Ofqual, WAG, CCEA, SQA etc) are familiar with.  I wonder whether future AOs might be encouraged to look beyond the traditional scope of vocational assessments to use technologies such as this which potentially meet more than one need for employers.

Technology-supported paper exams

[Declaration of Interest – this is an AlphaPlus product, so I’m biased.]  In working with AOs with lower candidate volumes and a distributed international community of test centres, it’s clear that it will take a long time for all to move to on-screen tests.  Meanwhile, the transportation costs of secure papers to the centres, and recovering the scripts securely from those centres contributes greatly to the cost of running the exam.  And things still go wrong – papers still have to be faxed on the morning of the exam.

AlphaPlus has produced a secure distribution system for papers, which allows centres to download a printable PDF of the exam paper (the questions to be answered) but with security features which discourage and make difficult the copying of the paper.  This relatively simple innovation has saved money on distribution costs, and improved examination processing.  There is more information about the SecurePDF product here: http://alphaplusconsultancy.co.uk/secure-introduction.  We’re going on now to investigate the options for low cost, low volume script scanning in order to complete the process of removing paper transfer to and from centres.

These are just four recent examples which hopefully demonstrate that e-assessment is diverse, and that there are new options opening up for AOs that have found traditional e-assessment inappropriate or unaffordable.

John Winkley, AlphaPlus

www.alphaplusconsultancy.co.uk