On-screen marking

I have often wondered why there are still paper-based tests delivered in an era when pen and paper has almost been completely eradicated from our day-to-day lives.

Even though tests are delivered on paper, it is still possible to digitise the back end processes if you like of essay-based and short response/open questions by scanning scripts and then marking onscreen.  As long ago as 2000, organisations such as UCLES were investigating such processes and the report they produced of their findings is worth a read:

http://www.cambridgeassessment.org.uk/ca/digitalAssets/113896_On_Screen_Marking.pdf

The benefits of digitising the marking process are many, but I have listed a few of the main ones below:

Security: there is no doubt that shipping scripts out to examiners and then back from the examiners to the central office is not as secure as having this information centrally stored with proper controls over access. How secure are the papers once they are in the hands of the examiner? There is no way to track what happens to these papers once delivered, where they may be left and once lost/destroyed they are never to be found. No digital back up with paper.

Cost: as petrol prices continue to soar (and anyone who fills their car up at the pump knows that there is only one way petrol prices move), it will become increasingly expensive to ship the papers to and fro. There is also the added cost of storing all that paper for the necessary period (no smoking in the warehouse please!).

Capacity: if you are shipping papers around to markers, it means that your pool of markers is likely to be limited by their geographical location. Once test responses are digitised, this opens up any geographical location with access to the internet in terms of your markers pool, greatly increasing your capacity to correct exams.

Efficiency: real time monitoring of script/response marking means that bottlenecks can be identified quickly and corrective action taken. No nasty surprises at the last minute.

Quality: standardisation of human marking is one of the most important aspects of ensuring that tests are fair, reliable, valid and legally defensible. Real time monitoring of marker performance allows for timely interventions. The automatisation of some of the repetitive, routine tasks that humans are not very good at getting right consistently is also another bonus. When you rely on humans to do these tasks, you can count on human error at some point as recent experience shows (http://www.bbc.co.uk/news/education-18785044).

That is not an exhaustive list of the benefits, but it is a pretty impressive one. Surely there are sufficient drivers there to ensure that nearly everyone is either doing on-screen marking or moving to it pretty sharpish. The reality on the ground is that this process is incredibly slow. Why is that? I have listed some of the arguments I have come across over the last few years (I am sure you could add to them).

• On-screen marking may not be as reliable as paper-based marking: this discussion could keep academics busy for decades, however there is an interesting piece of research published by Cambridge Assessment, which is worth a read (http://www.iaea2008.cambridgeassessment.org.uk/ca/digitalAssets/180506_Shaw___Imam.pdf)
• Our markers refuse to mark on-screen: I heard this remark recently when visiting an academic institution. Was this the case when offices were becoming digitised? Did people refuse to send emails preferring to send snail mail instead? Will we really have to wait for the old guard to retire and the younger generations to come through and refuse to mark paper-based scripts? If so it will be a long wait as retirement ages increase and youngsters struggle to get into the labour market.
• Suspicion and fear of new technology: it is true that nobody ever got sacked for asking people to mark paper-based scripts/responses. It is the default technology and has been since well before I was born. A shift to on-screen marking is seen as ‘a big thing’ and so people prefer to stick the same-old, same-old.

It would be interesting to hear your views on the relative merits of on-screen marking and paper-based marking, whether you are a practitioner of either or both. I have no doubt whatsoever that in the next decade or so, paper-based scripts will be the minority of tests delivered and on-screen marking the default medium will come sooner (as paper-based scripts will be digitised). Do you agree?

Author: Matthew White, International Baccalaureate Organisation