Essay-based Exams Online – Transition, not Transformation

In line with widespread initiatives to enhance the student experience of assessment and feedback in higher education, many universities have been exploring the potential of technology to support more authentic and engaging teaching and learning processes. There is much said and written about the potential for technology to transform existing systems of assessment but in some cases smaller, transitory or complimentary interventions involving technology can have a notable impact on student learning, their engagement and/or experience, and can address important changes in the way that students learn.

In 2011 the University of Dundee responded to a growing demand from its Schools for a resilient solution which allowed the secure presentation of essay-style examinations online and included standard word processing functionality. This request from Schools was in turn a response to feedback from students who challenged the fairness of being asked to handwrite for two or more hours when they were used to typing. Guided by the excellent work by Mogey et al. (2007) at the University of Edinburgh, the University of Dundee’s Library & Learning Centre (LLC) explored options to facilitate the online delivery, recording and marking of short answer/essay based examinations and ultimately opted to trial the ExamOnline assessment system developed by Intelligent Assessment Technologies (IAT – http://www.intelligentassessment.com/).

In contrast to Edinburgh, a desktop solution was favoured and it was a requirement that the system enabled both anonymous on-screen marking and the option to distribute submissions to PDF for marking offline. Further, it was a requirement that the chosen system saved candidate answers to both the server and local hard disk to ensure redundancy in the event of either network or PC failure. At the start of academic year 2011/12 a production licence was purchased and a carefully managed rollout implemented by the LLC initially involving two Schools – School of Computing and CEPMLP. This was subsequently extended in 2012/13 to include the School of Life Sciences (Learning & Teaching) and School of History.

The ExamOnline system has now been in production use for two full academic years. Over 4000 candidates have been examined to date with concurrent sittings of 160 students now routine. To accommodate the rapid growth in uptake of the system the University has extended its licence to accommodate projected future demand. Feedback from both staff and students has been positive with colleagues highlighting that the system offers students the opportunity to present their best answer rather than simply their first answer. The issue of illegible handwriting is overcome, and many staff members have commented that this has both accelerated the marking process and enabled them to award marks where previously they may not have been able to do so due to their inability to decipher written answers. Some students new to this method of examination were initially wary of the new format with some feedback received during the first year of use critical of the noise created by the number of students typing and concerns about typing speeds. As we now enter our third year of using the system, anecdotal feedback from staff indicates that students have quickly adapted to the new approach and negative feedback is now rare. An evaluation of both staff and student experiences of the system is planned by the end of academic year 2013/14.

While the approach described is neither particularly innovative or transformative (for examples of Innovative and Non-Objective Items in e-Assessment see this post by e-Assessment Association Board Member John Winkleywww.e-assessment.com/blog/innovative-and-non-objective-items-e-assessment-january-2013), the initiative demonstrates how technology can facilitate progressive enhancements to the assessment process with benefits for both staff and students. The question of whether unseen essay examinations are an appropriate method of assessment is for another post. Technology offers significant opportunities for us to reimagine the dominant modes of assessing and providing feedback in higher education but in an area where the stakes are so high transition rather than transformation may be necessary.

Author: Dr David Walker, University of Dundee

References

Mogey, N, Sarab, G, Haywood, J, Van Heyningen, S., Dewhurst, D, Hounsell, D and Neilson, R. (2007) The end of handwriting? Using computers in traditional essay examinations. Journal of Computer Assisted Learning. “4(1), 39-46.