Most Innovative Use of Technology in Assessment Award 2022

Sponsored by Vretta

Project Summary

"ACER facilitates standard setting in numerous assessment contexts. The pandemic prompted us to develop an online secure standard setting platform.

This raised new data security challenges. We needed a process for collecting judgments that was robust, but simple and straightforward for users. The solution needed to handle multifarious assessment formats and diverse contexts.

Adopting the ‘double-diamond’ product design process, we developed an online application named Signum. This product design process minimised the time from product ideation to delivery. Multiple trials provided valuable feedback that was translated into new features.

Signum is now being used for several Angoff variations, the Bookmark method, and Pairwise comparison studies. A continuous integration and deployment pipeline, utilising latest web-stack technologies, ensures a quality user experience.

Clients have been highly receptive to the software, and demonstrations have led to the use of Signum for other data collection and rating activities. Reliability is ensured through automated end-to-end testing. New features can be deployed within hours. The platform has passed a robust penetration test and handles complex content and diverse item formats through screenshots. The design and implementation of Signum demonstrates an agile approach to developing fit-for-purpose software for flexible use – a new contribution to the field."