2021 Winners & Finalists
Lifetime Contribution Award 2021
WINNER
Professor Geoff Masters AO, Executive Officer of the Australian Council for Educational Research (ACER)
Geoff Masters, AO, BSc, MEd UWA, PhD Chicago, FACE, FACEL, is Chief Executive Officer of the Australian Council for Educational Research (ACER) – a role he has held since 1998.
He has published widely in the fields of educational assessment and research. He is a globally renowned speaker on progressive assessment, including e-assessment. Under Professor Masters’ direction, ACER has grown from a national organisation based in Melbourne to one of the world’s leading educational research centres, specialising in assessment. ACER now has regional offices in five cities across Australia and five international offices.
As CEO, Professor Masters has presided over ACER’s embrace of e-assessment, including the development and delivery of the first digital assessments for OECD PISA in 2006 (science), 2009 (reading) and 2012 (mathematics and problem solving), using innovative interactive item formats; the first international assessment of computer and information Literacy – the IEA’s ICILS (2013 and 2018); the initiation of online adaptive assessment designs that have been implemented in national population assessment programs including the Scottish National Standardised Assessments and the Australian National Assessment Program – Literacy and Numeracy; and the establishment of a thriving in-house software development unit that has facilitated the delivery of 30 million e-assessments.
Kindly sponsored by
Best Formative Assessment Project Award 2021
WINNER
Prodigy Learning
Game-Based Formative Assessment for Young Learners in Computer Science
Prodigy has developed the world’s first assessments delivered through Minecraft. “Coding in Minecraft” is a new computer science curriculum, formative skills assessment and credential program delivered through Minecraft with formative evidence-based assessments hosted on Prodigy’s skillify platform.
The program engages young and diverse learners (from middle grades US / Key Stage 2-4 UK) through game-based learning in an immersive computer science and coding curriculum and formative assessments, rewarding their learning with credentials. The Coding in Minecraft pathway comprises of four courses with teacher-graded (formatively assessed) micro-credentials which immerse students in a Minecraft world to develop and prove their coding skills.
Minecraft is a game which promotes creativity, collaboration and problem-solving in an immersive environment. It is one of the most popular video games in history that transcends all platforms and appeals to all players regardless of age, demographics, or geography.
Coding in Minecraft is an easy to teach solution enabling teachers with little or no computer science or coding experience to deliver a standards aligned computer science curriculum and assessments. During COVID-19, it is worth noting that Coding in Minecraft is remote ready. Students can complete the learning and assessments from home with remote teacher formative feedback and grading.
FINALISTS
Xamplay Edutech Private Limited
Measurement Incorporated
Riiid Labs
Kindly sponsored by
Best Summative Assessment Project Award 2021
WINNER
Monash University
eAssessment: Delivering Authentic Assessment at Scale
Over three and a half years Monash University has led the global higher education sector in developing an innovative system for electronic-based summative assessments. Their online platform, eAssessment, has replaced all paper-based exams with an engaging online format enabling their whole-of-University authentic assessment strategy.
The advent of COVID-19 saw Monash extend their capabilities, providing an environment and supporting business process for a wholly off-campus yet secure student examination experience. To provide world-leading academic integrity, Monash integrated the eAssessment platform with its own online supervision solution – Monash eVigilation, developed wholly in-house.
eVigilation is an example of Monash University’s education innovation; co-developed with students and academics across all disciplines. It is the backbone of their invigilation approach with the highest standards of academic integrity. Students sit exams remotely, on their own device, under the supervision of Monash-trained staff. AI tools directly support invigilators, ensuring exam conditions are maintained. Student experience is enhanced with the ability to interact with the supervisor, flag concerns, and request assistance. Hailed by students and academics for its functionality, it has proven scalable and adaptable. Attracting attention from notable external bodies, Monash are leading the way in ensuring the global reach of quality remote assessment.
FINALISTS
University of Pretoria
Lexplore Analytics Ltd
Kindly sponsored by
Best Workplace or Talent Assessment Project Award 2021
WINNER
Tata Consultancy
TCS - National Qualifier Test.
Tata Consultancy Services, (TCS), would like to use this opportunity to share details of National Qualifier Test (NQT) designed and hosted by TCS iON, a product arm of TCS focused on quality solutions for Education.
Traditional talent recruitment in India involved companies, including TATA, itself a major employer in India, conducting visits to universities across the country.
This approach however put talented individuals studying in more remote or smaller institutions at a disadvantage. With TATAs long standing commitment to equality and diversity their Talent Management solution enables anyone, anywhere, regardless of connectivity, to compete equally in the job market. Also, their solution reduces reliance on travel and its well-known environmental effects.
Moreover, due to pandemic, campus recruitment and in-person candidate interviews have been eliminated to a great extent, so TCS envisioned virtual recruitment possibilities by bringing in a highly structured Remote Proctored and AI ML empowered Assessment. TCS has successfully implemented NQT twice in last 6 months with more than 400, 000 candidates in first instance in October 2020 and similar numbers in second instance in February 2021. Innovative technology and processes have been adopted to deliver NQT with shortest activation time, traceability, standardization, and security as key aspects.
FINALISTS
- PSI Services
Kindly sponsored by
Most Innovative use of Technology in Assessment Award 2021
WINNER
Bolton College
FirstPass
In September 2019, colleagues at Bolton College embarked on a tentative journey to discover if a computer could be trained to support teachers to assess student responses to an open-ended question; and if real-time feedback improved the quality of student work when responding to such a question.
Bolton College discovered that if we make use of natural language classification, natural language understanding and other tools a computer can indeed be taught to analyse and assess responses to an open-ended question. It is also possible to offer textual and graphical real-time feedback to students.
The work undertaken by Bolton College also enables teachers to create multiple classification models that can be used to support the formative assessment of numerous open-ended questions.
The emergence of this new assessment tool enables teachers to make use of a richer medium for assessing their students. Traditionally, online formative assessment activities are undertaken using closed questioning techniques such as yes/no questions, multiple-choice questions or drag-and-drop activities. Whilst valuable, this is a rather narrow way to undertake formative assessment. This new solution enables teachers to pose open-ended questions which can be automatically analysed and assessed by a computer
FINALISTS
- Tata Consultancy Services Limited
- Excelsoft Technologies Pvt Ltd
- Riiid Labs
Kindly sponsored by
Best Transformational Project Award 2021
WINNER
Cirrus Assessment and Chartered Accountants Ireland
Debuting a digital exam platform to the time-honored establishment of Chartered Accountants Ireland was nothing short of a monumental task, but Cirrus did not shy away from the challenge.
Faced with the rising costs of testing centers and the fast-paced demands of a world going digital, Chartered Accountants Ireland knew they needed to make an enterprising decision to transform their institution. CAI is a prestigious organisation founded in 1888 and is the third oldest accountancy body in the world, dating back to before the Irish republic. They were traditional in the way they approached assessments, paper was always the way exams were taken.
Cirrus signed the contract with the Chartered Accountants in January 2019, with the deadline of delivering their first exams at the end of March, with the additional need to develop two bespoke question types for them. Cirrus has never backed down from a challenge and started immediately with gathering a team of experts for CAI. Cirrus understood that there was much at stake for CAI, both reputational and economic, and therefore when successfully implemented it could mean the complete metamorphosis from old tradition to the digital age.
FINALISTS
- The British Council
- Royal College of Psychiatrists
- Riiid and TOEIC
Kindly sponsored by
Best Research Award 2021
WINNER
Purdue University and RM
Learning by Evaluation (LbE) with RM Compare
Research into formative peer assessment is certainly not a new area of study, however, deploying formative assessment at any reasonable scale has always been challenging, not least because of the additional workload this creates for teachers.
Moreover, traditional efforts around improving student learning and assessment often centre on teachers as the evaluators rather than the students themselves. Typically, the focus is terminal with an emphasis on measuring learning rather than stimulating or promoting learning.
The Learning by Evaluation research project, led by Purdue University in the USA, utilises an innovative online tool called RM Compare, which facilitates assessment for learning at scale through the use of adaptive comparative judgement techniques.
Focus for the project was a group of ~550 undergraduate students enrolled on the same course, where half of the group had the opportunity to evaluate prior cohort work using RM Compare as a primer to their assignment, whereas the other half did not. By engaging the first group with this short learning intervention (~20 minutes), it was found that they developed a stronger understanding of ‘what good looks like’ and significantly improved their attainment.
On-going research is looking at deploying this approach at scale in HE and K-12 contexts
FINALISTS
- Pairwise Pty Ltd
Kindly sponsored by
Best International Implementation Award 2021
WINNER
Excelsoft Technologies Pvt Ltd
An assessment solution implemented for exam delivery across Mexico by an Indian company.
CONSAR is the regulator of Mexico’s pension system, wanted to reduce the complaints raised by Mexico’s workers against the private company’s (AFORES) – that manage the worker’s pension accounts.
AFORES would comply with CONSAR directive to conduct regular testing for their agents, who handled all worker inquiries/clarification, but would not want to impact their business operations, and in a scalable and less expensive manner.
A complex project, where requirements constantly changed post-award and new stakeholders came into picture post-award. With limited timelines to execute the project, technical challenges and regulatory consideration followed by AFOREs – resulted in challenges to be solved at different phase of implementation.
Excelsoft, an Indian company would partner with TAEC and Compecer, both Mexican companies, to build a strong value proposition based on Excelsoft robust, secure and scalable eAssessment platform Saras Test & Assessment.
Excelsoft implemented the project in a country separated from India by 13+ hours time difference, having a different culture, language, and multiple stakeholders with diverging requirements. Excelsoft team would customize Saras to build an innovative proctoring solution - deliver remotely proctored exams across a low-bandwidth scenario to ensure integrity, security, auditability, and defendability of key exams delivered across Mexico.
FINALISTS
- Northern Consortium UK Ltd (NCUK)
- ClarityEnglish, Atlas English
Kindly sponsored by
