Anytime Assessments by AKAD University and Cirrus: a case study

Background

AKAD University aims to provide its students with a flexible, individualised distance-learning and examination experience. The institution has over 90 degree programs and strives to cater to a vast demographic, including individuals with full-time employment, families, those that reside overseas, or have physical disabilities that restrict them from attending a conventional university.  AKAD’s study program is 100% online, has no semesters and can be started at any time, and exams are completely flexible – the student decides when to take them.

AKAD’s looming challenge was to provide its 12,000+ students with exceptional online exam preparation and the flexibility to take exams from home at their convenience. Despite AKAD’s students predominantly learning online, they sought to leverage technology to create a more connected experience than what traditional lecture halls can offer.

Solution

AKAD arrived at the decision to choose Cirrus because of their Blueprint technology, which allows users to automatically create balanced assessments based on a set of criteria they select. Cirrus Blueprint has several ways to select and classify items, such as taxonomies, learning objectives, and topics. In the Cirrus platform, topics are highly flexible and customisable. AKAD could use topics to define item difficulty, thus creating a blueprint with difficulty levels and credit points built into it.

At AKAD, exams are crucial as they determine the credit points that students need to accumulate to complete a module successfully. Exam credit point values range from 45 to 240, with a specific minimum number of points required to pass the module. A time calculation has been determined for each item that equals points; the more time required for an item, the more credit points a candidate earns. For example, 1 minute to answer a question equals 1 credit point, and these questions would therefore also be classified as ‘easy’ questions. 5 points for a question that averages a 5-minute response time would be considered a ‘medium difficulty’ item, and so on. Taxonomies are also factored into the equation of how difficult an item is. Basically, AKAD, combined with Cirrus blueprint technology, has determined an equation for difficulty rating: taxonomy + minutes to answer = difficulty and credit points. In addition, Cirrus will commence work in 2023 on an algorithm that will make it even easier to deliver randomised, psychometrically balanced exams in a Eurostars-funded project.

AKAD has leveraged Cirrus technology in an unparalleled way by utilising blueprint technology for creating fair and legally defensible exams automatically. This technology enables AKAD to generate completely randomised exams on demand, giving students the flexibility to take their tests from anywhere around the world.

Implementation

As always, implementing significant changes to teaching methods will always come with resistance, from sceptics who cling to traditional practices and a “we’ve always done it this way” attitude to legal frameworks and data protection and the monumental task of getting students and teachers fully on board.

AKAD recognised these obstacles and proactively engaged students and teachers throughout the implementation process, gathering targeted feedback through extensive surveys and establishing exchange groups to address specific issues.

AKAD succeeded – particularly with mathematics exams – in developing tasks that effectively utilised the Cirrus platform, ensuring exams retained their quality and ambition when mapped online. These successes and the benefits of online exam preparation convinced even the staunchest of sceptics and procrastinators of the value of AKAD’s approach.

AKAD has developed an effective process for optimising online exams, including a quality assurance program. The authors are guided through the first 30 exams to ensure questions, taxonomy levels, and feedback are on point. Before the exams go live, a formal and technical quality check is performed, and three months after launch, the exams are reviewed again. Various factors are taken into account, such as grade, failure rate, and student feedback, to identify areas for improvement. Students can also provide feedback on individual questions. This showcases the strong partnership between AKAD and Cirrus, who share a common vision for online assessments.

The student satisfaction ratings speak volumes, with students giving AKAD’s online exams an impressive 8.6 out of 10. Furthermore, with a rating of 9.9, students report that they can study more flexibly thanks to this innovative exam format. AKAD’s forward-thinking approach has contributed significantly to the learning experience and opened doors for more accessible and personalised education.

Results: delivering assessments the scalable way

AKAD’s innovative approach to online exam delivery and automation through its partnership with Cirrus has paved the way for scaling and growth. With automatic grading and parallel process automation, exams begin immediately, markers are alerted, and grading is streamlined, making the solution scalable and manageable with minimal staff. AKAD’s 70,000+ tasks across 500+ exams and continuously increasing student enrollment is largely due to the success of their online exams and exam prep, helping transform education to be more accessible, personalised, and scalable.

AKAD is currently seeing the following significant wins:

• 93% of AKAD students now take exams online by choice with Cirrus. As per German Law, the option to take them in-person is still a possibility.

• AKAD’s fail-rate has decreased by 50% across the board, without affecting average grades.

• AKAD uses these grades to maintain consistent exam difficulty.

• Randomised, on-the-fly “Anytime Exams” can be taken by students whenever is convenient for them.

• The introduction of practice options and exam simulations has lowered exam anxiety and contributed to the decrease in failure rates.

• AKAD won the e-Learning Award 2022 in the ‘Assessment’ category for their innovative exams using Proctorio and Cirrus.

• 8.6/10 student satisfaction rate overall