Case study: Chartered Accountants Ireland and Cirrus Assessment

Background

Debuting a digital exam platform to the time-honoured establishment of Chartered Accountants Ireland was nothing short of a monumental task, but Cirrus did not shy away from the challenge.

An Accounting Body’s approach to assessments

CAI is a prestigious organisation founded in 1888 and is the third oldest accountancy body in the world, dating back to before the Irish republic. They were traditional in the way they approached assessments, paper was always the way exams were taken.

Challenge

Chartered Accountants Ireland was heavily dependent on paper-based assessments to successfully examine students whilst undergoing the ACA qualification at all levels.

Moving all of CAI’s high-stakes exams from paper to digital

Initially, Chartered Accountants Ireland’s desire had been to migrate all of its existing paper based examinations, roughly 25,000 paper-based assessments per annum, to an e-assessment platform on a phased basis over the next three to five years.

Need to speed up the implementation

A pilot program was to be rolled out covering 10% of the exams but due to COVID-19, lock-down measures had to be implemented in March. This pushed the entire project into overdrive.

“We started with a pilot program, and intended to roll out 10% of exams slowly over the course of five years. But because of the COVID lockdown measures, we suddenly needed to be online in just five months, covering ALL exams. Testing centres were closing and we needed a solution quickly.”

Ian Browne – Deputy Director of Education, Chartered Accountants Ireland

The challenge was tremendous, but if implemented properly, this new assessment platform would assist in reducing existing examination venue capacity issues, reducing workload relating to GDPR, increasing global reach of the ACA qualification and enhancing student satisfaction.

Partnership with Cirrus

In the autumn of 2018, CAI organised a tender process to select a provider for both remote proctoring and an end-to-end e-assessment platform. After a thorough process, the choice fell on Cirrus and ProctorU, who stood out from the rest by being innovative and user-friendly and gave CAI the confidence that they would be well taken care of.

“Cirrus and ProctorU stood out by being both innovative and user-friendly. And they gave us the confidence that we would be well taken care of and supported.”

Ian Browne – Deputy Director of Education, Chartered Accountants Ireland

The contract was signed in January 2019, with the deadline of delivering their first exams at the end of March that year. An additional request was the development of 2 bespoke question types for CAI.

Solution

Implementing the e-assessment platform across the whole CAI organisation

It was recognised that for the rollout to be successful, multiple stakeholders would need to be involved in the process and commit to working together to make this project a success.

This process included:

• Identifying and mapping IT competencies
• Organising a “kick-off” for all users involved
• Training of participants in project teams, users in need of extra IT support, super-users and administrators
• Individual coaching / assistance for project participants via Internet / phone.
• Frequent meetings of project group / management committee.
• Discussion panels in which participants exchange experiences, show progress and achievements, and discuss any challenges.
• Project completion including a final evaluation of the project

Safeguarding the security and reliability of high-stakes accountancy exams

Not only was there a short time-frame to develop bespoke functionalities, but the high-stakes nature of the assessments CAI is tasked to give made this project especially complex.

“Chartered Accountants Ireland are guarantors of their Member’s reputation, so it was an unquestionable must that Cirrus be able to deliver exams with complete security.”

Vegard Sivertsen – CEO, Cirrus

There had to be no risk of any leakage of CAI data or of 3rd parties, accidentally or otherwise, coming into possession of any CAI data.

Chartered Accountants Ireland are also Data Controllers and process their Member and Student’s personally identifiable data on their servers. Cirrus is a Data Processor within the meaning of the General Data Protection Regulations and had to satisfy Chartered Accountants Ireland as to their ability to secure CAI’s data and to meet their responsibilities under the GDPR. Cirrus was able to go above and beyond the requirements of what CAI needed in terms of GDPR in that no data tracks back to the student after the exam has been taken.

Results

Just five months after signing the contract with Cirrus, CAI delivered the first digital exams and since 2020, 100% of CAI high-stakes exams have been delivered online via Cirrus.

“Tonight, as a huge combined team, we managed to create a truly momentous milestone in the assessment of professional accountancy globally. Chartered Accountants Ireland’s first-ever remote proctored fully regulated exam completed successfully!”

Ian Browne – Deputy Director of Education, Chartered Accountants Ireland

Delivering exams anytime, anywhere

For CAI, the benefits of digital exams are numerous:

• They now have the ability to give exams to students all over the world in all circumstances. Tragedies like COVID-19 will never completely derail their students’ ability to take exams and continue prospering in their careers.
• Thanks to results being delivered more quickly due to digital grading, students are able to let accountancy firms know their results quicker, thus resulting in them being able to progress to a better payscale faster.
• CAI has also saved money on the administration costs of setting up, invigilating, and hosting exams in testing centres.

Ultra-fast implementation

Perhaps the most unique aspect of the project was not only the enormous task of making the switch from paper to digital, but the condensed timeframe in which it was done within. This meant that absolutely everyone on both teams needed to come together as one united force in order to make the digital shift happen smoothly. This transformation would not have been at all possible had it not been for the communication, support, hard work, and open-mindedness of these two companies.

Achievements

• 100% of CAI high-stakes exams online in 2020.
• Full regulatory approval of exams security.
• Annualised pass rates consistent with prior 3-5 years.
• Quicker exam results feedback for students.
• Reduced environmental footprint.
• Customer retention for CAI and growth into other markets.
• Reputational boost for CAI.