Innovation and Adaptation for Finance and Accountancy Assessments

The Surpass Community has strong representation from the finance and accounting sector, including Association of Accounting Technicians, Association of Corporate Treasurers, CPA Canada, Institute of Chartered Accountants in England and Wales, Institute of Chartered Accountants of Scotland, London Institute of Banking and Finance, and South African Institute of Chartered Accountants. With candidates in this field often studying whilst working in the sector, it’s important that assessments test real-world skills to produce work-ready candidates. 

In a recent Surpass Community webinar, Robbie Burns, Assessment & Quality Lead, and Ioana Raileanu, Senior Assessment and Quality Specialist from ICAS, and Andy Thomas, Director – PEP Examinations – Evaluation and International Assessment, from CPA Canada joined Jim Crawford, Business Development Director at BTL, to discuss the importance of being able to simulate authentic scenarios to test real-world skills in assessments. The audience also got a fascinating insight into how both organisations have taken different proactive approaches to ensuring testing can continue during current events.

Read on to discover more, or re-visit the webinar in full here. 

Adapting During a Global Pandemic

Like many certification bodies over the last year, ICAS and CPA Canada have both had to implement new ways of testing to ensure candidates can continue with their qualifications in the absence of centre-based testing. The webinar provided a fascinating insight into how two organisations delivering exams in varying scale found solutions that worked for them.

CPA Canada traditionally provide centre-based assessments across the country, the largest venue accommodating 1,200 students at once. CPA Canada are therefore no strangers to managing the logistics of large-scale exam delivery, but with hosting examinations in large venues currently impossible, the organisation came up with an innovative way to continue testing.

To continue delivering large single-day examinations for their high-stakes final accreditation exam, CPA decided to rent 10,000 hotel rooms across approximately 200 hotels, as close as possible to the original exam venue. The 10,000 rooms were all set up with exam-ready devices, provided by CPA Canada to add an extra layer of security and control, in addition to controlling the test environment – logistically and administratively an incredible amount of work.

For this novel delivery method, proctors walked up and down hotel hallways during the exam, with candidates leaving room doors open in order to be monitored. This approach allowed CPA’s assessment schedule to remain consistent, the exam to remain secure, and by all accounts, students were pleased with this unusual but extremely innovative approach.

Hear Andy Thomas talk more about this approach in this short clip: 

With more frequent exams and at a lower volume than CPA Canada, ICAS took a different approach to exam continuity, to best solve their specific challenges. For entry level exams, ICAS opted for an on-demand, remote testing model for a total of 45 weeks of the year, providing more freedom for the candidate to choose an assessment slot but making it ever more important to preserve exam integrity.

ICAS utilised LOFT (Linear-on-the-Fly Testing) functionality in Surpass to help maintain the integrity of the item bank and maximise test security. The LOFT functionality presents each candidate with a unique set of items to a test blueprint, whilst taking into account item exposure, ensuring no items are used repeatedly to the neglect of others.

Innovating with item types for real-world testing

As with any qualification preparing learners for a specific job role, being able to simulate true to life scenarios in an assessment is a high priority for those in the accounting and finance sector. This sector typically embraces the use of innovative question types to closely reflect real-world scenarios, and ensure candidates meet the competencies of changing roles within the space.

“More accurately reflecting how a candidate works in the real world today gives the organisations a better indication of how work-ready and well-equipped a candidate is.” – Robbie Burns, ICAS.

Both ICAS and CPA Canada gave an overview of how they construct exam content, including a mix of MCQ, long form, and spreadsheet simulation item types, often utilising Surpass functionality such as the ability to view extensive source material side-by-side with test content. Simulated environments allow greater control over what the candidate can access, whilst providing functionality that candidates are familiar with.

The panel then moved on to discuss the recently launched Task Based Simulations (TBS) advanced item type in Surpass which replicates the tools that would be used in the workplace, with functionality familiar to candidates, meaning they can focus solely on the assessment without being distracted by the technology. As with all Surpass item types, even this advanced item type with countless ways in which it could be used, is intuitive and straightforward to set up, and despite the complexity and permutations of response, is entirely auto-marked.

Get an overview of what the TBS item type has to offer in this short demonstration: 

CPA Canada plan to roll out this item type across their assessment programs, replacing the use of MCQs in entry level exams, and cases in higher level assessments. 

Andy Thomas highlighted five key benefits of introducing this item type:

• Consistency to be found in auto-marking
• Ability to replicate real-life scenarios and test real life skills
• Assesses the higher skills of Bloom’s Taxonomy, not just memory skills
• No human marking reduces cost and increases efficiency
• Analysis provides great metadata and binary data points for analysis

To watch the full discussion between Jim, Andy, Ioana and Robbie, visit surpass.com/webinars.

For further information on how Surpass might benefit your finance certification program, visit our dedicated finance and accountancy page surpass.com/finance.