Surpass
Is AI the answer? Why testing programmes still need humans in the loop
Just like a new roof isn’t going to help a house with a crumbling foundation, AI can’t turn a flailing testing program into a successful one.
Is AI the answer for testing programmes? What do assessment systems and testing programmes need to do to reduce pain points and be successful? In this thought-provoking article, Laura Steighner, Director of Test Development Customer Experience at Surpass Assessment, explores how underneath the excitement of AI, a testing program needs to get the basics right: "Just like a new roof isn’t going to help a house with a crumbling foundation, AI can’t turn a flailing testing program into a successful one."
Laura also shares feedback from her research with Surpass customers on their testing programs, and reveals what they want to see from assessment systems.
AI and automatic item generation (AIG) became the ‘it’ words at conferences more than a decade ago.
Back then, the issues I was dealing with in high-stakes testing were mostly focused on the cognitive burden and workload of my key subject matter experts (SMEs).
I argued that I would rather AI focus on reducing the time and effort that went into assembling test forms than producing a plethora of items that would create a backlog that still required technical review and revision – work that the SMEs would ultimately have to do.
Today, it is gratifying to see that AI is being used in ways beyond just AIG that can support test development efforts (e.g. enemy item detection, semantic searches, test construction). However, AI alone can’t fix a broken system.
Underneath the excitement of AI, a testing programme needs to already be getting the basics right. So what do assessment systems and testing programmes need to do to be successful?
Since I joined Surpass in 2025, I have talked to many Surpass customers about their testing programmes, how they leverage the tools at hand, and how we could better support their process.
Their feedback on what they want from assessment systems centered on usability of software and services and ensuring a smooth process working with their SMEs, capturing their authoring, edits, reviews, or other approvals in intuitive and accurate ways:
1. Data control: Customers should be able to manipulate their own data.
2. Content-first views: On-screen views should prioritise content and metadata.
3. Scalability: Software development should consider scale impact to ensure that usability does not falter and features work well when concurrent writing and reviewing sessions scale up.
4. Efficient navigation: Navigation and functionality should be efficient and intuitive to users.
To explore the wider issue of how testing programmes can improve, read the full article here
