Cracking the Code: Balancing Candidate Experience with Exam Integrity
By Casey Kearns, Director of Product Management, Pearson VUE
There’s an interesting relationship between security and customer experience. In an attempt to thwart a small group of unknown bad actors, the experience of good actors is to some extent lessened. There are plenty of familiar examples at various stakes levels: intrusive body searches at airports, car theft alarms that mistakenly go off, corporate password creation and maintenance policies… as consumers, nearly every aspect of our lives is affected by this relationship. So, of course, it is expected that high-stakes exams – those with significant conseqeuences to test takers, community safety, employers, or the profession – would also be subjected.
Our technology-enabled world has changed everyone’s expectations. We’ve all come to expect fast and easy provider-to-consumer interactions with very little friction. But just because technology lets us deliver high-stakes exams without friction, does that mean we should?
Just as a thought experiment, consider the most “frictionless” experience for testing. Perhaps it would be an honor system where test takers get the exam paper and key any time they want, and simply report back their performance. A passing score earns a certification. Everyone is being honest, right? Of course, one would question the integrity of the exam. The scores would not be reliable or valid. It would not be a consistent experience. The content would not be protected. Bad actors could self-certify, and eventually the certification would have no value in helping others determine the true abilities of the holder.
While this kind of testing experience is sometimes used for pre-tests and practice tests, it isn’t acceptable for tests that determine outcomes such as the ability to practice in a profession, immigration status, university admission, and career advancement. It opens the door to most, if not all, of the categories of cheating like knowing the exam ahead of time, working with someone else, submitting someone else’s work, using external resources during the exam, sharing content, etc. Cheating impacts the integrity of the exam and ultimately impacts the credibility and trust of the profession.
Over the years, various methods have been created and deployed to detect and deter cheating. Considering our honor system exam above, we could use a single, monitored testing location as a way to control the content and hopefully intervene should any perception of cheating be observed. This comes at the expense of experience—what about test takers who don’t live close to your testing location and need to travel? What about those who might have other equally important commitments at the exact time of the test? What if there are too many test takers for the venue to comfortably accommodate?
Now we’re talking about possibly offering the exam at more times and locations. We may need multiple forms for our exam, and maybe we’ll need to delay releasing scores until we’ve had a chance to compare across cohorts. Perhaps we’ll establish eligibility rules and have candidates show their identification when they arrive, so we know they’re the right person. Maybe we’ll hire additional staff for additional, continual surveillance. This additional level of security comes at the expense of experience—no immediate result, an application process, ID requirements, and longer testing check-in times to establish identity and eligibility.
I think you get the point. And I’m not here to challenge the use of any of these tactics (or the many others that can be and are used daily). As an industry, however, we are being challenged by test takers with updated expectations formed through their consumer experiences. While some “friction” may be necessary, we should be doing what we can to make the experience better. After all, airport passenger background checks and light screening have made traveling easier, advanced GPS and connected home systems have improved car alarms, and the move to passwordless authentication has made connecting to applications easier– all while maintaining the intended level of integrity.
I’ll leave you with a few questions to consider:
- How have you seen your test taker expectations change?
- How do you decide which security methods are the right ones?
- Are there risks to public safety or to the profession that must be considered if integrity is sacrificed in any way?
- Are there ways to improve experience without sacrificing integrity? Are there ways to improve or maintain integrity without sacrificing experience?
- How much do you weigh pre- vs. during- vs. post-delivery security methods?