AI & ChatGPT: Challenge or Opportunity for e-Assessment?

Within 5 days of OpenAI launching ChatGPT at the end of November 2022, 1 million people had registered as users. Social media was suddenly awash with comments, many heralding ChatGPT as a gamechanger, and immediately experimenting by asking the chatbot questions.

Matt Wingfield, Chief Executive of the e-Assessment Association has spoken to Board members to get their thoughts on the implications of such developments for assessment, and more specifically e-assessment.

Let’s start with AI

Matt Wingfield reflects that, “Chatbots are not new and the use of AI in general is becoming an increasing part of our everyday lives, whether obvious or discreet.” eAA Executive Board Member Patrick Coates agrees, “AI has been around for a while now and is already being used in lots of different industries.”

Tim Burnett, eAA Board Member and Independent Assessment Marketing Consultant, already uses AI to support his marketing work as a consultant, “it helps me but it's not ready to take over my job. That time might come, and the important thing is not to be blind to that fact.”

eAA Vice Chair, Paul Muir adds “AI has been around for 10+ years helping in the marking of assessment and creating high-stakes, valid assessments. AI is not the problem, how it is used is.”

Now for the gamechanger

However, ChatGPT is, arguably, a gamechanger, generating a lot of interest for its ability to interact in a conversational way, write code and answer essay-style questions on complex topics.

What are the implications for assessment, and more specifically e-assessment? Here are some thoughts following Matt Wingfield’s discussions with some of the Board Members of the Association. In a follow-on article, we will bring you some views of individual members, and we want to hear from you too. AI is here to stay and assessment and technology such as ChatGPT cannot be ignored.

Martha Gibson, eAA Partnerships Secretary and Enhancements & Innovations Manager at Heriot-Watt Online at Heriot-Watt University in Edinburgh writes, “Having just got comfortable with the changes we had to make to our HE assessments during Covid-19, we’re presented with another seismic challenge which could potentially wrestle the security blanket from us.”

Hearing about ChatGTP while taking part in a panel discussion at a University Conference in Delhi, eAA Board Member, Teresa Jacobs wonders who else will remember exactly where they were when they first heard about it.

“My initial feeling is that in India this will drive assessment back to exams, still largely paper based. For UK commercially orientated Awarding Organisations, especially those working internationally and less under the scrutiny of OFQUAL, and universities where assignments are a key mode of assessment this has surely got to trigger response. Ghost writers were pretty much treated like ghosts in academia - nobody believed in them. This isn’t a ghost. It’s a huge charging elephant.”

After playing around with ChatGPT, Teresa came back to add, “I thought I had been alarmist but as an EQA I am concerned. I have since played about with ChatGPT and now feel that it is not going to yield completely referenced responses at higher levels - but it’s still too big to ignore.”

Tim Burnett, eAA Board Member and Independent Assessment Marketing Consultant, writes, “I find the news of the release of ChatGPT as interesting and significant, but not alarming. Organisations can help mitigate the risks of this kind of technology and the abuse it attracts by using multiple methods to fairly test candidates. High-stakes tests should never really be dependent on one single test on a single day. The testing industry, like a lot of industries, is monitoring closely the role AI has to play in how develop tests, mark responses, detect cheating and of course how it can be used to cheat the test.”

Board Member’s Patrick Coates and Dan Howard both agree with Tim’s take on it. Tim continues to say, “We're all travelling on a fast-moving river of change when it comes to AI, there's little point trying to paddle backwards, it's a waste of energy. What we need to keep doing is working together to navigate it properly and reduce the chance of it getting out of control. We need to be constantly looking ahead to what may or may not happen next. People cheat, people have always cheated, technology-based assessment just means we know more about what is happening than we might have done in the past.”

eAA Vice Chair, Paul Muir says that ChatGPT is “just the latest in a long line of ‘developments’ in the assessment sector and while it is often a knee-jerk to see it as a bad thing, we should be looking to how the technology behind it and its use can be turned to a positive.”

Matt Wingfield, eAA Chief Executive also sees this as an opportunity to innovate, “ChatGPT represents an exciting development of AI technology and as with all forms of innovation, it gives us an opportunity to reflect on the way that we do things. From a learning and assessment perspective, the rise of more sophisticated chatbots shouldn't be seen as a challenge. They are an opportunity to provide innovation around how we support learning, and to ensure that assessment is appropriately designed for modern-day learning.”

Matt concludes, “There are already proven examples of where chatbots and AI have been embraced by education to amazingly positive effect - for example the award-winning ADA and FirstPass projects from Bolton College - where the technology not only enriches learning but enables teachers to focus on supporting the development of higher order skills in their students. Chatbots and AI are here to stay, and we will see lots of further innovation in this area, so let's embrace these advancements to ensure that we are delivering learning and assessment approaches fit for the 21st Century.”

Graham Hudson, Chair of the eAA, reached out several of our individual members asking them for their thoughts. Melanie Atkin sees challenge that can be overcome and turned around, “ if AI is smart enough to write unique and creative responses to questions, then it negates the argument that AI can’t mark essays.”

Watch out for articles from our individual members. We want to hear from you too - does this worry you, are you already embracing this technology, do you see this as a challenge or opportunity? And even, where were you when you first heard the news!

More views from eAA members

Thunderbolts, Vinegar, and Rug-Pulls – how ChatGPT will change the assessment sector by Geoff Chapman

More Articles

AI breakthrough ChatGPT raises alarm over student cheating. Financial Times, 18 December 2022 

‘Google is done’: World’s most powerful AI chatbot offers human-like alternative to search engines. Independent Newspaper, 2 December 2022 

We made the ChatGPT AI take a GCSE History exam, and had it marked by a teacher (inews.co.uk) iNews.co.uk, 7 December 2022 

ChatGPT - Wikipedia 

Some thoughts on Twitter

Sam Altman @Sama https://twitter.com/sama/status/1598038815599661056?s=20&t=knT1U81_JgTFu0NqgfBgSw