the e-Assessment Association

A responsible approach to AI in assessment

A responsible approach to AI in assessment

A blog by Paul Muir, Chief Strategy & Partnership Officer at Surpass Assessment & eAA Vice-Chair

We enjoyed some fantastic discussions about the future of AI, at the Association of Test Publishers (ATP) Conference this March. 

The Surpass innovation demonstration on Generative AI and Surpass Copilot discussed the explosive and transformational impact of generative AI on the education and assessment industry. Our sector has been grappling with how we can harness the power of this new technology, while remaining cautious of its inherent risks and issues 

A demonstration of ‘Copilot’ revealed how it marries the experience of Subject Matter Experts (SMEs) with the speed, efficiency, and innovation of generative AI. Working in harmony with SMEs, augmenting their expertise and knowledge, it can effortlessly generate high-quality distractors, and the rationale for each one.  

What makes this AI offering unique in the industry is that it is a community-driven initiative with initial features and subsequent development roadmap decided by the Copilot Advisory Board (CAB), to help drive the future of assessment. 

Advisory Board members include: 

  • Finance – Chartered Professional Accountants of Canada (CPA) 
  • Healthcare – National Board of Medical Examiners (NBME) 
  • Legal – National Conference of Bar Examiners (NCBE) 
  • Engineering and Manufacturing – EAL 
  • IT – Microsoft 
  • Education – Scottish Qualifications Authority (SQA) 
  • External legal expertise – Mark Weinstein 

 The community-led Board has now released a full set of AI principles* which assessment technology providers can use to guide their incorporation of generative AI into their software.  

 *The assessment technology provider will be accountable for ensuring all AI functionality adheres to these principles. 

Generative AI should be: 

Customer-driven 

Customers must maintain complete authority over the adoption and use of AI technology, in alignment with their own organizational principles, objectives, and guidelines. 

 Accurate & transparent 

AI technology outputs should be accurate and include justifications and citations to sources so that users can determine whether outputs are accurate, reliable, and appropriate for their intended uses. 

 Unbiased 

AI technology should incorporate processes that continuously aim to identify and eliminate any bias and discrimination for its designed use, so that it does not cause harm to others. 

 Secure & private 

AI technology must have robust enterprise security controls and compliance with current industry standards, so that no customer-provided data (including prompts and outputs) is accessible to unauthorized parties or used for training data. 

 Human-monitored 

AI technology should be designed to enable ongoing monitoring of AI outputs for error detection and correction, and to ensure ease of human oversight. 

 Intuitive 

The inclusion of AI technology should be engineered to be intuitive and ensure a seamless user experience, so that only minimal additional training is required for users and SMEs. 

 Embedded 

AI technology should be embedded directly within existing user interfaces wherever possible and appropriate, so that users and SMEs can take advantage of AI at the touch of a button within their existing workflows and processes. 

 Expertly directed 

A dedicated, multi-disciplinary Advisory Board should provide insights and thought leadership so that the responsible use of AI technology within assessment products and services is thoughtfully and effectively developed. 

Taking these principles, combined with the deep industry knowledge of the Copilot Advisory Board, we are setting out the future of Generative AI at Surpass Assessment and will be using that knowledge to support the development of AI Principles across the wider industry as part of ATP’s AI Advisory Group. 

Come with us on our journey – share your thoughts and experiences as we navigate what will no doubt be an exciting but turbulent 12 months for AI in assessment! 

Share this: