Beyond Multiple Choice Session 6: AI Can See More. That Doesn’t Mean It Knows More.

The Future-Ready Assessment Ecosystem: Institutions

Assessment organisations are navigating rapid technological change, evolving learner expectations and increasing demands for efficiency, quality and governance. Building a future-ready assessment ecosystem requires institutions to rethink how assessment is designed, delivered and managed.

This week focuses on the organisational challenges and opportunities shaping modern assessment. Sessions will explore AI in assessment operations, strategic planning, data-driven decision making and AI governance, providing practical insights into how institutions can build resilient, responsible and scalable assessment systems.

About this session

Where Human Judgment Belongs in the Future of Assessment

AI is giving assessment systems unprecedented visibility into learner behaviour. It can detect patterns, compare activity, identify anomalies and surface moments that might otherwise go unnoticed.

But visibility creates a dangerous illusion: that because a system can see more, it necessarily understands more. A learner looks away from the screen. Another person enters the room. A second device appears. An answer changes unexpectedly. Each may be observable. None, by itself, explains intent.

As assessment becomes more automated, the challenge is no longer simply whether AI is accurate. It is whether institutions are asking technology to make judgments that the available evidence cannot support.

In this session, Brandon Smith will examine the growing gap between observation, interpretation and decision in AI-enabled assessment. Using real-world examples, he will explore how seemingly objective signals can become misleading when stripped of context, and why adding more detection does not necessarily create more trustworthy decisions.

The session will introduce a practical model for designing assessment systems around three distinct functions: machines observe, evidence informs, humans judge. It will also explore when automation is appropriate, when human intervention becomes essential, and how institutions can create defensible processes without turning every irregularity into an accusation.

The question for the next generation of assessment is not simply What can AI detect? It is What are we willing to let it conclude?

What attendees take away:

A framework for separating observation, interpretation and judgment
A way to identify where automated assessment systems may be making unsupported leaps
Practical principles for determining which decisions can safely be automated
A model for human oversight that adds context without simply recreating manual assessment at scale