Fair Assessment in a Digital Age: Are We Asking the Right Questions?

Immense moral questions face the assessment world as the mid-21st century approaches. For example:

• What evidence about learners, from the large amount now available from different sources (often garnered routinely through digital technology), is it morally right to use to make educational assessments?
• What processes are morally justified to obtain that evidence?
• Are there assessment practices in 2026 that will be thought morally wrong a quarter of a century later?

Significantly, we need to ask ourselves (and each other) whether validity – ‘the most fundamental consideration in developing and evaluating tests’ (AERA, APA, & NCME, 2014, p. 11) and ‘one of the most important attributes of test quality’ (ETS, 2015, p.15) - can be regarded as a moral value. 

I argue here and elsewhere (Nisbet & Shaw, 2024) that philosophical accounts of fairness have largely been based on two fundamental concepts – equality and desert. Of the range of senses of ‘fair’ that can be applied to assessment (see Nisbet & Shaw, 2020): of these, perhaps the most important are a relational sense - which is a form of equality - and a retributive sense, where a fair outcome is an appropriate reward or penalty for what has gone before, and hence is deserved. The relational sense of fairness lies at the heart of the received view of assessment fairness. However, the retributive sense must not be ignored. The concept of desert is often associated with individuals (each candidate gets the grade he or she deserves), rather than with groups, and is linked to the sense of ‘fair’ in which a fair outcome is what those affected can legitimately expect. 

The balance between the concepts of equality and desert can be understood using two different perspectives. 

• Focusing on the test itself, a fair assessment accurately measures the relevant knowledge, understanding or skill of the test taker, and discussion of fairness in this sense often uses the language of ‘accuracy’ and ‘reliability’. 
• A broader perspective sees the assessment in context, with a fair assessment outcome seen as ‘deserved’ because of the effort that the student has done or because it matches some other evidence of the student’s ability.

Importantly, the two philosophical bases raise two questions for the future of a fair and just digital assessment:

Will people be treated the same in relevant respects?

Will test-takers and users of the assessment outcomes get what they have a right to expect/what they deserve from it?
Across our industry and against the backcloth of technological innovation, our future focus should continue to centre on the primary measurement standards that must be met to legitimise a proposed test use (Messick, 2000, p.1). In other words, reliability, validity, and increasingly, fairness.  

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing. Washington, D.C.: American Educational Research Association.

Educational Testing Service (ETS). (2015). ETS Standards for Quality and Fairness. Princeton, NJ: ETS.

Messick, S. (2000). Consequences of test interpretation and use: the fusion of validity and values in psychological assessment. In R.D. Goffin & E. Helmes (Eds.). Problems and Solutions in Human Assessment: Honoring Douglas N. Jackson at Seventy (pp.3-20). Norwell, MA: Kluwer Academic Publishers.

Nisbet, I., & Shaw, S. (2024). Educational assessment in a changing world: Lessons learned and the path ahead. Routledge

Nisbet, I., & Shaw, S. D. (2020). Is assessment fair? Sage Publications Ltd.