The e-Assesment Association Company Logo
the e-Assessment Association

Regulating Technical Qualifications Consultation

Regulating Technical Qualifications Consultation

Response from The e-Assessment Association

In July 2018, The Office of Qualifications and Examinations Regulation (Ofqual) that regulates qualifications, examinations and assessments in England, launched a consultation on their approach to regulating Technical Qualifications.

The consultation has now concluded and the full outcome can be found on the UK Government website. Click here to read the full report.

Initially raised by individual members, the e-Assessment Association agreed that it was appropriate to comment on specific questions within the consultation and we have published our responses below. Please do not hesitate to contact us for further information on this matter, or should you feel the Association could comment on other such consultations worldwide.

The eAssessment Association (eAA) is a not-for-profit membership organisation based in the UK for consumers of, producers of and those with an interest in e-Assessment globally. We have been dedicated to furthering technology enhanced assessment adoption since 2008 and have a membership of some 1,600 individuals and commercial organisations drawn from all sectors of formal education and work-based learning.

Since 2011 we have sponsored the cross Awarding Organisation government advisory group on e-assessment related matters - the eAssessment Advisory Group (eAAG) - which was originally founded by BECTA in 2005 to advise policy relating to the use of technologies to support and enhance the delivery of assessment across all sectors of education.

This response has been written with reference to opinions and suggestions drawn from across the eAA’s membership and is thus aimed at providing a balanced response from this broad community of education and assessment professionals.

Q12: To what extent do you agree or disagree with our proposal to set a requirement that that all assessments for Technical Qualifications should take place in May/June?

[X] Strongly disagree

The association’s members believe that constraining the assessment of technical qualifications to one annual ‘window’ will not only disadvantage learners by negatively impacting on the timeliness of assessments, but more significantly will inadvertently constrain the adoption of innovative e-assessment technologies.

Schools and colleges lack of the necessary technical resources (computer devices and/or connectivity) to support the effective use of e-assessment technologies to deliver assessments when these are constrained to a single, ‘big-bang’ assessment window. This constraint will reduce the opportunity for schools/colleges and assessment organisations to adopt innovative, technology-based approaches that could boost the authenticity and validity of the assessment of the technical qualifications.

The use of on-demand assessment, supported by the use of e-assessment technologies, has permeated though many vocational and professional qualifications and examinations in the last two decades. On-demand testing, e-portfolios for project work and device agnostic assessment have become ‘everyday familiar’ to millions of vocational and professional learners, employers and educators. Given this strong usage base for on-demand, e-assessment based assessment, it is therefore very surprising that the consultation appears to be tethered to the General Qualifications regime, schedule and practices.

This is incongruous and anomalous with existing vocational and professional education practice. Awarding organisations are driving e-assessment practices and norms that 21st Century stakeholders need to support vocational and technical education and employment. Assessing the knowledge core with an underpinning theory exam and a skills assessment via project is now, by default, supported via e-assessment – delivering high quality in a timely, robust and valid manner. Occupation-specific requirements/ qualifications with a licence to practise, such as security and construction, have been delivered on-screen for many years.

Contemporary learners will be baffled by having to wait for one opportunity to take an exam, probably on paper, to complete their T-Levels.

T-Levels represent a once-in-a-lifetime opportunity to release learners and educators from hard-boiled, historic practices, drive efficiencies, and deploy world-leading qualifications that leverage in-situ, proven e-assessment technologies and processes. It is therefore disappointing that the technical qualification consultation guidance appears to be misguided and misaligned to contemporary vocational and professional education practices, and will thus inadvertently prove restrictive to achieving a world-leading qualification.

Furthermore, it is well proven that the use of e-assessment technologies can provide greater insight and opportunity to improve assessment reliability regardless of whether the assessment is taken in one or many windows. It therefore seems short-sighted to mandate a single, annual assessment window in the belief that this will ensure assessment reliability, when this will inadvertently constrain the adoption of assessment technologies that have been proven to help increase and monitor assessment reliability.

The use of e-assessment technologies to support high-stakes vocational and professional exams is well proven, trusted, scalable, valid and highly reliable. It is being used extensively across the UK and beyond to deliver world-class assessments.

We therefore urge Ofqual not to inadvertently constrain the potential for schools, colleges and assessment organisations to deliver innovative, world-class assessments for technical qualifications using e-assessment technologies, by insisting on a single assessment window. The Government’s own ETAG Report of 2016 states that “Ofqual should actively support and encourage the Awarding Bodies to develop digital technology-enabled assessment” and the development of a new qualification, with relatively low entry numbers, provides the ideal opportunity for this to be implemented.

Finally, we recognise that in the past, Ofqual has been unwilling to specify how assessment is delivered, leaving that decision to the Assessment Organisations operating professionally within the market. However, for T-Levels, with monopoly supply, and complex regulatory arrangements, if Ofqual is not pushing for innovation in assessment, through the use of e-assessment then it is very unlikely to occur, and learners will miss out on valid assessment of their knowledge and skills.

Q34: To what extent do you agree that we should require core skills assessments to be marked by the awarding organisation?

[X] Disagree

Q35: To what extent do you agree that we should require Occupational Specialism assessments to be marked by the awarding organisation?

[X] Disagree

Whilst we strongly agree with Ofqual’s desire to ensure that the approach to assessing core skills and Occupational Specialisms must be robust in order to support reliability, the maintenance of standards over time, we disagree that the only way to achieve this is by requiring the Awarding Organisation to mark all such assessments.

Ofqual’s suggested approach to this is incongruous and anomalous with existing vocational and professional education practice, where the adoption of e-assessment approaches to assess skills and the application of core knowledge has been hugely successful with no loss of public or employer confidence in the validity or reliability of the assessment. The opposite is, in fact, the case. Assessing core skills via project is now, by default, supported via e-assessment – delivering high quality in a timely, robust and valid manner.

In addition to providing more timely, agile assessments, the most significant advantage to these established vocational and professional e-assessment practices is that of assessment validity and scalability, with no loss in assessment reliability.

E-portfolio-based project assessments have been common place in these sectors and offer several advantages over paper-based projects:

  • Significantly increased portability, coupled with a significantly reduction to the risk of loss of work (over a paper-based portfolio);
  • Intuitive support for on-going, synoptic based assessment, so formative and summative – to help drive up the quality of teaching and avoid teaching to a single assessment;
  • Increased, more transparent tracking of assessment standards and reliability, leading to less dependency on full sample moderation;
  • Reduction in time taken to undertake a summative assessment gateway;
  • Increased involvement of key professional stakeholders in the assessment process – including employers;
  • Reduction in workload and costs for Assessment Organisations, without eroding assessment reliability.

A significant number of Assessment Organisations, including those focused primarily on General Qualifications, now utilise robust, scalable e-assessment technologies to deliver core skills assessments in wide variety of vocational and professional contexts. The use of such approaches is popular with schools, colleges, training providers, employers and most importantly learners, as such approaches, particularly e-portfolios, offer increased assessment validity and transparency of progression, with no negative impact on operational cost or assessment reliability.

Whilst we recognise that in the past, Ofqual has been unwilling to specify how assessment is delivered, leaving that decision to the Assessment Organisations operating professionally within the market. However, for T-Levels, with monopoly supply, and complex regulatory arrangements, if Ofqual is not pushing for innovation in assessment, through the use of e-assessment then it is very unlikely to occur, and learners will miss out on valid assessment of their core skills and Occupational Specialisms related knowledge. The Government’s own ETAG Report of 2016 states that “Ofqual should actively support and encourage the Awarding Bodies to develop digital technology-enabled assessment” and the development of a new qualification, with relatively low entry numbers, provides the ideal opportunity for this to be implemented.

We would therefore urge Ofqual to move away from a model that mandates all practical assessments being delivered purely by Assessment Organisations, and to more proactively encourage the use of innovative e-assessment based approaches to the assessment of core skills and Occupational Specialisms.

Q52: We have not identified any ways in which our proposals will prevent innovation by awarding organisations offering Technical Qualifications. Do you have any comments on this assessment? Please provide specific examples.

Whilst we recognise that the decision to award a single licence to an Awarding Organisation within each T-Level ‘route’ was not of Ofqual’s design, it should be noted that we feel that this approach to T-Levels will significantly reduce the investment in better assessment with the aid of technology by awarding organisations.

The assessment process has been significantly improved by the use of technology, very little progress has been made by General Qualification providers who regularly make headlines for poor quality processes and marking mistakes. The opportunity to continue improving assessment will be lost with a single licence approach.

The General Qualification sector has been for some time experiencing difficulties in recruiting markers in sufficient numbers. Placing additional marking pressures on the education sector at an already difficult time of year will only lead to poorer quality marking, inconsistency and delay.

As noted elsewhere within our response to this consultation, e-assessment based approaches present many opportunities for better assessment. By restricting the opportunities to use e-assessment, the market will lose the following opportunities:

  • Adaptive Assessment can only be delivered in a digital context. Adaptive assessments can often result in shorter more focused and personal assessments, perfect for the vocational and technical sector. In addition, Adaptive assessment can provide you with greater detail on the performance of the whole cohort and not just the top and bottom candidates;
  • Technology is opening the whole to many members of the community who have traditionally been denied access to many services and opportunities in education and employment. E-assessment is the answer to providing inclusive learning and assessment. By inadvertently blocking the use of e-assessment, there is a significant risk of turning back the clock on an inclusive education system;
  • E-assessment presents a richer environment to test in. Vocational and technical subjects are by their nature a dynamic environment, significantly more so than the knowledge recall that is expected in General Qualification assessments. With e-assessment, assessments can be designed to dynamically contextualise environments and provide candidates with simulated/real visual experiences;
  • A candidate focused on exemplifying skills and occupational specialisms will not have the same approach to academic study and therefore examination preparation will be an usual activity form them. By deploying a traditional examination process for T-Levels, candidates will be required to spend large amount of time, normally dedicated to learning their subject, preparing for the process of assessment thus undermining, rather than enhancing learning.

Finally, we feel that Ofqual must take account of the fact that they will in effect be regulating a monopoly supplier of each T-Level. Market forces work differently in this setting than for existing general and vocational/workplace qualifications and in this respect there is strong evidence that:

  • Suppliers are unlikely to innovate once the T-Level contract is won – they will focus on their contract arrangements with their sponsor (DfE and IFA) rather than on delivering outstanding assessment;
  • Monopoly suppliers of assessment, regulated by Ofqual, have been conspicuously slow to innovate (Wales, Scotland and Northern Ireland are all implementing e-testing in KS1-3 assessments, meanwhile STA, regulated by Ofqual, have undertaken almost no innovation in this area, despite the evidence of very considerable benefits that are beginning to emerge).

We would therefore urge Ofqual to use their regulatory framework to encourage the active use of e-assessment approaches that support innovation within assessment delivery. In so doing, we hope that this will mitigate as many of the implicit disincentives for Awarding Organisations to adopt innovative e-assessment based approaches within the single licence to practice model being advocated for the T-Levels.

Resources categories

Upcoming Events

Quicklinks