The impact of EU AI Act on e-assessment

The European Union has introduced new legislation on artificial intelligence, the EU AI Act. It lays the foundations for the regulation of AI in the EU. 

The majority of obligations of the act will apply to providers (developers) of systems deemed ‘high risk’ which are: 

• Those that intend to place on the market or put into service high-risk AI systems in the EU, regardless of whether they are based in the EU or a third country. 

• Third country providers where the high risk AI system’s output is used in the EU. 

Those who deploy AI systems in a professional capacity are defined as ‘users’ and also have some obligations, though less than providers. 

What is a ‘high risk’ system? 

Annex III includes a list of systems considered to be ‘High Risk’ under the act and subject to additional requirements. Two specific categories which are relevant to organisations in e-Assessment are: 

• Education and vocational training: AI systems determining access, admission or assignment to educational and vocational training institutions at all levels. Evaluating learning outcomes, including those used to steer the student’s learning process. Assessing the appropriate level of education for an individual. Monitoring and detecting prohibited student behaviour during tests. 

• Employment, workers management and access to self-employment: AI systems used for recruitment or selection, particularly targeted job ads, analysing and filtering applications, and evaluating candidates. Promotion and termination of contracts, allocating tasks based on personality traits or characteristics and behaviour, and monitoring and evaluating performance. 

Exceptions include systems which: 

• perform a narrow procedural task; 

• improve the result of a previously completed human activity; 

• detect decision-making patterns or deviations from prior decision-making patterns and is not meant to replace or influence the previously completed human assessment without proper human review; or 

• perform a preparatory task to an assessment relevant for the purpose of the use cases listed in Annex III. 

High risk AI providers must: 

• Establish a risk management system throughout the high risk AI system’s lifecycle; 

• Conduct data governance, ensuring that training, validation and testing datasets are relevant, sufficiently representative and, to the best extent possible, free of errors and complete according to the intended purpose. 

• Draw up technical documentation to demonstrate compliance and provide authorities with the information to assess that compliance. 

• Design their high risk AI system for record-keeping to enable it to automatically record events relevant for identifying national level risks and substantial modifications throughout the system’s lifecycle. 

• Provide instructions for use to downstream deployers to enable the latter’s compliance. 

• Design their high risk AI system to allow deployers to implement human oversight. 

• Design their high risk AI system to achieve appropriate levels of accuracy, robustness, and cybersecurity. 

• Establish a quality management system to ensure compliance. 

Providers whose AI system falls under the use cases in Annex III but believes it is not high-risk must document such an assessment before placing it on the market or putting it into service. 

When will the act come into force? 

On 12 July 2024 the EU AI Act was published in the Official Journal of the European Union. It will come into force on 1st August (20 days after publication) with  key implementation milestones between 6 months and 36 months after this date. 

Find out more 

Read the high level summary of the act here: https://artificialintelligenceact.eu/high-level-summary/ 

Use the EU AI Act Compliance Checker to determine whether or not your AI system will be subject to these.   https://artificialintelligenceact.eu/assessment/eu-ai-act-compliance-checker/ 

Use the AI Act Explorer enables you to explore the contents of the Act in an intuitive way, or search for parts that are most relevant to you  https://artificialintelligenceact.eu/ai-act-explorer/ 

See the overview of the implementation timetable here: https://artificialintelligenceact.eu/developments/