Generative AI in Education, UK Government publishes reports

This Month the UK Government published two reports on the insights from teachers, education leaders and pupils on the potential uses of Generative Artificial Intelligence (GenAI) in education.

Commenting on these reports, eAA Board Member, Garrett Sherry said, “These studies from the UK Department for Education are a great step in the journey to realising the possibilities for GenAI to enhance the learning and assessment experience in the education sector. Implemented responsibly, GenAI technology has the potential to improve teaching, learning and assessment as well as enhancing the overall student experience.”

About the reports

GenAI is increasingly finding its place in the educational sector, with the potential to transform routine tasks, enhance learning outcomes, and reduce the workload of educators. The two reports on Use Cases for Generative AI in Education explore insights on the potential uses of GenAI in education and delve into the associated opportunities and challenges.

Hackathons: A Collaborative Exploration of GenAI in Education

Between September 2023 and March 2024, a project was undertaken to explore how GenAI could support teachers, school leaders, and students. The project included hackathons that brought together educators, policy experts and data scientists to brainstorm and test various applications of GenAI in the classroom. These hackathons were instrumental in identifying key use cases for GenAI, ranging from lesson planning and material generation to student assessment and feedback.

One of the most promising outcomes from these events was the development of a Proof of Concept (PoC) tool designed to assist in marking students’ work and providing personalised feedback. This tool aimed to reduce the time teachers spend on grading, allowing them to focus more on creative and interactive aspects of teaching.

Key Findings: Opportunities and Risks

The user research conducted alongside the technical development of the PoC tool revealed several key insights into how GenAI might be effectively utilised in schools:

1. Time-Saving Potential: One of the most significant advantages of GenAI tools is their ability to save teachers time. For example, the PoC tool developed during the project could assess Year 4 students’ writing against the national curriculum, provide detailed feedback and generate revision activities. This automation of routine tasks could allow teachers to invest more time in lesson planning and direct student engagement.
2. Standardisation of Feedback: Teachers expressed optimism about GenAI’s potential to reduce subjectivity in marking. By providing consistent feedback, GenAI tools could help standardise educational assessments across different teachers and schools, ensuring a more uniform evaluation process.
3. Concerns Over Teacher Roles and Relationships: Despite the benefits, there were concerns that the use of GenAI might alter the traditional role of teachers, particularly in providing personalised feedback. Teachers feared that an overreliance on AI could lead to a deskilling of educators and weaken the teacher-student relationship, which is crucial for student motivation and learning. As the e-Assessment Association, we have seen how AI has been used to enhance the role of teachers
4. Risk of Overreliance: The report highlighted the risk of teachers becoming overly dependent on GenAI tools, potentially leading to a decline in their professional judgment and a reduction in the quality of human oversight in the classroom. This was a particular concern for less experienced teachers who might be more inclined to rely on AI-generated feedback.
5. Need for Training and Guidance: To successfully integrate GenAI into education, teachers emphasised the need for adequate training and expert guidance. Without proper support, the adoption of these technologies might be met with resistance, particularly from educators who are less familiar with AI.

Moving Forward: Balancing Innovation and Human Touch

The integration of GenAI in education presents a double-edged sword. While it offers significant opportunities to streamline educational processes and enhance learning, it also brings challenges that need to be carefully managed. The key to successful implementation lies in balancing the efficiency of AI with the irreplaceable human elements of teaching.

The findings from this research underline the importance of developing GenAI tools that complement rather than replace teachers’ roles. By focusing on enhancing, not diminishing, the teacher’s role in the classroom, GenAI can be a powerful tool in modern education.

As educational institutions continue to explore the possibilities of GenAI, it is crucial to keep the focus on empowering educators, preserving the teacher-student relationship, and ensuring that AI tools are used responsibly and effectively. With the right approach, GenAI can help create a more dynamic, efficient, and personalised educational experience for all.

The technical report, Use Cases for Generative AI in Education, delves into the development and testing of a proof-of-concept (PoC) tool designed to assess student work and generate feedback in line with the National Curriculum. This report highlights the potential, challenges, and future implications of integrating GenAI into educational settings.

Aims and Justifications

The PoC tool, developed in collaboration with the National Institute of Teaching (NIoT) and the Department for Education (DfE), was built to explore the practical applications of Large Language Models (LLMs) in education. Specifically, the tool focuses on evaluating Year 4 Literacy work, providing feedback, and generating revision activities. The primary goals were to reduce the time teachers spend on routine tasks, improve consistency in educational assessments, and test the feasibility of using GenAI for these purposes.

Development Phases

The project was structured into three main phases:

1. Minimum Viable Product (MVP) Build: The initial phase involved designing and building the core functionality of the tool. This phase engaged a group of educators to define user requirements and provide early feedback, ensuring the tool’s alignment with educational needs.
2. User Testing: The second phase involved extensive user testing with teachers, focusing on the tool’s performance and usability. Teachers were asked to review feedback generated by the tool for synthetic student work, assessing its alignment with educational standards.
3. Optimisation: The final phase involved running experiments to improve the tool’s performance, focusing on prompt-engineering techniques and incorporating key reference data from the National Curriculum. This phase aimed to refine the tool’s ability to generate informative and accessible feedback that meets educational guidelines.

Key Learnings and Insights

The development and testing of the PoC tool yielded several key insights that could guide future efforts to integrate GenAI into education:

• Engagement with Educators: Involving educators throughout the development process is crucial for ensuring that the tool meets their needs and integrates seamlessly into their workflows.
• Highlighting Strengths and Weaknesses: The tool must focus on both the strengths and weaknesses of student work, providing a balanced perspective that supports student development.
• Customisability: The ability to customise feedback and tasks to suit the specific needs of educators and students is essential for the tool’s effectiveness.
• AI and Human Expertise: Blending deterministic AI approaches with human expertise can improve the accuracy and relevance of feedback, especially when dealing with nuanced educational content.
• Data Privacy and Ethics: Addressing data privacy concerns and ensuring transparency in how student data is used is critical for gaining the trust of parents and schools.

AI Safety and Error Detection

A significant portion of the report focuses on the technical aspects of the tool, particularly error detection and AI safety. The tool uses a combination of LanguageTool (LT) and GPT models to detect and classify errors in student work. The report details the iterative improvements made to the error detection pipeline, which significantly enhanced the tool’s accuracy.

To ensure safety, the tool incorporates guardrails such as content moderation and protection against prompt injection. These measures are especially important in educational settings, where the potential for misuse or unintended consequences is a concern.

Implications for the Future

The report concludes with a set of recommendations for future work in this area. These include further refining the error detection process, expanding the tool’s capabilities to cover more curriculum areas, and exploring ways to integrate Optical Character Recognition (OCR) for processing handwritten student work. Additionally, the report highlights the need for ongoing research and development to optimise the performance of GenAI tools in educational contexts.

Conclusion

These studies on the use of GenAI in education present a strong case for its potential to transform teaching and learning. While there are challenges to overcome, the insights gained from this PoC project provide a solid foundation for the continued development of AI-powered educational tools. By focusing on the needs of educators and students, and ensuring that GenAI complements rather than replaces human expertise, GenAI can play a crucial role in shaping the future of education.

At the e-Assessment Association we have seen how AI has begun to be integrated into learning and assessment across the globe. Initial fears about the negative impact of AI in education have thus far been largely unfounded. Used responsibly, AI has the potential to enhance, rather than diminish, the educational experience, benefiting both teachers and students alike. The winners and finalists of the e-Assessment Association 2023 International e-Assessment Awards provide great examples of how AI has been used to enhance learning and assessment. Read more about these examples here.

Read the full reports here: https://www.gov.uk/government/publications/generative-ai-in-education-user-research-and-technical-report