AI in Content & Test Development

PSI’s AI test development solution is a fundamental shift in how assessment content is created, validated, managed, and maintained at scale. Rather than applying automation to existing workflows, PSI has built an agentic AI operating layer that captures, structures, and operationalises institutional assessment knowledge (standards, evidence, and expert judgment) across the entire test development lifecycle. Designed specifically for high-stakes credentialing assessment, the solution combines purpose-built AI agents with rigorous human oversight to deliver faster content development without compromising validity, fairness, or defensibility. Subject matter experts (SMEs) remain central as content architects, guiding, reviewing, and refining AI-generated items within a transparent, auditable workflow that strengthens consistency over time. The solution has been proven in a large-scale U.S. insurance licensure programme, where AI-generated items met or exceeded psychometric performance benchmarks compared to human-authored content, while significantly reducing development effort. Beyond efficiency, the system demonstrated how structured institutional knowledge can be reused and strengthened across development cycles, improving quality, scalability, and SME confidence. PSI’s AI test development solution moves AI in assessment from pilot to practice, establishing a scalable, ethical, and defensible model for the future of test content development.

EPAM Systems is a leading global IT service provider with over 61,000 professionals. To ensure our employees' skills meet high business standards, we traditionally created internal training courses and assigned them as mandatory for everyone. However, this approach presented several challenges: experienced specialists spent dozens of hours studying material they already knew, the company couldn't precisely measure the quality of skill acquisition, and the total time spent on training became an incredibly expensive resource. To solve this, we have developed an innovative AI-driven skill evaluation ecosystem. Currently, the project consists of three powerful standalone modules that allow engineers to verify their qualifications in minutes instead of hours: 1. Quiz Workflow for theoretical knowledge. 2. Oral Skill Evaluation Agent that checks depth of understanding through natural dialogue. 3. Practical Task Workflow that creates unique coding challenges to ensure integrity. While these modules are currently used as specialized tools, our ""to-be"" state is to integrate them into a single, seamless platform. This will allow an employee to select a skill and immediately undergo a comprehensive mixed-method evaluation. As a result, specialists will only spend time learning what they don't already know, while the company receives a precise, real-time map of global expertise.

Inteleos certifies healthcare professionals who perform diagnostic imaging and other critical medical procedures. To maintain exam security and validity, they need a constant supply of new exam questions. The experts who write these questions are practising clinicians and specialists who volunteer their limited time. They know their field deeply, but writing exam questions is a very different skill from clinical practice. Creating a good exam question is tough. It must test the right knowledge at the right level, with distractors that reflect genuine clinical misunderstandings rather than obviously incorrect options. A single question can take hours to develop and review. Inteleos introduced Surpass Copilot, AI assistant that drafts questions based on their approved clinical guidelines. Experts review, refine, and approve each item. The AI provides a starting point; clinicians make the decisions. Better use of clinician time means more quality content and stronger certification programmes. Inteleos's 2025 pilot demonstrated that AI-assisted items were accepted at rates comparable to those of traditionally authored items, indicating that this approach enables experts to consistently produce content that meets established standards. SME quotes - “I find Copilot intuitive and straightforward” and “I’m delighted that creation of new questions… has become super easy and enjoyable.”

Chartered Accountants Australia and New Zealand (CA ANZ) trains and certifies professional accountants. Their CA Foundations programme — covering subjects like tax law, auditing, and financial reporting — requires thousands of high-quality exam questions, each precisely aligned to what candidates are expected to learn. Writing these questions traditionally requires specialist accounting academics, takes considerable time, and is expensive. When CA ANZ recently brought CA Foundations in-house after a long-standing university partnership ended, they faced an urgent need to build new question banks across all their subjects. Jai is an AI assistant built specifically to help assessment teams with this challenge. CA ANZ's team uploads their learning materials, syllabi, and study guides into Jai. The system reads, organises, and deeply understands that content. Then, following specifications set by the assessment team — which topics, how many questions, what difficulty levels, what question formats — Jai generates complete exam questions, each grounded in CA ANZ's own materials rather than generic knowledge. Critically, Jai does not replace human judgement. Every generated question is reviewed by CA ANZ's subject matter experts before it can enter a question bank. Jai handles the labour-intensive first draft at scale; the experts make the final call on quality.