the e-Assessment Association

AI-Assisted Item Grouping and Test Blueprint Development

AI-Assisted Item Grouping and Test Blueprint Development

A blog by Vishwanath Subbanna, Principal Pre-sales Consultant, Excelsoft Technologies 

Test blueprinting is a robust process in the assessment lifecycle that facilitates efficient item grouping and ensures the creation of high-quality, well-balanced tests. It enables you to systematically organize the items based on characteristics such as similarity, meta-data, domain parameters, factors derived from item analytics, and others. This streamlines selecting the appropriate questions and determining the optimal number of questions when creating tests.

A few characteristics used in organizing the items:

  • Similarity: Similarity involves categorizing items that evaluate the same knowledge or skill using comparable content or formats. For instance, it organizes all multiple-choice-single-response, multiple-choice-multiple-response, and fill-in-the-blank items on a particular topic.
  • Meta-data: Organize items based on additional information about them, such as their author, creation date, geography, language, competency, skill, learning outcomes, and learning objectives.
  • Subject/domain parameters: Categorize items by the specific subject area or domain they cover (e.g., Math, Science, History).
  • Item psychometrics: Group items based on item analysis data such as:
    • The difficulty index indicates how difficult an item is for average-ability test-takers.
    • The discrimination index shows how well an item differentiates between high-performing and low-performing test-takers.
    • The guessing factor estimates the likelihood of a test-taker getting an item correct by chance alone.
    • The exposure parameters track the frequency with which an item has been used in past tests to prevent overuse and maintain test security.
    • The aging parameters analyze information currency and remove outdated questions from the pool presented to the test-takers.

Test and Item administrators can leverage item grouping to:

  • Construct item groups: Organize the items logically using the various grouping methods.
  • Develop test blueprints: Define a test’s desired structure and content by specifying various item filters.

Leveraging AI for constructing ability-based, well-balanced, and effective test blueprints:

By generating a set of filters, AI can significantly improve test quality by reducing the manual effort required from SMEs. SMEs can focus more on refining and fine-tuning the AI-generated test blueprints to ensure they precisely align with the desired test objectives and outcomes.

Construct item groups

  • The AI engine processes pre-defined curriculum and semantic content models to categorize and group items for test blueprints.
  • It identifies metadata constraints to meet test objectives and outcomes.
  • AI marks adversary items (enemy items) based on similarity scores, grouping them by difficulty and discrimination index for test construction, distinguishing between practice and high-stakes tests.

Develop test blueprints

  • AI can be used to build test blueprints by providing test objectives and outcomes as prompts.
    • Eg: 1:  Build a Science test blueprint to deliver tests for the students studying in grade 10 having an ability distribution of 40-High difficulty, 30-Average difficulty, and 30-Low difficulty.
    • Eg: 2: Build a Math test to validate the knowledge and comprehension level of the students in grade 9. Ensure to add more questions from the topic of Linear Algebra and Quadratic Equations.
    • Eg: 3: Build a Logical reasoning test for postgraduate students located in the African region.
  • AI can be used to validate the existing blueprints for their effectiveness and generate quick reports.
  • AI can direct test administrators to build a blueprint by asking leading questions.

 

Share this: