Protecting Assessment Validity in FE

In FE and Skills, assessment is the primary way we gauge learner achievement. But what if our assessments are accidentally measuring the wrong things? Assessment validity is simply about ensuring a test, observation, or assignment accurately measures the knowledge, skills, and behaviours it is designed to. Invalid assessment gives a false picture of achievement, misinforms teaching, and can lead to learners progressing without the secure competence they need. Protecting validity is a core quality assurance function.

Spot Construct-Irrelevant Factors

This happens when an assessment result is influenced by factors that are not relevant to the intended skill or knowledge. For example, a learner may fail a practical plumbing assessment not because of poor plumbing skills, but because the written instructions were too complex. This introduces irrelevant barriers to a learner demonstrating their competence.

Practical steps:

• Review assessment tasks for unnecessarily complex language or jargon that is not a core part of the subject.
• Separate the assessment of different skills. If assessing technical knowledge, do not let poor IT skills be the primary barrier to a learner showing what they know.
• Consider if the format of the assessment - for example, a long-form essay - is the best way to measure a practical or vocational skill.
• Ensure any required resources like tools, software, or templates are fully accessible and that learners are familiar with them before the assessment.

Check for Construct Under-representation

This is the opposite problem: the assessment is too narrow and fails to cover the full scope of the curriculum content it is supposed to evaluate. A single multiple-choice quiz on a complex topic is a classic example. It might test factual recall but misses application, analysis, or evaluation, giving an incomplete picture of achievement.

Practical steps:

• Map assessment methods against your curriculum's key knowledge, skills, and behaviours for each provision-type.
• Use a range of assessment methods - for example, practical observation, professional discussion, projects, presentations - to capture a holistic view of a learner's competence.
• Review past papers or assessment banks. Do they consistently focus on the same few topics while ignoring others?
• Ensure assessments cover not just what is easy to test, but what is most important for a learner to know and be able to do.

Standardise Your Assessment Administration

Inconsistent administration can be a major threat to validity. If one group of learners gets clearer instructions, more time, or a quieter environment than another, their results cannot be reliably compared. This is a critical issue for fairness, inclusion, and the overall trustworthiness of your achievement data.

Practical steps:

• Create simple, clear checklists for staff administering assessments, covering key instructions, timings, and environmental conditions.
• For practical observations, ensure all assessors are clear on the parameters of their interaction with the learner during the task.
• Provide identical, unambiguous instructions to all learners. Read them aloud and check for understanding before the assessment begins.
• Ensure all learners, including those with reasonable adjustments, are given the conditions they need to perform at their best.

Strengthen Scoring and Moderation

Even with a well-designed task, subjectivity in marking can undermine validity. An assessor's personal bias, or even fatigue, can lead to inconsistent scoring between learners or across different assessors. Robust standardisation and moderation are the essential safeguards against this.

Practical steps:

• Run pre-marking standardisation meetings where assessors mark sample work together and agree on the application of the mark scheme.
• Conduct blind double-marking on a sample of assessments to check for consistency between markers and identify any divergence.
• During internal moderation, focus the dialogue on why a particular mark was given, referencing specific evidence against the criteria.
• Look for patterns in marking, such as a single assessor's marks being consistently higher or lower than the team average, and use this as a basis for supportive professional dialogue.

Where this fits in QualityHero

Robust assessment is fundamental to evaluating learner 'Achievement' and the impact of 'Curriculum, teaching and training'. The evidence from your validity checks, standardisation records, and moderation meetings can be logged in the Toolkit Areas module. Findings from these processes can inform your provision-type self-assessment in the SAR module and generate targeted improvement actions in your QIP.