Calibrating Devolved Quality Activities

Introduction

Many providers devolve quality assurance activities like learning walks, work scrutiny, and professional dialogue to curriculum managers or team leaders. This promotes ownership and local responsiveness, but it also carries a risk: inconsistency. If different managers apply criteria differently, the entire QA system can become unfair and unreliable. How do you ensure rigour and consistency without creating a bureaucratic burden? The answer lies in proactive calibration.

Establish a Shared Quality Framework

Before you can calibrate judgements, everyone must be working from the same playbook. A centrally-imposed checklist is rarely effective; a co-constructed framework built with your team leaders is far more powerful. This ensures buy-in and a common understanding of what constitutes high-quality provision.

• Anchor to the Toolkit: Base your internal framework on the current inspection toolkit. Use headings like 'Curriculum, teaching and training' and 'Participation and development' to build a shared language.
• Use Concrete Examples: Move beyond abstract terms. Instead of 'good pace', define what that looks like in different contexts. Use anonymised examples of learner work or short video clips of teaching practice to anchor discussions about what 'expected standard' or 'strong standard' looks like.
• Keep Criteria Simple: For each QA activity, create clear and concise criteria. A learning walk might focus on three key questions, not a checklist of 20 items. This makes the activity more focused and the calibration process more manageable.

Run Joint Moderation Sessions

The most effective way to align judgements is to make them together. Regular, structured moderation sessions move calibration from a theoretical exercise to a practical, collaborative process. This is a crucial element of whole-provider leadership and governance.

• Moderate Collectively: Schedule meetings where all managers involved in QA bring evidence. This could be a sample of assessed learner work, notes from a learning walk, or a recording of a teaching segment.
• Judge Independently, then Discuss: Ask each manager to apply the agreed criteria independently first. Then, reveal the judgements and facilitate a professional dialogue about the differences. Focus discussions on the evidence, not the person.
• Document the Rationale: When a consensus is reached, briefly document the 'why'. For example: 'We agreed this demonstrated a strong standard for achievement because the learner consistently applied the skill to three unfamiliar contexts.' This builds a bank of 'case law' for future reference.

Use Shadowing and Paired Activities

Calibration isn't a one-off event. It's an ongoing process of professional learning. Building in shadowing and paired activities helps embed consistent practice, especially for new managers or those less confident in their quality roles.

• Pair Up for Consistency: Have two managers conduct a learning walk or work scrutiny in the same area and then compare their findings and feedback. This provides immediate, low-stakes calibration.
• Offer Supportive Shadowing: A central quality lead can shadow a curriculum manager as they carry out their QA duties. The purpose is not to check up on the manager, but to act as a critical friend and offer coaching on their approach and the quality of the feedback they provide to staff.
• Focus on the Feedback: The ultimate goal of QA is improvement. Use these paired sessions to calibrate the quality and impact of the feedback being given to practitioners. Is it developmental, specific, and actionable?

Analyse QA Outcomes for Divergence

Your own QA data is a powerful tool for identifying potential inconsistencies. By analysing the outcomes of devolved activities at a whole-provider level, you can spot patterns that might indicate a need for recalibration.

• Review the Distribution of Judgements: Look at the data from observations or work scrutinies broken down by department or manager. Are there significant outliers? A team where every observation is 'strong standard' may be just as much of a red flag as a team where none are.
• Triangulate with Other Data: Compare the outcomes from your internal QA with other key metrics. If a manager's observation judgements are consistently high but the provision-type's achievement or learner voice data is low, it warrants a supportive conversation about calibration.
• Make it a System-Level Query: Discuss these patterns in leadership and quality meetings. Frame it not as 'Why are your grades different?' but as 'What does this pattern tell us about the consistency of our QA system?'

Where this fits in QualityHero

Ensuring consistency in devolved QA is a cornerstone of a robust quality system. This process of calibration, moderation, and analysis provides leadership teams with confidence in their self-assessment judgements. The Toolkit Areas and Leadership Reports modules in QualityHero are designed for this, allowing you to capture outcomes from all QA activities in one place. You can then easily filter and analyse data by provision-type, manager, or activity to spot inconsistencies and evidence the joint moderation that underpins a fair and impactful quality cycle.