Systemic or Isolated? Investigating a QA Finding

Investigating a Quality Finding: Systemic or Isolated?

A healthy quality system is not one that finds nothing wrong; it is one that effectively identifies areas for improvement. A critical learner comment, a concerning observation, or a dip in achievement data are all valuable starting points. However, a single finding is just that - a single point of data. The immediate challenge for any quality leader is to determine its significance.

Is this an isolated incident, specific to one tutor, learner group, or circumstance? Or is it the tip of an iceberg, indicating a wider, systemic issue that requires a whole-provider response? Acting too broadly on an isolated fault wastes resources, while treating a systemic problem as a one-off allows poor practice to continue. A proportionate, structured investigation is key.

Define the Hypothesis Clearly

Before you begin, translate the initial concern into a clear, testable question. This sharpens your focus and prevents the investigation from becoming a vague and time-consuming search for problems. A well-defined hypothesis makes it easier to know when your investigation is complete.

• Initial finding: An observation report notes that a tutor's questioning did not effectively check understanding for all learners.
• Poor hypothesis: "Is questioning weak across the provider?"
• Better hypothesis: "Is the ineffective checking for understanding seen in the Level 2 Health and Social Care observation typical across that provision-type?"

This specific question defines the what (checking understanding), the who (Level 2 Health and Social Care), and provides a clear success criterion for the investigation.

Plan a Targeted Evidence Trajectory

Resist the urge to immediately deploy your most resource-intensive QA tool, such as a sweep of formal observations. Instead, plan a proportionate trajectory that starts with the easiest-to-gather evidence to quickly prove or disprove your hypothesis.

• Start with existing evidence: Can you review existing documents? For the hypothesis above, you could look at session plans from other tutors in the team. Do they include planned hinge questions or other checks for understanding?
• Move to dialogue: If documents are inconclusive, talk to people. A group discussion with learners from the provision-type or a professional dialogue with the curriculum team can provide rich qualitative evidence quickly.
• Deploy new activity: Only if the issue remains unclear should you commission new, targeted QA. This might be a series of short, focused learning walks looking specifically at questioning, rather than full-blown observations.

Triangulate with Different Evidence Types

To confidently determine if a problem is systemic, you must see evidence of it from multiple, independent angles. Relying on a single source is risky; what looks like a trend in one dataset may be an anomaly when viewed in a wider context.

If work scrutiny suggests progress is slow for apprentices in a particular sector, triangulate this finding:

• Learner Voice: In progress reviews, do these apprentices struggle to articulate the new skills and knowledge they have gained? Do they know what they need to do to improve?
• Employer Feedback: Do workplace mentors report similar concerns about the apprentice's progress or ability to apply learning on the job?
• Achievement Data: Does progress review data align with the work scrutiny findings? Are there any patterns in achievement rates?

If the answer to these questions is yes, you have strong evidence of a systemic issue at the provision-type level.

Distinguish Between Proximate and Root Causes

As you gather evidence, your investigation should seek to understand not just what is happening, but why. Differentiating between the immediate cause and the underlying root cause is crucial for ensuring your improvement actions are effective.

• Proximate Cause: This is the most immediate reason for the issue. For example, a single tutor's poor assessment practice might be due to them being new to the team and unfamiliar with your processes.
• Root Cause: This is the fundamental system or cultural factor that allowed the issue to occur. If multiple tutors are using assessment poorly, the root cause might be a lack of effective standardisation, insufficient professional learning on assessment design, or an ambiguous internal policy.

Addressing a proximate cause fixes one problem. Addressing the root cause prevents it, and similar problems, from happening again.

Where this fits in QualityHero

This process of targeted investigation is central to effective quality improvement, ensuring your QIP is focused on the right problems at the right level. QualityHero supports this by allowing you to connect different pieces of evidence to a single improvement theme in the Toolkit Areas module. You can link learner voice, observations, and meeting notes to build a complete picture, helping you distinguish an isolated event from a systemic trend and create targeted actions in your QIP.