The audit component of summative assessment provides an opportunity for GP registrars to learn an important skill for general practice. Dr Russell Steele explains how to get it right
The audit component of summative assessment is the training ground for a core part of GP life.
Audit is the method by which most GPs improve their standards and increase their income while still doing things that are relevant to everyday practice. It can be stimulating and demanding, and can introduce a competitive element between partners and practices, preventing stagnation and improving job satisfaction.
It is possible to pass the MRCGP or even win a research prize without demonstrating audit skills. However, GP registrars must show an understanding of audit to pass summative assessment. Some, who otherwise do well, fail the audit component simply because they have ignored, or not sought, advice on the subject.
Audit or review?
An audit should not be confused with a review. For example, a PCT may ask for figures that show how many patients with raised cholesterol are on statins, but if it does not ask for quality markers such as how many of them should be on statins this is not an audit. Rather it is a collection of facts with an implication that it is relevant to quality – a review.
Criteria and standards
The first commandment for GP registrars completing the summative assessment audit should be to use the headings given in the summative assessment guidance.
Audits in general practice should be about quality and improving quality. Quality cannot be assessed until the item to be measured has been defined. This is the criterion.
Audit criteria should be SMART:
- Realistic (Relevant)
- Time limited
For example, the criterion ‘patients with controlled hypertension will have their blood pressure checked’ is not SMART enough. A better one would be ‘patients with controlled hypertension will have their blood pressure checked at least once every 6 months’. Introducing the time element makes it more relevant.
The choice of criterion should be explained, even if it’s only ‘because it seems a good idea’ although it is helpful to justify your choice. References are not essential, but it helps to use them where possible because they often contain details that can justify both criteria and standards.
Once the criterion has been agreed the target should be set. The target, or success rate, is the standard for the audit.
Setting a target means deciding what success rate is achievable now, what may be possible in the future, and what is desirable. It may be desirable to achieve 100%, but this is not usually possible. On the other hand, a 10% success rate is not desirable either as it implies poor quality.
The standard should be set at a level that is challenging but not unreasonable. It could be quite high if the practice has a programme running already, but lower if the team has decided that the topic is worth working on, and wants to get better at it.
The reason for choosing a standard should be justified. Sometimes the justification is that the partners feel it is a standard they want to achieve, with no reference to the literature. However, if no-one has any idea about what standard to set, references may help by showing what other practices have achieved.
It is a good idea to involve the whole team when setting both criteria and standards.
The audit cycle
Effective medical audit is a cycle, or rather a spiral because each time round the cycle the standards are increased (Figure 1 below) unless the initial standards are set too high to achieve.
|Figure 1: The audit cycle/spiral|
Collecting data means finding out the current situation. This usually involves counting something, such as how many patients in the practice are on treatment for hypertension and how many of these had their blood pressure checked in the previous 6 months.
Analysing the results is a way of answering the questions, "How well did we do?” or, "Did we meet our target (standard)?”
If the standard has not been met, it’s time to ask why and consider how it might be reached next time round. It does not matter if it has not been met as long as plans are made to improve in the next cycle.
Planning change when the standard has not been met involves finding out why it has not been achieved. Often it is because the standard was set too high, or because there was a problem agreeing the details of the audit.
Other questions to ask may include:
- Are all those who need to be involved actually involved?
- How can they become more involved?
- Who has responsibility for taking action?
- Would putting reminders in notes help?
- Does the protocol need changing?
- Is each step of the audit clear?
If the standard has been reached, the next stage is to think about how to improve. The standard should be raised for the next cycle and ideas for improvement discussed in the same way as when standards are not met.
After the changes have been discussed and agreed, the spiral continues with more data gathering, more analysis and more planning of change. Quality improves and audit becomes a comfortable part of everyday general practice.
How to pass summative assessment audit
A group of audit assessors in the South Western region found there were several common mistakes in the audits they assess. The advice in Box 1 (below) should help you avoid these errors.
|Box 1: How to carry out a successful audit|
Together with the two commandments, following this advice will help you reduce your chances of failing or having to resubmit your summative assessment audit.