Dr Mark Charny advises on how to prioritise topics for audit and the importance of measuring outputs and outcomes
Q With so many things we could look at and serious time pressures, how do we prioritise what we audit?
A Deciding what to audit is a matter of judgement. Sometimes you are obliged, or effectively obliged, to audit an aspect of care. Where you have a choice, you may find the following checklist useful:
- Do you have any reason to believe that there is currently a problem? If not, you might make this less of a priority (other things being equal).
- If there seems to be a problem, does it have significant negative implications for the organisation and/or workload of practice staff, or might it do so in the future? An audit can more than pay for itself if it results in changes that streamline the way care is given, by improving working conditions, reducing the possibility of error, or making it easier to achieve the end result with less effort.
- Are any suspected deficiencies in patient care likely to have any serious clinical or other consequences for patients? Concerns about critical elements of care deserve attention because of the need to reduce the possibility of harm to patients and to improve outcomes – indeed, failure to address these issues could result in litigation or professional disciplinary procedures.
- Is the proposed audit concerned with aspects of care that may be important to patients as ‘customers’? Some aspects of care are more important to patients than an analysis based on clinical harm or benefit might suggest. For example, the arrangements for obtaining repeat prescriptions may be safe, but not user-friendly. As well as being a good thing in itself, a positive relationship between the practice and patients brings benefits in terms of better concordance, fewer complaints, and fewer consultations.
- Is there a consensus on what constitutes good care? Where there is no currently accepted best way of doing things (for example, a national guideline, clear evidence from a Cochrane review, or a consensus among practice staff), audit is unlikely to achieve useful results – unless the audit is explicitly designed to generate consensus where none exists.
- Is someone interested in taking the lead on the proposed audit? Audit without action is no audit at all. Although many people may be involved in the care being audited, action needs to be led and co-ordinated by someone with the time and interest to make sure that things that need to be done get done, and that the momentum does not get lost in the sea of competing priorities.
- Is there a realistic prospect of changing things, if they need to be changed? If the chances of making changes are low, it may not be worth embarking on an audit, even if the issue is important – unless you use the audit results to raise awareness with a view to persuading people who can make the change happen – in which case the objective is really to change colleagues’ views rather than the care. This will depend on the skills of the project leader, how much control the practice has over the problems, and the extent to which those involved agree that there is a problem and are committed to solving it.
- Can change be achieved within a reasonable timescale? If the timescales are long, enthusiasm will be lost, people will move jobs, and other things will change. All this makes running the audit as a project much more difficult.
- Is it easy to give the proposed audit a focus, if it does not already have one? The more general the question, the more difficult it is to organise and effect change. A question like “Is blood pressure recorded in the notes every year for all patients over 65 years of age?” is much easier to answer and do something about than a question like “Are we doing everything we should to reduce coronary heart disease?”
- Can the aspect(s) of care being audited be measured? Not everything that is important is measurable – and, of course, not everything that is measurable is important!
The checklist above provides a basis which you can use to prioritise topics to be audited. It may be useful to generate a list of possible subjects for audit by brainstorming, and then get audit participants to rate each subject on the basis of agreed criteria.
Do this individually if you want to explore differences in view, and particularly if there is a domineering individual in the group. Do it in a group if team dynamics are good and you want to generate consensus.
It is important to be realistic about the time available and to stick to an agreed programme. It is much better to make a few things work than to leave many loose ends.
Q What are the principles of measurement?
A Audit depends on collecting data, and then doing something useful with it. Proper attention to measurement makes it easier to collect data, and makes it more likely that the data are meaningful.
In general, audits measure either process (what is done) or outcome (what happens as a result of what is done) or both. They do not usually measure inputs (what resources are available for doing the job).
Outcome may cover outputs (what happens in the short term, such as an operation completed successfully), and true outcomes (what happens in the long term, such as the treatment resulting in improved quality of life).
Although it is important to remember that care is given in the expectation that it improves outcomes, it is generally more practical to measure outputs than outcomes: outputs occur sooner and are less subject than outcomes to external influences.
Measurements must be explicit (the test is that someone who has never seen the definition before would know exactly what is intended), reproducible (in a given situation, the measurement by the same individual at different times would be the same), unambiguous (in a given situation, the measurement by different individuals would be the same), and valid (they bear a known relationship to what one would really want to measure if one could).
There is extensive literature on measurement, but these questions can be answered – using commonsense, knowledge of the subject, and a very limited pilot – by anyone willing to apply themselves.