In the second of a new series of Q&As, Dr Mark Charny answers some frequently asked questions on audit in general practice
Q Why does clinical audit always need to involve so much data collection? Wouldn't it be better to concentrate more on improving clinical care?
A It is a common misconception that clinical audit is a euphemism for data collection. Clinical audit describes a quality improvement process that consists of setting a standard or objective, measuring whether this has been met, considering the reasons why it has not been met, and taking action to improve the way that care is given in the future.
How much data collection is involved depends on what is being audited. The general rule is to aim for no more precision than is required to tell you whether the objective has not been met, and give some indication of why this has happened – sufficient only to be reasonably sure that the action you propose to take is warranted and likely to improve the situation in the future.
Another misconception is that audit always requires the collection of data on a specially designed form. Although data are needed, they can come straight from existing medical records.
In some audit designs, very few cases are included and no additional data are gathered. For example, significant event audits collect data only on patients to whom something out of the ordinary has happened, such as: patients who develop cervical cancer within one year of a normal cervical screening smear; patients who wait longer than 24 hours in A&E; maternal deaths; and 'near misses' in the course of surgery.
Here, by definition, the case is included because it is unsatisfactory and, usually, rare. Being rare, few cases need to be included: even one case may be sufficient for an audit. Because there are so few cases, it is usual for those involved in the patient's care to review the casenotes to consider why the situation arose.
In general, the rule is 'horses for courses'. Audit participants should always feel confident that all the data to be collected are needed, and that the number of cases is neither too few (in case a false picture is given) nor too many (where additional effort yields no extra information).
Q Our local acute trust has suggested that we do an 'interface' audit on referrals to general surgery outpatients. The audit, which has been designed by a surgical registrar, involves collecting data on 1500 patients from the 18 practices in our PCG. Is this reasonable?
A Without details of the audit, it is not possible to give a definite answer, but the figure does seem high.
The number of cases to be included in any audit is not as easy to calculate as sample sizes in RCTs, because audit questions are inherently less precise than research questions. Audit is a practical activity, not a theoretical one.
Sample size in audit is driven by:
- The precision of the answers required (usually overestimated)
- The variation in the data (the wider the variation, the more imprecise the results for any given sample size)
- The size of the units being observed (the bigger the unit, the smaller the sample needed: for example, if you compare three PCGs of 20 practices, the sample size needed is not as large as would be needed if you were comparing the 60 practices with one another).
You are right to question the sample size. Ask the surgical registrar to explain the basis of the proposed sample.
Q We have been asked by our PCG to collect a lot of data on CHD, as part of clinical audit and clinical governance, but it looks to us more like management information. Is there a difference?
A This depends on what you mean by 'management information'.
If you mean data that are collected as part of a national initiative, with the intention of allowing the Department of Health to write reports and publish league tables, then there is a big difference.
If you mean data that are used locally by those delivering the care and participating in the audit to understand how they can do better, then there is a very close relationship.
Management is about providing the resources and systems to help people do a good job, and to support people who need to change. These are essential characteristics of effective audit – without them, people cannot be expected to deliver acceptable care.
Although some data might serve both local and national purposes, your question implies that you and your colleagues did not feel that you could relate the data to issues that you felt were of clinical importance.
Common-sense tests are:
- Do the data items seem clinically relevant?
- Does the number of cases included seem appropriate to the question(s) being asked?
- Does the timescale over which the data are to be collected seem reasonable?
- Who will do what with the data and will it help you and the patients?
ìn a publicly funded healthcare system, politicians will inevitably want to measure what is going on. As with many aspects of life, it is about the balance between central requirements and what is needed to deliver and improve the service locally. If the balance seems wrong, discuss it further with your PCG and see whether you can come to a compromise.
Can we solve your problem?
If you have a question or a problem on IT, audit or medico-legal issues that you would like to put to our experts, please contact us by:
Post: Guidelines in Practice, The Chapel, Park View Road, Berkhamsted, Herts HP4 3EY
Fax: 01442 862650
Website: feedback page