UK Junior Doctors’ Experience of Clinical Audit in the Foundation Programme

Andrew Cai, John Greenall and Dau Col Dau Ding

INTRODUCTION

The Foundation Programme^1,2,3 is a 2-year, ubiquitous, vocational curriculum undertaken by newly qualified doctors wishing to proceed onto speciality training in the United Kingdom (UK). Since 2006, Foundation Year Trainees in the UK have been required to complete one clinical audit during their two year programme. We review the practice of audit and doctors’ attitudes to the difficulty in performing audits at a National Health Service (NHS) hospital trust comprising three hospital sites in the South East of England. The Foundation Programme demands that Foundation Year Trainees are able to consider the relevance of clinical audit and describe the audit cycle with regard to developing patient care, clinical governance and risk management. They are expected to undertake a clinical audit and recognize how it relates to the improving clinical standards and addressing clinical governance¹. Clinical audit can be defined as the processof reviewing the delivery of care to identify deficiencies sothat they may be remedied⁴. Whilst it was initially used in assessing medicalpractice against local standards, audit ‘has evolved conceptuallyas a mechanism through which evidence-based guidelines can beintroduced into routine clinical practice’⁵. Apart from fulfilling the requirements of the syllabus, reasons for audit include professional education and the opportunity to improve patient care⁶. Barriers to audit might include: disagreement amongst professionals as to what constitutes a good audit⁵;organisational impediments; and a lack of resouces⁶. This study therefore sets out to investigate the level of audit activity in a hospital trust in South East England amongst all Foundation Year Trainees. Importantly it will also assess doctors’ attitudes and views towards the audit process and perceived or actual barriers to their completion. METHOD Questionnaires were sent to all Foundation Year 1 (F1s = 63 in total) and Foundation Year 2 (F2s = 56 in total) Trainees in the trust (119 doctors). The study group involved trainees in the Foundation Programme from 31^st July 2007 to 30^th July 2008. Doctors who had been transferred out of the trust were not included in the study. There were no doctors who had transferred into the trust and were in the Foundation Programme. A study representative at each of the 3 hospital sites was tasked to distribute the questionnaires. Trainees were asked to complete the questionnaires in an informal setting and to return them directly to the site representative. The study environment was variable, and questionnaires were distributed and completed on the wards or at group teaching sessions. Participants were given the choice of completing and submitting their form immediately, or submitting it at a later date. Data collection was commenced 11 months after the trainees had commenced employment in the trust and concluded after 2 weeks. This was invoked as many trainees were clearing annual-leave requirements towards the end of their hospital posting, and the consensus that very few audits would be officially completed at that stage of training in the summer. Questions were drawn from previous studies to the barriers to audit in our Trust. In the first section of the questionnaire, participants were asked about: “the number of all audits attempted or applied for”; “the number of new audits attempted or applied for”; “the number of audits completed and presented so far”; and “the number of audits started but never completed”. The second part of the questionnaire assessed subjective opinions on barriers to completing audits. Participants were asked to rate the following 5 statements on a comparative scale of 1-5 (1 being “strongly disagree” and 5 being “strongly agree”): “The audit department is helpful in approving audits”; “senior staff are helpful in involving me in audits”; “I can complete audits within official working hours”; “most audit opportunities are in my area of interest”; “most audit opportunities are of clinical value”. Results were collated and tabulated and presented at local meetings where feedback was received. RESULTS Ninety-two out of a possible 119 (77.3%) Foundation Year Trainees completed the questionnaire (57/63 - F1s, 35/56 - F2s). There were 106 total attempts at audit for the F1 trainees and 65 total attempts for the F2s. Most trainees had attempted 1 or 2 audits in their respective year (42 F1s at 73.7% and 23 F2s at 65.7%). 5 F1s (8.8%) and 3 F2s (8.6%) had neither attempted nor applied for any audits.  Ten F1s (17.5%) and 9 F2s (25.7%) had attempted more than 2 audits (Table 1). Table 1: Number of audits attempted by trainees

 The results for the total number of completed audits (i.e. an audit that included data collection, analysis and formal presentation to the respective department) are summarized in Table 2. For F1s, 32 out of a total 106 attempted audits were completed (30.2%), this percentage rising for F2s (38/65; 58.5%). Thirty-three (57.9%) F1s and 10 F2s (28.6%) failed to complete any audit, with a number able to complete one audit presentation in the year: 18 F1s (31.6%) and 16 F2s (45.7%). Table 2: Number of audits completed by trainees

 With respect to new and original audits attempted by trainees, this was achieved by 66.7% of F1s and 74.3% of F2s (Table 3). There was no formal data on the number of audit loops being closed. Table 3: Number of new audits designed by trainees

  With regard to barriers to completion of audits (Table 4), results were notably equivocal for “helpfulness of the audit department and senior staff” (both averaging 3.1 on the comparative scale of 1-5), and “the clinical value of the audits available” (mean score 3.2). The mean score for “completing audits within official hours” was 2.1 with a similar trend observed in “the audits available in an area of interest” (mean score 2.6). Table 4: Trainees’ experiences with audit

    ¶Key:   1= strongly disagree;    2=disagree;    3 = equivocal;    4 = agree;    5 = strongly agreeNB: Some forms were incomplete, and therefore responses may not add up to 92. CONCLUSIONS Although audit is well established to be beneficial in improving clinical practice⁷, this study suggests that trainees under-perform against the curriculum of the Foundation Programme. Historically, the level of audit activity amongst doctors has been low; for example, McCarthy (1997) demonstrated that whilst doctors see the conceptual value of audit, approximately one-third only had presented their data at a pertinent audit meeting⁸. These results have been replicated in numerous other studies^9,10,11. We believe that this data-set is the first available for junior trainees who have undertaken the Foundation Programme curriculum, with a good response rate of 77.3%, and incorporates the contractual pressures invoked by a European Working Time Directive (EWTD)-compliant Rota^12,13. While the results show that the majority of respondents (>90%) had attempted an audit, most significantly the majority of audits that were started were not completed. A large percentage of F1s (57.9%) and F2s (28.6%) failed to complete an audit at all. Similar numbers have been reported, even among senior pediatric trainees at registrar level, where one study demonstrated that whilst audit activity was above 90%, only 16% had completed the audit cycle¹⁴. One possible explanation is that many trainees appear to have a sub-optimal comprehension about audit and its process. Our consensus was that some trainees attempted audits that were too large or unmanageable, or even of insufficient quality, in striving to achieve a peer publication from their work. When realized that the publication value is poor, or that the audit design is flawed, many trainees lose interest and fail to complete. Another concept highlighted by this study is confusion over the definition of a “completed audit”. For consideration of completion of an audit, a trainee has to demonstrate both the ability to collect the data and present it to among his peers in a formal meeting. This generally amounts to completion of 5 out of the 6 stages of the audit loop¹⁵. Surgical morbidity and mortality presentations had been considered audit by some trainees, as they were termed by the trust as a “surgical audit”. However, the overall clinical consensus is that they are not audit but formative educational meetings because no systemic local or national standards were employed for comparison. This poor understanding of audit has been well described previously¹⁶. Potential barriers to the completion of audit include some of the issues raised in this study. In this sample, doctors were equivocal about whether the barrier was the audit department or lack of senior support. This reflected the variability of experience as well as the lack of teaching of the purpose and methods of audit in the undergraduate curriculum. They were also equivocal about the clinical value of audits they had completed. By comparison, a study in Leeds showed that less than half of the 232 respondents were aware of subsequent change in clinical practice and 27% felt it was “a waste of time”⁷. However this study did not focus on the junior doctor in the beginnings of their postgraduate training. Trainees felt that an additional barrier to audit completion included difficulty in completing audits within their working hours. All Foundation Year Trainees in the trust were working to a EWTD-compliant Rota during the year, where trainees did not exceed 48 hours a week of on-site hospital clinical duties. Trainees also found it difficult to undertake audits in their area of clinical interest. Although part of the reason is circumstantial - the Foundation Year Programme mandates that trainees rotate around various core specialties - this may also reflect a lack of understanding of what the audit cycle actually incorporates, and how it is not formal research in itself¹⁵. Approval of audit studies was also thought to be problematic because such meetings only took place monthly with a pre-determined agenda, and consequently, this meant that approval might take several months to obtain for trainees who would actually be based in the trust for no more than 12 months in 3 different specialty departments. There were a number of limitations of the study, one being the small sample size. Secondly, in asking trainees to rate each of the six statements from 1 to 5, trainees who did not complete audits tended to score 3 (neither agree or disagree), and as the results above show, they represented a considerable proportion. A larger sample size and a semantic differential scale (rating responses between 1 and 7) might have been more discerning. The fact that some trainees may have included “audits” which on reflection did not meet the criteria for inclusion was not only interesting but may also have distorted results. Finally, audits that involved joint effort among trainees, but were presented only by one of them in the absence of the others were still regarded by some trainees to be “completed and presented” by all of them. This study has highlighted a number of issues which need to be addressed for clinical audits to be successfully completed during the Foundation Programme. The authors believe that poor completion rates are most probably the result of poor understanding of audit. Potential solutions include teaching medical students concepts of audit; giving structured teaching early in the Foundation Programme; instituting regular audit meetings; incorporating audit as part of contracted working hours; defining audit more clearly among trainees and clinical staff and encouraging more cooperation and integrative liaison with the audit department to process audit proposals quickly and efficiently. Additionally, doctors’ contractual pay-bandings should reflect any out-of-hours work undertaken on audits that improve clinical governance for their Trusts. However, in spite of all these considerations, we speculate that because trainees are only in each post for no more than 4 months during their foundation years, and with the restriction of working hours, the expectation of foundation year trainees to have undertaken and properly understood an audit cycle, implemented change and closed the audit loop is unrealistic. It would be more helpful to the trusts and trainees for audits to be part of the specialty training programme onwards, where trainees stay in a department for a longer time even as they move from one team to another. Further studies might consider in detail the difficulties in each step of the audit cycle¹⁵ and explore: Foundation Trainees’ use of the audit department; guidance from senior members of staff; and perceived benefits in clinical practice. Ultimately, audits must implement change¹⁷ and all truly successful clinical audits should aid in some way to achieving our fundamental goal in medicine; that being the best clinical practice and best quality of care.

References

1) The Foundation Programme Curriculum. AoMRC; 2007.

2) Dolphin T. Foundation programme assessment: the trainee's guide. Br J Hosp Med. 2006;67(4):M64-6.

3) Velineni R. Foundation year for newly qualified doctors: a house officer writes. BMJ. 2005;24:331(7518):697

4) Crombie IK, Davies HTO, Abraham SCS, du V Florey C. The Audit Handbook: Improving Health Care through Clinical Audit. Chichester: John Wiley; 1993.

5) Miles A, Bentley P, Polychronis A, Price N, Grey J. Goals and Methods of Audit should be reappraised. BMJ. 1996;313:497.

6) Johnston G, Crombie IK, Alder EM, Davies HTO, Millard A. Reviewing audit: barriers and facilitating factors for effective clinical audit. Quality in Health Care. 2000;9: 23-36.

7) Greenwood JP, Lindsay SJ, Batin PD, Robinson MB. Junior doctors and clinical audit. J R Coll Physicians Lond. 1997;31(6):648-51.

8) McCarthy MJ, Byrne GJ. Surgical audit: the junior doctors' viewpoint J R Coll Surg Edinb. 1997;42(5): 317-8.

9) Tabendeh H, Thompson GM. Auditing ophthalmology audits. Eye. 1995;9(supp): 1 -5.

10) Hart. What can audit tell us? BMJ. 1993;306: 480.

11) Parfrey PS, Gillespie M, McMannomaon PJ, Fisher R. Audit of the medical audit committee. Can Med Assoc J. 1986;135: 205 -8.

12) Cairns H, Hendry B, Leather A, Moxham J. Outcomes of the European Working Time Directive. BMJ.2008;337: a942.

13) Flowerdew A. European Working Time Directive: Trusts are already developing solutions to directive. BMJ. 2008;337: a1586.

14) McGlone L. Audit activity of trainees in the West of Scotland. Arch. Dis. Child. 2004;89: 1177.

15) Gnanalingham J, Gnanalingham M, Ganalingham K. An audit of audits: are we completing the cycle? J R Soc Med. 2001;94:288-289.

16) Nettleton J, Ireland A. Junior doctors' views on clinical audit--has anything changed? Int J Health Care Qual Assur Inc Leadersh Health Serv. 2000;13(6-7): 245-53

17) Eccles MP, Deverill M, McColl E, Richardson H. A National Survey of audit activity across the primary-secondary care interface. Quality in Health Care.1996;5: 193-200.

The above article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.