Determining the effectiveness of educational interventions in paediatric emergency care.

OBJECTIVES & BACKGROUND: Demonstrating educational interventions improve patient care can been difficult. This study aimed to perform an extensive assessment of a bespoke e-learning package on the NICE Feverish illness in Children Guideline to aid the development of an evaluation model.

METHODS: An e-learning package was designed featuring an interactive traffic light table page. For each of the 5 domains and the three risk categories there was a video describing the features within that domain. The e-learning package also contained a pre- and post- learning section consisting of a demographic questionnaire, an attitudinal survey and knowledge testing. The weblink to the e-learning package was distributed to 12 Emergency Departments throughout England in August 2012.

RESULTS: A total of 202 junior doctors expressed an interest in undertaking the intervention, 115 (56.9%) commenced the package, 91 (45.3%) completed the pre-learning and 58 (28.7%) completed the post-learning.There was an average 1 point improvement (scale 1-10) in perceived safety, confidence and competence in relation to management of febrile children after the package had been completed (p<0.0001). The Knowledge Test scores improved significantly in MCQs (Z -3.942, p<0.001). ROLMA matrices, a previously published methodology of describing adherence to evidence based practice, were used to examine case-notes of children with undifferentiated febrile illness at the lead site (n=48 patient pre-intervention and n=20 patients post-learning). There was a trend to improved evidence based management in the post learning group but this was not significant (Chi-Square 2.776, p=0.096). Overall data in the lead department demonstrated a rise in admissions of children with undifferentiated fever compared to 2011 but less than the 5 previous years despite a rise in presentations over that time.In respect of satisfaction and success with the e-learning system 83.6% (56/67) participants agreed or strongly agreed.

CONCLUSION: It is important any educational intervention is thoroughly examined as positive findings in respect of satisfaction or knowledge gain may not equate to behaviour change. The continued poor assessment of educational interventions results in potentially continued poor practice or educational technique. A new framework for evaluation (the 7Is) will be presented as a solution.