Supervised learning events: direct observation of procedural skills pilot.

BACKGROUND: Workplace-based assessment (WBA) is the assessment of specialist competence based on what a trainee doctor actually does in the workplace. Between January 2014 and January 2015, all UK occupational medicine (OM) trainees were invited to test a suite of direct observation of procedural skills (DOPS) tools designed in a supervised learning event (SLE) format. The Faculty of Occupational Medicine (FOM) Workplace-Based Assessment Advisory Group (WBAAG) studied feedback on the new format.

AIMS: To assess the utility of the redesigned tools, including their acceptability, feasibility, usability and key aspects of their reliability and validity.

METHODS: The face and content validity of the new forms were assessed by a comprehension trial (CT), inter-rater reliability by a video scoring exercise and usability and acceptability by an electronic survey of trainees and trainers.

RESULTS: The CT of trainees and trainers indicated that the face and content validity of the revised tools were acceptable. Inter-rater reliability video assessments indicated there was consistency of grading among trainers. Sixty-eight per cent of trainees and 95% of trainers agreed that the redesigned tools were an improvement on the current WBA DOPS tools and 83% of trainees indicated the new tools encouraged them to reflect on their performance.

CONCLUSIONS: The results from this pilot study provided evidence to support a request to the General Medical Council (GMC) for the new SLE-DOPS forms to be used for WBA in OM. These changes were accepted by the GMC in January 2016 for implementation in April 2016.