We have located links that may give you full text access.
Exploring subjectivity in competency-based assessment judgements of assessors.
AIM: The aim of this research was to measure variations in assessors' judgements of a student dietitian's performance and to explore the influence of group discussion on their judgements.
METHODS: The assessments of a student's performance, as observed from a video recording of an authentic nutrition consultation, were measured pre- and post-group discussion by 26 experienced assessors using a mixed-methods questionnaire. The instrument included a validated 7-point visual analogue scale (VAS) rating (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). Scales were analysed descriptively and qualitative responses coded for key themes.
RESULTS: No agreement was found in assessors' rating in either the pretest (median = 4, range = 5) or post-test (median = 4, range = 4); however, the discussion led 78% of participants (20/26) to change their VAS ratings (9/26) and/or confidence levels (16/26). Three themes emerged from the thematic analysis of the participants' global descriptions of performance: (i) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (ii) discourse leads assessors to more holistic judgements; and (iii) multiple sources of evidence and student reflections are necessary for credible judgement.
CONCLUSIONS: This research questions the notion that 'actual' performance can be objectively measured and, rather, considers assessments as 'interpretations'. This research calls for an integrated interpretivist student-centred approach to competency-based assessment.
METHODS: The assessments of a student's performance, as observed from a video recording of an authentic nutrition consultation, were measured pre- and post-group discussion by 26 experienced assessors using a mixed-methods questionnaire. The instrument included a validated 7-point visual analogue scale (VAS) rating (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). Scales were analysed descriptively and qualitative responses coded for key themes.
RESULTS: No agreement was found in assessors' rating in either the pretest (median = 4, range = 5) or post-test (median = 4, range = 4); however, the discussion led 78% of participants (20/26) to change their VAS ratings (9/26) and/or confidence levels (16/26). Three themes emerged from the thematic analysis of the participants' global descriptions of performance: (i) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (ii) discourse leads assessors to more holistic judgements; and (iii) multiple sources of evidence and student reflections are necessary for credible judgement.
CONCLUSIONS: This research questions the notion that 'actual' performance can be objectively measured and, rather, considers assessments as 'interpretations'. This research calls for an integrated interpretivist student-centred approach to competency-based assessment.
Full text links
Related Resources
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app
All material on this website is protected by copyright, Copyright © 1994-2024 by WebMD LLC.
This website also contains material copyrighted by 3rd parties.
By using this service, you agree to our terms of use and privacy policy.
Your Privacy Choices
You can now claim free CME credits for this literature searchClaim now
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app