Royal College of Surgeons in Ireland
Browse

The utility of high-stakes technical performance assessments in surgical training: establishing a framework across the continuum from selection to certification

thesis
posted on 2025-01-29, 11:20 authored by Conor Toale

The traditional Halstedian of apprenticeship training in surgery has undergone significant revolution in recent decades 1. Evolving patient expectations regarding the role of surgical trainees in their care2, an increased emphasis on theatre efficiency 3, and well explored concerns regarding perceived operative competence and confidence of graduating trainees 4-6, has led to a re-evaluation of the training paradigm. ‘Competency-based’ approaches to outcome-driven training and assessment are well established across training jurisdictions7 8, leading to the development of nominally time-independent postgraduate surgical programs 9. The surgical specialties pose a unique challenge in this regard, due to the requirement for robust and reliable methods of teaching and assessing competence in operative skill 10. Determinations of competence, in the context of the surgical training programme of Ireland and the United Kingdom, are made at Annual Review of Competence Progression (ARCP) meetings and are informed by performance in workplace-based assessments (WBAs), operative numbers (as recorded by trainees in an electronic logbook), and educational supervisor reports.  From August of 2021, surgical training in this jurisdiction has moved to an explicitly outcome-basedcurriculum11. Trainees are measured against high level outcomes-‘Capabilities in Practice’ and ‘Generic Professional Capabilities’. This training programme is nominally time-independent11. In order to transition to a truly competency-based paradigm, operative skill will need to be assessed and certified through an objective, standardised, and valid approach. 

The role of simulation in surgery has evolved in tandem with competency-based education12.  Simulation is commonplace in surgical training curricula 13, and is increasingly being deployed as a method of assessing a trainee’s competence 14 15. Studies have demonstrated the safe transfer of simulator-acquired operative skill to the operating theatre 16. Whether simulation-training ultimately translates to improved patient safety and surgical outcome measures has not yet been firmly established 17. Simulation has most notably been used in high-stake assessments through the implementation of the Colorectal Objective Structured Assessment of Technical Skill by the American Board of Colon and Rectal Surgery14 18. Such assessments have not been used for higher specialist training in the UK and Ireland to date. Beyond end-of-training certification, simulation (or other technical performance assessments) could be used as a measure of skill or competence at a number of key checkpoints or high-stakes scenarios in surgery, namely: trainee selection, progression through training, and autonomy granting. 

This thesis will seek to explore the use of high-stakes technical performance assessments of operative skill and competence in surgical training. The validity, acceptability and reliability of technical performance assessments will be explored, using both quantitative and qualitative methods, at three key time points: trainee selection, in-training progression, and competence certification in senior training. For the purpose of this thesis, ‘performance assessments’ are defined as assessments that take place outside of the workplace context, in comparison to ‘workplace-based assessments’. The validity of psychomotor assessments of fundamental technical aptitude will be examined at entry to Core Surgical Training. The validity of simulation-based assessment will be assessed during Core Surgical Training. Finally, we will seek to develop and validate a simulation-based assessment curriculum suitable for the assessment of operative competence in senior Higher Specialist trainees in General Surgery. 

Findings from this work will be used to inform a blueprint, or proposed framework, regarding the use of technical performance assessments in high stakes scenarios surgical training, from trainee selection to certification as an independent practitioner. 

History

First Supervisor

Mr. Dara Kavanagh

Second Supervisor

Dr. Marie Morris

Comments

Submitted for the Award of Doctor of Philosophy to RCSI University of Medicine and Health Sciences, 2024

Published Citation

Toale C,. The utility of high-stakes technical performance assessments in surgical training: establishing a framework across the continuum from selection to certification. [PhD Thesis] Dublin: RCSI University of Medicine and Health Sciences; 2024

Degree Name

  • Doctor of Philosophy (PhD)

Date of award

2024-05-31

Programme

  • Doctor of Philosophy (PhD)

Usage metrics

    Theses and Dissertations

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC