- system itself is located on a secure site at: <www.elogbook.org>. - which is counter to the core values of the system. - Considerable interest from overseas in the orthopaedic curriculum and in particular with the PBA tools has led to a number of proposed international pilot projects.. - International compatibility of surgical training systems is a key issue in relation to making it possible for trainees to complete part of their training overseas but, at a wider level, may have considerable consequences for the mobility of surgical labour. - It is hoped that we will, in the near future have the opportunity to combine the progress made in both the Non-Technical Skills for Surgeons (NOTSS) project (see Chapter 2 in this volume) and in PBAs by either producing a new assessment tool based on the PBA or to integrate behavioural markers from NOTSS into the existing PBA.. - For PBA, as for all elements of the orthopaedic curriculum, we have produced PowerPoint guides available through the website. - PBAs have been an attempt to maintain and improve the high quality of surgical training in the UK. - We will have to monitor their progress for some time before we will be able to see whether, in the midst of many other changes, they have been successful.. - Department of Health (1993) Hospital Doctors: Training for the Future. - The report of the Working Group on Specialist Medical Training (the Calman Report).. - Department of Health (2007) Trust, Assurance and Safety – The Regulation of Health Professionals in the 21 st Century. - (2002) Unfinished Business – Proposals for Reform of the Senior House Officer Grade. - <http://www.iscp.ac.uk>. - Retrieved from <http:/www.. - bristol-inquiry.org.uk/final_report/index.htm>. - <http://www.constitution.org/mac/prince.txt>. - Available at: <http://www.ocap.. - org.uk/orthocurriculum/Content/01_Intro_160707.pdf>. - Pitts, D., Rowley, D.I. - (2002) A competence assessment tool for the Dynamic Hip Screw. - Galasko Competence Working Party report to the JCHST. - London: Joint Committee on Higher Surgical Training.. - Available at: <http//:www.ocap.org.uk/curriculum>. - Available at: <http://www.rcgp.org.uk/docs/ISS_. - (2002) Competence Working Party report to the JCHST. - Availably at: <http://www.the-shipman- inquiry.org.uk>. - Implementing the Assessment of Surgical Skills and Non-Technical Behaviours in the. - Operating Room. - introduction to the Study. - The Sheffield Surgical Skills Study is currently evaluating the validity, reliability, feasibility and acceptability of three different workplace-based assessment tools for rating surgeons’ technical and non-technical skills in the operating room. - This chapter describes the design, methodology and implementation of the study. - It focuses on the problem-solving approach taken by the research team to address the practical issues of implementing this broad study of behaviours, drawing upon some of the successes and barriers we encountered, to illustrate this. - It is intended to provide valuable lessons for researchers in the field of surgical skills assessment, and for those involved in implementing workplace based assessment into surgical training.. - Traditionally, surgical training in the UK has been based upon an apprenticeship and examination model without formal assessment of technical or non-technical skills. - Trainees undertook a set number of years of training and passed the Intercollegiate Examination of the Royal Colleges of Surgeons (FRCS) to achieve their Certificate of Completion of Specialist Training (CCST) for consultant practice. - Progress in surgical competence was historically achieved through many years and long hours spent in the operating room. - Although log books formed a useful record of surgical experience (Galasko and Mackay 1997), they did not provide evidence of competence (Thornton et al. - However, opportunities to gain experience in the operating room have decreased due to shorter training time following the Calman Report (Calman 1999) and the changes in working practices following the European Directive on Hours of Work (Department of Health 2003).. - Over the last 15 years there has been a move to competency-based surgical curricula in the UK, driven by the introduction of regulations for training by the Postgraduate Medical Education Board (PMETB). - The transitions in surgical training have been described previously by Pitts and Rowley in Chapter 3 of this book.. - Background to Surgical Skill Assessment Tools. - The surgical skill assessment methods developed by the GMC Performance Procedures (Beard et al. - 2005b) and by the medical royal colleges and specialty associations responsible for postgraduate surgical training, are based upon the demonstration of surgical competencies and standards of competence. - The need for robust methods of assessment for technical and non-technical surgical skills is axiomatic, as they underpin the competency based assessment strategy and curricula for all UK surgical specialties.. - Procedure Based Assessment (PBA) and Objective Structured Assessment of Technical Skill (OSATS) are two of the tools being considered in this study. - The overall assessment strategies and individual assessment tools they have adopted conform to the assessment principles laid down by the Postgraduate Medical Education and Training Board (PMETB 2008), and the assessment tools are also designed to measure all the domains of Good Medical Practice (General Medical Council 1998).. - PBAs are embedded within the Orthopaedic Curriculum and Assessment Project (OCAP – <www.ocap.org.uk>) and the Intercollegiate Surgical Curriculum Programme (ISCP – <www.iscp.ac.uk>. - The development of the PBA with examples of the assessment tool is covered by Pitts and Rowley in Chapter 3.. - Objective Structured Assessment of Technical Skill (OSATS) was introduced by the Royal College of Obstetricians and Gynaecologists (RCOG – <www.rcog.. - org.uk>) as a requirement of their New Training and Education Programme, launched in parallel with ISCP in August 2007. - 1994, Martin et al. - The observation of real-time surgical performance in the workplace is essential in the authentic assessment of competence. - the operating theatre has good authenticity for assessing surgical competence, since this method approximates to the ‘real world’ as closely as possible. - Our study seeks to further examine the validity and evaluate the reliability of the PBA tool. - However, there have not been validity and reliability studies performed for the ten OSATS of obstetrics and gynaecology procedures used by the RCOG.. - The third tool considered in this study is the Non-Technical Skills for Surgeons (NOTSS) tool (Yule et al. - This tool is not currently used in a formal way for training in the UK. - However, there is increasing recognition of the need for training and assessment in non-technical skills because of the importance of these skills for patient safety.. - Purpose of the Surgical Skills Study. - The aim is to evaluate the validity, reliability, feasibility and acceptability of three different methods of rating the technical and non-technical skills of trainee surgeons in the operating room across a range of different procedures and surgical specialties.. - The three tools under evaluation in the study are:. - NOTSS: Non-Technical Skills for Surgeons.. - The PBA forms for index procedures used by each UK surgical specialty can be downloaded from the ISCP (<www.iscp.ac.uk>) and OCAP websites (<www.. - ocap.og.uk>. - The OSATS forms used by the RCOG can be downloaded from:. - <www.rcog.org.uk/resources/public/pdf/section6_at.pdf>. - The NOTSS rating form and booklet are available from <www.abdn.ac.uk/iprc/notss>.. - The study commenced in April 2007 at a large UK teaching hospital NHS foundation trust and is due to be completed in June 2009.. - Our intention is to perform between 400 and 500 assessments of surgical procedures.. - To date we have completed 240 cases.. - We are assessing trainee surgeons using the tools for those cases which have the informed consent of the patient. - Each case is judged for complexity by the supervising consultant.. - Assessments of their technical and non-technical. - Direct observation by assessors in the operating room.. - We are currently filming approximately 20 per cent of the cases using a picture in picture technique which records the operating field and the operating room.. - Filming is performed by medical illustration technicians with audio provided by microphones fitted to the trainee surgeon and supervising consultant. - The videos will also provide rich data on the non-technical skills of trainee surgeons in the operating room for collaborative work with the NOTSS team.. - The implementation of the study within a surgical specialty is illustrated by the flowchart in Figure 4.1.. - Relating the Study Design to the Research Aim. - The study aim encompasses several research questions. - We have outlined the main questions below, showing how they have driven the overall study design, and provided examples of how we have addressed them within the study. - Our research questions take into account the assessment characteristics proposed by Van der Vleuten (1996) in his model of assessment utility.. - Validity can be described in a number of ways depending on the context of the assessment. - For us, it refers to evidence presented to support or refute the interpretation of assessment scores, i.e., the degree to which the scores of the assessment reflect the intention of the assessment. - In the case of the assessment Figure 4.1 Flowchart of the study implementation. - tools included in this study, the intention is for the assessment scores to reflect the technical and non-technical surgical competence of the trainee being assessed.. - Our study design provides many sources of validity evidence and these will all be used to support or refute the validity of the three assessment tools. - As one example, if the assessment tools are valid for the assessment of surgical competence, we would expect scores to increase with the trainee’s level of training and experience. - We have ensured that the study includes all grades of trainees and that our demographic questionnaires include questions addressing years of surgical experience and the number of index procedures previously performed by the trainee.. - Reliability refers to the reproducibility of assessment scores. - Reliability within this study is a measure of how well an assessor’s score of the surgical competence for a particular trainee would reflect any assessor’s score when the trainee carried out the operation on any patient. - The use of generalizability theory for the analysis of assessment scores within the study is fundamental in providing the most elegant estimates of assessor variability and case variability, which represent the greatest threats to the reliability of real time assessments in the workplace (Downing 2004).. - Are the Assessment Tools Feasible in Practice?. - There are a number of strands to consider within the scope of assessment feasibility, including the time and resources required for implementation as well as cost effectiveness of the assessment strategy.
Xem thử không khả dụng, vui lòng xem tại trang nguồn hoặc xem
Tóm tắt