- In order to guarantee a sufficiently wide range of assessment, each surgical specialty has selected a number of index procedures. - These procedures are selected on the basis of their broad accessibility to trainees, observability and in most cases an aspect of the procedure which contributes something unique to the assessment range. - By the end of training all the index procedures must be scored at the defined competence level of four. - Naturally in the early years an intermediate score is inevitable for all or some domains. - They are linked to the learning agreement and work in synergy with other tools that vary to some degree between specialties.. - Historically the roots of PBA go back in the authors’ experience to the early 1990s 1 when a desire to evaluate the change in performance before and after a fracture fixation course lead to the development of a 20-item multisource feedback tool assessing performance in inserting a dynamic hip screw (DHS) into a fractured neck of femur. - The potential of this approach went unrecognized until 2002 when the recommendations of the JCHST Competence Working Party (Rowley et al. - Parallel developments in other specialties led to the development of the Operative Competence (OPCOMP) tool by Jonathan Beard in vascular surgery (Thornton et al. - In 2005 the tool was introduced to all surgical specialties through workshops conducted for the Specialist Advisory Committee Chairs (Pitts 1 The first PBA was designed in 1994 as a follow up to a project investigating the change in competence following a training course (Oliver et al. - 1997) but was not published until the report of the JCHST Competence working party (Rowley et al. - and Rowley 2005) and minor amendments made to the wording of elements and domains to make them accessible to the widest possible user group. - Since this time they have been embedded in both the trauma and orthopaedic (T&O) curriculum (Pitts et al. - In the latter case some minor changes have been made but the instruments remain broadly identical.. - Features and characteristics of the surgical workplace, alongside the personality of the surgical team and requirements of assessment have influenced the development of the PBA.. - Surgical environment The surgical environment is special and although many aspects of it may be simulated, there is at present no adequate simulation of the high stakes of a real operative procedure. - This imposes considerable constraints on assessment, not the least of which is the central purpose of providing an overwhelmingly safe service to the patient. - Not only do the physical circumstances of the operating environment vary but also the composition of the team, type of instruments in use (even for similar procedures) and most fundamentally the patients in whom there is a wide variation of largely similar anatomy and variation in the severity of disease.. - Nature of the surgical task The basic separation of surgical procedures into emergency and elective shows that some procedures are conducted on suitable patients who may be screened and selected for surgery by a variety of measures beforehand whereas others will arrive unscheduled with possibly life threatening conditions in a variety of states of ill health. - The trainer (a consultant surgeon) is primarily responsible for the care of the patient and often for the leadership of a large team of professionals during the operative procedure. - Scale of the community The orthopaedic community is one of the largest in surgery comprising over 40 per cent of practising surgeons. - In the UK this involves approximately 3000 surgeons (including trainees) in over 450 hospital locations.. - This imposes considerable demands on the innovation process, not the least of which being to provide effective assessor training for the entire community Connecting to patients All patients who undergo surgery in a UK teaching hospital consent to part of their care being undertaken by trainees under supervision. - Curriculum requirements The same principles that have guided the development of the orthopaedic curriculum as a whole also guided the design of PBA. - Orthopaedic curriculum (Pitts et al. - They have been designed with the intention of gaining as much support from the orthopaedic community as possible in order to facilitate their implementation.. - Competence focused – There are debates about the nature or meaning of the word ‘competence’. - Within our particular competence model we look not only for the three key domains i.e., knowledge, skills and attitudes, but also for the unique combination of those domains in areas such as professional judgement. - With these factors in mind we have tried to keep PBA s straightforward and sympathetic to the paucity of time in rapidly changing settings to learn complex tools.. - From the beginning every effort has been made to try to ensure that the PBA’s architecture is sufficiently open to allow synergy with new developments and requirements.. - Driven by the trainee – The triggered nature of the PBA puts responsibility into the hands of those who hold largest stake in seeing training happen – the trainees. - Does the implementation of the whole system make a valid improvement in the outcomes of training? Are the index procedures selected for assessments a valid choice? Is the internal structure of each assessment valid in terms of the measures of performance it proposes? A major problem in this area is the lack of previous measures of surgical competence. - Reliable – PBA should be understood by all in the same way. - Efforts have been made to link PBA closely to accepted practice so that a firm foundation of agreement can be laid for the future.. - Usable – The circumstances in which PBAs are used dictate that this area is of primary concern. - Holistic in approach – It was clear from early observations that many problems encountered amongst trainees had their roots in the area of non- technical skills. - Elements of the PBA address these skills (and highlight them for assessors as well as trainees). - It is hoped that more elements of current non-technical advances will be incorporated into PBA in the future.. - Formative and summative – The notion of a summative assessment where a trainer (possibly external) observes a trainee’s performance in a pass/. - On one hand there seemed to be insurmountable logistic and resource problems but more importantly, training in the workplace is an ongoing activity and assessment should resonate with its formative nature. - workplace assessments should be formative, giving feedback to the trainee to inform and guide her/his future performance. - It was noted, however, that such assessments would, as a whole, be a useful summary of the trainee’s ability to learn and progress. - The successful completion of a PBA is not seen as a licence to operate in that procedure but as a single component of a wider assessment of the trainee’s ability to learn operative procedures and perform them on a variety of patients with differing degrees of severity and complexity in their condition.. - In the 1994 PBA, it was envisaged that the rater/assessor could be a scrub nurse, senior colleague or peer. - The rating of any element was made on the basis of how much evidence there was for the judgment. - For example, one element of the instrument asked about skin preparation, with three options: ‘Was it prepared aseptically/dry prior to draping procedure/ensure no pooling of antiseptic solutions below patient?’. - (NB: the early version posed the questions in a very different way.) The available scores were:. - 1 = no evidence whatsoever that the stage/task/activity has been completed. - For the early versions of the later PBAs, we chose a similarly simple scale but from a different assessment viewpoint. - By this time we were not trying to measure the impact of training, we were attempting to capture a snapshot of the trainee’s behaviour in order to assess competence. - We also considered the inclusion of an extra column that could be marked if a trainee showed excellence at particular points (star quality) but eventually concluded that the simplest rating options would be the most effective.. - A number of factors influenced the choice of the simple scale.. - The first was that we needed to cater for the possibility that not all items would be assessed. - There could be no guarantee that the trainee would be able to complete the whole procedure for a variety of reasons and to complete part of the assessment would be of great benefit to more junior trainees (mirroring actual training practice).. - The consequence of this was that the detail of the observation would only be recorded by the assessor at the end of the procedure. - The final change to the rating scale came after a meeting in which the PBA was discussed by individuals (surgeons, educators and administrators) who had not been part of the original design group. - One person in particular found it difficult to grasp the nominative nature of the scores and insisted on trying to calculate a minimum average score for the PBA. - The inclusion of the global assessment at the end of the PBA was one of the elements acquired from the merger with the OpComp tool. - The inclusion of this domain enables a qualitative triangulation of the other domains which has proved extremely beneficial for the reasons of adding an element of overall professional judgement as described above.. - The power of the PBA assessment rests in part on the fact that the PBA assesses the same competencies in a variety of procedures with a broad range of suitably qualified assessors. - At a later stage the PBA which related to specific procedures was reviewed by a further series of individuals and groups.. - Validity of Index Procedures. - A further triangulation of the selection of index procedures was made using the orthopaedic electronic logbook to check that all selected procedures were accessible to trainees in sufficient numbers (Pitts et al. - A final review of the procedures’ list was made using a further group of surgeons, during a south east training conference, who reviewed the list from the point of view of procedures that they felt they would, in their practice, be able to use to assess trainees Reliability. - Establishing the inter-rater reliability of the PBA tools proved extremely difficult within the time and budgetary constraints of the PBA Orthopaedic Competence Assessment Project (OCAP) project. - Describes an operative plan without the full use of the clinical and investigative material. - Fails to check that the site has been marked. - Ensures that the relevant information such as investigative findings are present. - During the procedure asks theatre staff to look something up in the notes. - Table 3.4 validation worksheet example taken from T&O curriculum (Pitts et al. - for viewing by raters was abandoned due to the difficulty of obtaining sufficiently high quality footage of a lengthy procedure and persuading sufficient numbers of surgical trainers to spend time scoring it. - The positioning of the PBA tool has, from its inception, been as a device ‘designed by surgeons for surgeons’. - We have further supplemented this with a number of audits in various aspects of the PBA (and curriculum) acceptance and adoption by the orthopaedic community and this is described below. - Prior to the launch of the curriculum materials into the orthopaedic trainee population in 2005 a small survey was conducted of trainee activity using trainees attending the annual British Orthopaedic Association (BOA) congress. - In the process of introducing the PBA and other curriculum tools, a number of briefing meetings were held across the UK, with varying numbers attending. - At each of these meetings a survey was issued with questions relating to different tools, including the PBA. - that it was a good idea and, to a lesser extent, that it would work, although all the outcomes tended to the positive.. - It is in the process of being replaced by the Annual Review of Competence Progression (ARCP). - The primary purpose of this tool was to find out what was happening in the field. - The secondary purpose was to send a clear message that the Specialist Advisory Committee (SAC) was taking note of progress and would (and did) investigate instances of non compliance in a low key way. - Later Years of Training. - Orthopaedic trainees often specialise further in the later years of training preparing for a career in a sub-specialty such as spine, joint replacement, hand surgery etc.. - The online version of the orthopaedic curriculum (OCAP Online) was launched in August 2008
Xem thử không khả dụng, vui lòng xem tại trang nguồn hoặc xem
Tóm tắt