Home The Quarterly 2011 Setting Standards for Specialists

The Quarterly

Search

Setting Standards for Specialists Print E-mail
The Quarterly 2011

 


This article was written by Dr Susan Keam, derived from material presented by Dr Chi-tim Hung on Monday 6 September 2010 at RACMA/HKCCM 2010.

An online discussion on "Setting Standards for Specialists" gave participants an opportunity to ask questions and exchange experiences with Dr Chi-tim Hung. Listen to the audio recording here.



What are standards?

Standards can be defined as a goal or measure of progress towards the goal (Wojtczak & Schwarz 2000) . They can be divided into different types, such as those relating to content, those relating to performance and practice and those relating to processes.

The objectives of setting standards in medicine include

  • showing expected norms
  • identifying targets for improvement (individual and systematic)
  • providing credentialling benchmarks for doctors
  • reducing variation in practice
  • promoting patient safety and the health of the population
  • ccountability


Challenges we are facing in medicine

The medical knowledge explosion:
Biomedical knowledge doubles every 20 years, and it is estimated that we need to read 20 papers every day to keep up-to-date with medical knowledge (Grol & Grimshaw 2003). In addition, we are faced with rapid technological advancement. For example, the advent of laparoscopy and robotic surgery are new technologies that create significant learning problems for surgeons who completed their specialist training before the technology development.
Globalisation of medicine:
While globalization of medicine, with cross-border education and the need for global standards for medical education, is of more pertinent interest to undergraduate medical education, it does have an impact on postgraduate training also.
The blurring of traditional professional boundaries:
Lately we have noticed that the traditional divide between medicine and surgery has disappeared where physicians are doing interventional procedures and surgeons are doing more minimally invasive procedures in place of open major surgery, and anaesthetists sit somewhere in between. The blurring of traditional professional boundaries can cause problems in areas such as sedation associated with endoscopy and other procedures, nonsurgical tracheostomy, and in new subspecialties such as sleep medicine.
The inverse relationship between years of practice and performance:
Studies have shown that doctors who have been practising for many years are more likely to have a reduction in factual knowledge, and a decreased likelihood of adhering to care standards, and this may be associated with poorer outcomes (Choudhry et al. 2005). In a US study of recertification examinations (Rhodes & Biester 2007), investigators found that the failure rate increased with the length of duration since initial certification (2006 failure rates of 4.5% [10yrs since certification]), 6.2% [20yrs] and 9.4% [>20 yrs]). In addition, for surgeons, the failure rate in recertification was higher in those who practiced in isolation. These factors raise concerns about quality and safety.
Knowing-Doing Gap and Knowledge Deficit:
McGlynn et al. (2003) have done a study of twelve metropolitan areas in the US in which they contacted several thousand patients to assess the quality of care. Investigators found that overall, only 55% of patients received recommended care (recommended care assessed included diabetes mellitus with regular HbA1c checks, % of elderly patients offered pneumococcal vaccines, colorectal cancer screening). The quality indicator scores for achieving recommended care varied according to disease (e.g. from 78.7% for cataract to 10.5% for alcohol dependence) and the type of care (e.g. from 73.4% for encounter or intervention to 18.3% for counselling or education). Some have argued that this variance may be due to lack of documentation of care provided, but this cannot be proven (McGlynn et al. 2003; Steinberg 2003).

This type of study has been repeated by Mangione-Smith et al. (2007) in paediatric ambulatory care. Results were similar to those seen in adult patients, with just under one half of patients (46.5%) receiving the recommended care (67.6% for acute, 53.4% for chronic and 40.7% for preventive care; range of 92% for upper respiratory disease to 34.5% in preventive care for adolescents) (Mangione-Smith et al. 2007).


Are current mechanisms for setting standards good enough?

There are several different mechanisms for keeping knowledge and standards for specialists up-to-date during their working lifetime (Figure 1), but as might be expected, these vary from country to country.

alt



Setting practice standards for specialists is a moving goal post, because the standard expected has to move synchronously with practice evolution and the pace of technological adoption. For example, standards in surgery have changed with the introduction of robotic surgery, and this technology is something that newly graduated house surgeons are more likely to be familiar with than more experienced specialist surgeons. When setting standards, we need to distinguish whether we are setting standards for graduating specialists or for life-long specialist practice, as content will differ. Whatever the case, we need to keep our standards and practice up-to-date.

The move towards establishing standards is a controversial topic among doctors, especially, as we are moving from a more passive “attending conference” CPD (Continuous Professional Development) process, to an active CPD programme. Questions frequently asked when we set CPD standards are “why do we need these changes?”, “who designs them (people in ivory towers, or practicing clinicians)?”, “how do we know they will work?”, “how can I fit them into my busy schedule?”. So it is important that we understand and explain how standards are developed, who developed them and for whom. One of the more important questions to consider is whether or not standards are mandatory, and if they are mandatory, how are they going to be enforced. A question often asked is that if a standard is not mandatory which implies that it doesn’t matter and if it is not measured, why has the standard in the first place. Thus, to be effective, standards must matter and be measured; however, it must be remembered that standards alone are unlikely to change behaviour (Wojtczak & Schwarz 2000).


Current Issues with Specialist Training

General specialist or subspecialist

For graduating specialists in the 21st century, there is too much to learn for any specialty, and thus there is a need for sub-specialization. In Hong Kong we have been looking at optimising work hours capping at 65 hours per week, down from the 80- or 90-hour week that some trainee physicians and surgeons used to work. This change has had a downstream effect on specialist training. When the original training programme was designed about 15 years ago, it was based on a very long working week; optimising working hours has limited the clinical exposure over the 6-year training period, and in turn this has had an impact on the standard of the training. So we have to ask the question “What kind of standard do we expect after 6 years of training in Hong Kong?”. Are we going to produce a general specialist, or a general specialist with sub-specialty interest, or simply a subspecialist?

In the literature, there is some discussion on subspecialist Internal Medicine training in the US in 21st century. Huddle and Heudebert (2008) argue that since there are so many topics that need to be learnt in Internal Medicine, training should not mirror current internal medicine practice, where there is a continuity of care between the inpatient and outpatient care. Instead, he has proposed that Internal Medicine training should be based predominantly on wide and deep traditional inpatient care. Training in outpatient care is only subsidiary to inpatient training and the necessary skills can be acquired through training blocks. As might be expected, this has generated a lot of discussion. While the model may destroy the idea of a ‘generalist consultant’ in traditional internal medicine training, Huddle and Heudebert believe it enables the internist trainee to make an accurate diagnosis out of ill-defined and complex patient problems. Residency education needs to consider technological and scientific advances to ensure residents are learning the fundamental skills of all physicians (Humphrey 2008).

In response to Huddle's comments, Duffy (2008) says that the curriculum cannot be ever-expanding, and instead of a time-based model of training, a competency-based model should be used. The model proposed by Duffy indicates that the first 2 years should be used as foundation training, using practice-based learning where the instructor leads the trainee through reflection in and on the actions required to learn diagnostic skills. Then in the remainder of the training period, the trainee focuses on gaining skills for the particular subspecialty area. Maintenance of Certification (MOC) then helps to identify and recognize proficiency in providing focussed care within internal medicine. It is important to note that training standards need to adapt to the evolving demands of specialty practice.

Defining competencies

In Hong Kong, how do we then keep our postgraduate medical education up to date so that we can produce the new Hong Kong specialist? To answer this, we have developed the New Hong Kong Specialist Competency Domains. These are generic core competencies in the following areas, and are definable, measurable and assessable:

  • Professional Expertise
  • Health Promoter
  • Interpersonal Communication
  • Team Player
  • Academic
  • Manager-leader
  • Professionalism.

Two key competencies that the doctor-in-training must achieve are that of communicator and of learner/teacher/trainer. Good medicine depends on a good and therapeutic doctor-patient relationship, and last year we published a book in our Hong Kong aiming at enhancing communication between patients, carers and staff. Communication skills should form part of specialist training, and should be formally assessed during the training process, be it formative or summative. Trainees must also learn to be an effective learner and be aware of transferability of knowledge. They also need to manage uncertainty, ambiguity and complexity, be able to work out solutions from basic principles, know their limits, understand teamwork and know when to call for assistance. Teaching is a good way to learn and retain knowledge, and so we emphasise a shared responsibility in mentoring more junior trainees.

Curriculum Planning

New specialists require constantly evolving skill sets and this requires careful curriculum planning. It is important to define and review the requirements regularly to meet our needs. Specialist Colleges should define outcomes and objectives of training at different stages in definable, measurable and assessable terms, such as competencies, and then organise them into training modules. In addition, the training of specialist doctors should be a combination of competency-based and timed-based training.

Simulation plays a big role in the future training and assessment of trainees and existing specialists. In Hong Kong, Specialist Colleges are encouraged by the Academy to build more skill and simulation laboratories and develop simulator-based training. Trainees and existing specialists, especially those with performance issues should have adequate exposure to simulators. In the longer term, some parts of simulator training should be mandatory and be made widely available so that it can precede practicing on the patient. Simulators should also be developed for assessment purposes, ideally starting in areas where the technology is mature and ready.

Assessment should be well rounded, and include skills and attitudes as much as possible. There needs to be a quality assurance process to ensure that examinations meet the required standards. Formative assessment is important and resources need to be injected to allow it to be conducted properly. A matrix of assessment tools to match the competencies needs to be developed if this is to be implemented successfully.

Can CPD provide enough safety?

Another question that needs to be asked is whether our Post-Fellowship mechanisms for ensuring competence are good enough, and whether we can rely solely on the efforts of individual specialists. The success of CPD, Recertification or MOC depends heavily on buy-in from Specialists. We need to consider whether these mechanisms should be mandatory, regulatory or accreditatory. Recent thinking has moved away from focussing the attention on only a small group of outliers (‘bad apples’) to looking at the whole specialist population to enhance the performance of all doctors because everyone is blind to their own deficiencies.

Internationally, not all jurisdictions are making CPD/CME mandatory. In Hong Kong, it is mandatory for specialists, but not for non-specialists. The more important thing is that whatever CPD/CME programme we are promoting there should be a process of structured self-assessment otherwise doctors will tend to choose areas in which they are already proficient missing areas where learning is most needed. Thus, the programme should be personalised and should match individual learning needs and feedback on implementation of a performance change is important (Lowe et al. 2009). We need to make sure that areas involving new technologies (e.g. robotic surgery) are not missed, and that actual performance is monitored. Real life problems encountered by doctors are often ill-defined. Therefore learning should not be separate activities but integral to thinking and doing. Self assessment must include a review of performance in real-life settings including the workplace (Collins 2009).

The Role of Re-certification and Maintenance of Certification (MOC)
Recertification was originally introduced by the American Board of Medical Specialties (ABMS) over 30 years ago (Weiss 2010). This comprises a ‘pen and paper’ assessment at regular intervals every 7-10 years, and confers the right to practice at specialist level. However, questions have been raised as to whether recertification actually improves medical standards (Sutherland & Leatherman 2006). Only half of the studies have shown positive results, and as mentioned earlier, performance deteriorates in line with time since graduation. For example, when we look at the results of the treatment intensification for hypertension treatment study, the proportion of doctors achieving good standards decreased by 21.3% for every decade after last board recertification (Turchin et al. 2008). Another study found that there was a correlation between physician scores in Cognitive Skills and better process of care for Medicare patients (Holmboe et al. 2008).

More recently in the United States, Specialist Boards have moved from recertification to MOC, which incorporates CPD and recertification into the six core competencies of Accreditation Council for Graduate Medical Education (ACGME). This was adopted by ABMS in 2000 and American Board of Surgery in 2005. MOC is divided into four parts where the first three parts mirror the old recertification process, as follows (Rhodes & Biester 2007)

Part I: Professional standing: practising licence
Part II: CME with emphasis of self assessment
Part III: cognitive expertise (regular secure exams)
Part IV: Evaluation of performance in practice

  • Measuring competence and outcomes (e.g. wound infection)
  • Evidence based process measures (e.g. participation in Surgical Care Improvement Project)

Nevertheless, this process is not without challenges. It is seen by some as being burdensome or intrusive, there is little evidence to demonstrate superior competency and improved care for those who elect to recertify and those who do not, and the requirement for maintenance of certification may block access to high calibre physicians (Gloth 2006). In addition, assessment of competence is still difficult, the validity of cognitive exam questions is doubtful, and there is no evidence that taking and passing exams is responsible for the measured improvement. One author has gone as far as suggesting that clinical performance should be examined, rather than cognitive expertise (Loxterkamp 2009).

The role of patients in setting standards for specialists is becoming more important (Miles 2010). The most profound change in medicine in last millennium has been the introduction of informed consent (La'zaro 1999). Care should be delivered within the context of what patients need and want, and in partnership with patients at individual and national levels. The message from patient groups is clear: “show us the data to allow us to trust you”.

The virtues of clinical experience in specialists should not be ignored. Just because seasoned specialists may have less knowledge of new procedures and techniques does not mean that they will have poor outcomes and that we should get rid of them. In a recent study, two groups of experienced surgeons were put into a simulated crisis situation. One group was proficient in laparoscopic surgery; the other group was not but had undertaken a basic laparoscopy course. The outcome for the group of senior surgeons not proficient in laparoscopic surgery was that they always assigned unfamiliar technology to assistants as they were aware of their own deficiencies and asked those who were proficient in the area to do the job. Although not statistically significant, this group of senior surgeons converted to open laparotomy faster than controls (2.4 vs 3.3 min) because they felt more comfortable with open laparotomy; controlled bleeding faster in the open mode with a smaller amount of blood loss (2555 vs 2725 mL). The authors concluded that age of the surgeon is not a good criterion for mandatory retirement (Powers et al. 2009).

Credentialing in Australia and New Zealand is governed by law, and credentialing and recredentialling of all practising doctors occurs at regular intervals. The process involves peer review and management to achieve rigorous scrutiny of the individual doctor’s practice of quality improvement (Logon 2006) . In contrast, in the US, most hospitals do not routinely use board certification to ensure physician competence at their institutions (Freed et al. 2009).


Setting Practice Standards

Practice standards are required for specialists to define the minimum level for new or specialized skills. If you look around the websites of various Medical Associations, there is no lack of clinical practice guidelines; however, practice standards tend to be set for individual specialties, and there are few guidelines that are applicable to a range of specialties.

alt



Here in Hong Kong, we have identified areas that we have to focus our attention on for practice standards, and these are those cutting across professional boundaries (e.g. sedation, sleep medicine) and those with new technologies or tools that are becoming part of regular practice (e.g. laparoscopic procedures, robotic surgery).

A defined process is required when deciding the area where a standard has to be set. For instance, there needs to be a way to collect feedback and evaluate performance. Standards should be evidence-based and focus on clinical care gaps. Other questions requiring answers include ‘should guidelines cover exceptions only, or all situations” , and “should they be inter-collegiate where the procedure crosses professional boundaries”? Sedation was a good example of this in that in the face of increasing demand, there is an inadequate supply of anaesthesiologists worldwide to provide procedural sedation which has led to doctors of other specialties getting involved in the process. However, the inherent risks of sedation are high, and thus a standard for anyone providing sedation has to be set.

International collaboration
Although there are differences in health services delivery between countries, there is no need to reinvent the wheel when it comes to identifying core competencies for practice standards as there are a number already available, such as the seven core competencies for CanMEDS, six core competencies for ACGME, and the basically similar consensus statement for United Kingdom. The question to ask is “is there a need for international standards for post-graduate education and should we be accrediting training institutions?” and the answer thus far seems to be “yes”.

While there are advantages in having international standards (e.g. incentives for improvement; basis of national evaluation; facilitate reforms; facilitate international exchange; basis of accreditation), there are also some disadvantages (the focus on minimum requirements might drive quality down unless there is the differentiation into basic and good practice standards to stimulate quality improvement; interference with autonomy; risk of conformity; feeling of control; no consideration of local differences; may increase brain drain (Karle 2006).

Several International Associations have been formed, two of which are of interest. The International Association of College and Academy Presidents (IACAP) was formed in 1998, drawing from physicians and surgeons groups. The Hong Kong Academy of Medicine is a member. The mission statement of the association “To foster worldwide collaboration in medical specialties to improve health and healthcare” was adopted at the RCPSC Jasper Meeting in 2000 with the chief executive of the Royal College of Physicians London (RCPL) as permanent secretary. The association had met annually at different conferences, with the host organization and the RCPL deciding on agenda.

Items discussed by IACAP include Specialist training; examination, training and mutual recognition; exchange of Fellowship; evaluation of effectiveness of education activity; re-certification, revalidation, CPD and CME; training against service delivery; work hours; government's interference over professional autonomy; healthcare and medical education issues in developing world; surgical education and training and future trends; modernizing medical careers; self sufficiency in health human resources; work force issues and migrating of health professionals. However, ICAP has been quiet recently, and may need to be reactivated to achieve international collaboration in the areas we are interested in.

The World Federation of Medical Education (WFME) is the umbrella organization for six regional and national associations for medical education related to the WHO. Their mission is to improve health through high quality medical education. WFME (2003) published a set of global standards on undergraduate, postgraduate medical education and on CPD. The WFME has set frameworks in nine areas (mission and outcome, training process, trainee assessment, trainees, staffing, training setting and educational resources, evaluation of training process, governance and administration), and have identified 38 sub-areas as performance indicators. They set basic standards, as well as standards for quality development. The WHO / WFME Strategic Partnership was founded in 2004, and is an international task force. It is not involved in actual accreditation, but promotes the formulation of standards and self or external evaluation.

Importantly, we need to consider and discuss whether there should be an international body that oversees all the other accreditation bodies. Whatever the outcome of such a discussion is, such a body should operate within a legal framework, be trustworthy, visible and transparent and based on academic competence, efficiency and fairness (Karle 2006).

Conclusion
Transparency and accountability are important in setting standards for specialists. The emphasis is no longer on finding the bad apples, but assessing the performance of the entire medical workforce. We need regular review of training / education and to discuss how much sub-specialization is optimal. A combination of strategies to enhance performance of specialists including credentialing and practice standards would be ideal in achieving better quality and safety for patients.


Dr Chi-tim Hung
Vice President (Education & Exams), Hong Kong Academy of Medicine, Hong Kong; Hospital Chief Executive, Queen Elizabeth Hospital, Hong Kong / Cluster Chief Executive, Kowloon Central Cluster
 

References:
Huddle, TS. Heudebert, GR. (2008) 'Internal medicine training in the 21st century', Acad Med., vol. 83, no. 10, October, pp 910-5.
Humphrey, HJ. (2008) 'Commentary: Osler in a brave new world', Acad Med., vol. 83, no. 10, October, pp 897-9.
Karle, H. (2006) 'Global standards and accreditation in medical education: a view from the WFME', Acad Med., vol. 81, Suppl. 12, December, pp S43-8.
La´zaro, J. (1999) 'Doctors' status: changes in the past millennium', Lancet, vol. 354, Suppl. 4, pp SIV-17.
Logan, RL. (2006) 'Recredentialling in New Zealand may inform UK recertification', BMJ., vol. 333, no. 7571, October, pp 755.
Lowe, MM. Aparicio, A. Galbraith, R. Dorman, T. Dellert, E. American College of Chest Physicians Health and Science Policy Committee. (2009) 'The future of continuing medical education: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines', Chest, vol. 135, Suppl. 3, March, pp 69S-75S.
Loxterkamp, D. (2009) 'Five easy answers: where the ABFM cognitive exam has gone wrong', Fam Med., vol. 41, no. 3, March, pp 210-2.
Mangione-Smith, R. DeCristofaro, AH. Setodji, CM. Keesey, J. Klein, DJ. Adams, JL. Schuster, MA. McGlynn, EA. (2007) 'The quality of ambulatory care delivered to children in the United States', N Engl J Med., vol. 357, no. 15, October, pp 1515-23.
McGlynn, EA. Asch, SM. Adams, J. Keesey, J. Hicks, J. DeCristofaro, A. Kerr, EA. (2003) 'The quality of health care delivered to adults in the United States', N Engl J Med., vol. 348, no. 26, June, pp 2635-45.
Miles, PV. (2010) 'The future of maintenance of certification: a reaction to the paper by Kevin B. Weiss, MD', J Am Board Fam Med., vol. 23, Suppl. 1, March-April, pp S42-5.
Powers, K. Rehrig, ST. Schwaitzberg, SD. Callery, MP. Jones, DB. (2009) 'Seasoned surgeons assessed in a laparoscopic surgical crisis', J Gastrointest Surg., vol. 13, no. 5, May, pp 994-1003.
Rhodes, RS. Biester, TW. (2007) 'Certification and maintenance of certification in surgery', Surg Clin North Am., vol. 87, no. 4, August, pp 825-36.
Steinberg, EP. (2003) 'Improving the quality of care--can we practice what we preach?', N Engl J Med., vol. 348, no. 26, June, pp 2681-3.
Sutherland, K. Leatherman, S. (2006) 'Does certification improve medical standards?', BMJ., vol. 333, no. 7565, August, pp 439-41.
Turchin, A. Shubina, M. Chodos, AH. Einbinder, JS. Pendergrass, ML. (2008) 'Effect of board certification on antihypertensive treatment intensification in patients with diabetes mellitus', Circulation., vol. 117, no. 5, February, 623-8.
Weiss, KB. (2010) 'Future of Board Certification in a New Era of Public Accountability', J Am Board Fam Med., vol. 23, Suppl., March-April, S32-S39.
Wojtczak, A. Schwarz, MR. (2000) 'Minimum essential requirements and standards in medical education', Medical Teacher, vol. 22, no. 6, January, pp 555-9.
World Federation of Medical Education. (2003) 'Global standards for quality development in medical education'. Accessed at http://www.wfme.org/ [24th April 2011].




{mosloadposition _myarticle}