ABSTRACT
Lifelong learning and self-assessment are tenets of medical education and health care improvement; quality and patient safety care are essential to the accreditation of organizations providing either continuing medical education (CME) or patient care; accredited CME providers must assess the learning needs of physicians: Accredited health care organizations must document physician participation in education that relates to the nature of care, treatment and services provided by the hospital. The credentialing and privileging of medical staff requires ongoing focused professional practice evaluation based on six general competencies, including compassionate care, medical knowledge, practice-based learning and improvement, effective communication, demonstrated professionalism and coordinated systems-based practice. As those charged with assessment and program evaluation are challenged to produce valid and reliable results to improve education and health care, United States licensing authorities are defining good medical practice and considering competency-based maintenance of licenses. The present paper offers a framework to advance the discussion of relative value credits for gains assessed in knowledge, competence and performance of physicians. A more synchronized and aligned consortium of medical licensing boards, specialty boards and organizations granting practice privileges is recommended to inform the design of education and physician assessment to assure quality and patient safety.
On the basis of tested knowledge and contract with society, physicians in the United States are sheltered from pure market competition. The medical profession is granted authority to control entry into medical education and specialized practice, while the American public controls medical licensure and holds physicians accountable, beyond measures of profits and losses, for patient safety and consistently high quality care.1,2 Because no clinician can retain all the facts necessary to deliver the right care every time to every patient,3 lifelong learning and self-assessment are required to help physicians deliver the best possible care,4 and evaluation is necessary to inform the design of activities meant to educate physicians toward the delivery of best practices.4
This paper reports recent advances made by accreditation, licensing and certification organizations comprising a system intended to assure patient safety and high quality health care. Then, it presents a framework to more closely align physician learning, competence and performance and to advance public debate and reflection on implementation of the framework as a coordinated system of education and care.
EDUCATION AND ACCREDITATION
The early professional education of physicians includes a great deal of time spent in curricula planned by others: the learning is structured by teaching institutions. As their undergraduate medical education and specialty training are completed, physicians spend less time in required classroom and clerkship experiences, and the curriculum progresses from a more static fund of facts and content to an increasingly dynamic course of study that involves patients and the search for scientifically valid solutions to the problems that threaten patients' health.2–4
Across undergraduate5, graduate6 and continuing medical education7, a central concern of medical educators is the gap between what a physician knows and what a physician needs to know to succeed in patient care. Organizations accredited to provide continuing medical education (CME) are required to assess the educational needs that underlie the professional practice gaps of physicians who participate in CME activities. Those needs may be expressed as gaps in medical knowledge, competence or performance, including patient outcomes.8 Accredited providers are required to plan programs of CME with content to match the practicing physician's current or potential scope of professional activities and to implement instructional techniques designed to assure the desired results of the CME activity. By regulation, all scientific research mentioned or used in support or justification of a patient care recommendation must conform to the generally accepted standards of experimental design, data collection and analysis and all activities are expected to be delivered free of commercial influence.8 For the Accreditation Council for Continuing Medical Education (ACCME), the agency responsible for oversight of nationally accredited CME providers, efforts to assure patient safety and to improve the quality of patient care are essential to the mission, policies, standards and criteria that guide the processes of organizational self-study and accreditation.7–8
CURRICULUM AND CERTIFICATION
The American Board of Medical Specialties (ABMS) and its 24 independent member certification boards accept responsibility for establishing and maintaining high standards for the delivery of safe, quality medical care by certified physician specialists in the United States.1,9–11 The Maintenance of Certification (MOC) program of the ABMS requires achievement in professional standing, life-long learning and self-assessment, cognitive expertise and performance in practice.11,12 (See Box 1.)
Box 1. Four Components of the American Board of Medical Specialties Maintenance of Certification Program
Professional Standing involves close monitoring of all medical licenses held by a physician. Licensure is checked at least once by the board within the MOC cycle, with random audits conducted by select boards. Any “final state action” regarding licensing by a state medical board (SMB) is reported to the ABMS.
Lifelong Learning and Self-Assessment includes CME, generally reported as hours of participation, ranging from 10 to 50 per year. Self-assessment opportunities include examinations, tests, and educational modules (many Web based), and linked to the content of the appropriate specialty.
Cognitive Expertise requires passing a proctored, secure, closed-book examination within the prescribed years of the MOC cycle, often toward the last half or last third of the cycle. Examination is conducted under direction of the individual board.
Performance in Practice involves assessment, determining what the physician does on the job, in practice, and what the physician does to improve his or her practice. Methods include review of operative cases or logs, office record reviews or audits, and linkage to practice guideline measures and national performance benchmarks.
The Accreditation Council for Graduate Medical Education (ACGME), which oversees the accreditation of specialty training programs; the Liaison Committee on Medical Education (LCME),13 with oversight responsibility for undergraduate medical education; the ACCME and the ABMS embrace six general competencies that were designed originally to guide the accreditation of residency training programs and the ongoing certification of medical specialists. The competencies ask physicians to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence and improve their patient care.12,14,15 The effect is twofold: teaching programs are prompted to deliver educational activities that address the six general areas of knowledge and skill and physicians are encouraged to execute personally identified curricula accommodating of: 1) compassionate patient care, 2) proven medical knowledge, 3) measured practice-based learning and improvement, 4) effective interpersonal communication skills, 5) demonstrated professionalism and 6) coordinated systems-based practice. (See Box 2.)
Box 2. The Six General Competencies of the American Board of Medical Specialties and the Accreditation Council for Graduate Medical Education
Medical knowledge about established and evolving biomedical, clinical and cognate sciences, and the application of this knowledge to patient care.
Patient care that is compassionate, appropriate, and effective for the treatment of health problems and promotion of health.
Professionalism as demonstrated by commitment to professional responsibilities, adherence to ethical principles and sensitivity to a diverse patient population.
Interpersonal and Communication Skills that result in effective information exchange and teaming with patients, their families and other health professionals.
Practice-based learning and improvement that involves evaluation of one's own patient care and outcomes, assimilation of scientific evidence and targeted improvement efforts.
Systems-based practice as demonstrated by an awareness of and responsiveness to the systems and environments that affect the delivery of health care and the ability to best coordinate those systems to provide optimal care
LICENSURE, COMPETENCE AND EDUCATION
The Federation of State Medical Boards (FSMB) is the membership association of the nation's state and territory licensing and disciplinary authorities authorized through state medical practice acts charged legislatively in nearly all states by a medical practice act to regulate the practice of medicine.16 Medical boards are responsible for protecting the public by maintaining high standards for licensure and discipline of physicians. Assessment for licensure occurs primarily at the time of initial licensure. It requires a medical degree and passage of the United States Medical Licensing Examination, or the Comprehensive Osteopathic Medical Licensing Examination, which act as independent audit mechanisms ensuring basic competence. Licensing authorities agree they have a responsibility to assure the public that physicians, upon renewal of a medical license, are maintaining their competence, but the licensing authorities do not control the resources necessary to assure the competence of every physician.17
The FSMB recently initiated a series of summits involving participants from academia, regulatory agencies, payers, hospitals, the public sector, providers and professional organizations.18 A public statement of competence and a document titled Good Medical Practice USA (GMP USA) were drafted.18 The latter is organized according to the six general competencies that inform the maintenance of certification program of the ABMS and guide accreditation throughout medical education.5–7 Through GMP USA, the public should know what to expect from a good doctor.
The FSMB continues to consult with ABMS, the ACCME, the American Academy of Family Physicians (AAFP), the American Osteopathic Association (AOA), the American Medical Association (AMA), the Council of Medical Specialty Societies (CMSS), the Joint Commission and the National Board of Medical Examiners (NBME) to evaluate physician licensure, post-licensure assessment and to appraise the role of CME and other measures in ensuring the ongoing competence of physicians. To avoid duplication of effort for practicing physicians, a maintenance of licensure committee of FSMB includes representation from the ABMS. It seems possible that the requirements for maintaining board certification, under new guidelines, will be an important component of maintaining licensure to practice medicine.
SAFETY SYSTEMS AND THE ACCREDITATION OF HEALTH CARE
In the United States, there are approximately 635,000 physicians involved in direct patient care, and more than 338,000 of them see patients in hospitals.19 Ninety-six percent of the hospital beds in the United States are found in hospitals accredited by the Joint Commission, an organization seeking to continuously improve the safety and quality of care provided to the public, through the provision of health care accreditation and related services that support performance improvement in health care organizations.20 It is likely that standards of the Joint Commission affect nearly all physicians who care for hospitalized patients. Especially during recent years, Joint Commission standards reflect safety science concepts adapted to health care from the nuclear power and airline industries; accordingly, not only is the competence of physicians, nurses and pharmacists considered in accreditation decisions, but also the hospital's culture, leadership and systems for providing care. As a result, Joint Commission standards require the hospital's clinical and administrative leaders to create a culture of safety and continuous improvement and to adopt a systems perspective in identifying and harnessing opportunities to improve safety and quality.
New and revised standards for credentialing and privileging the medical staff20 require ongoing and focused professional practice evaluations that are based on the six general competencies. Accredited hospitals are expected to show physician participation in continuing education that is developed according to priorities identified by the medical staff and relating to the type and nature of care, treatment and services provided by the hospital. Physician participation is seen as fundamental to the teamwork required to assure patient safety and the evaluative findings of performance improvement activities must be used in designing continuing education.21
A FRAMEWORK FOR IMPROVEMENT IN EDUCATION AND PATIENT CARE
Credit for Participation (Level 1)
Individually or in teams, the primary rationale for participating in CME is the opportunity to gain new knowledge and skills, ultimately to provide safe patient care and improved patient outcomes.22,23 The credit granted for participating in CME forms an important link for medical license renewal, specialty certification and practice privileges. It also affords physicians an opportunity to create portfolios for tracking progress toward attainment of individual goals in education and practice.
The value of credit awarded simply for attending CME has long been in question;24 and for just as long, the question has been answered with regulation requiring CME participation. The underlying rationale holds that participation in lifelong learning and self-assessment is critical to the lifeblood of medicine, and physicians participate in education as responsible citizens of the profession.25 Of late, policy makers have turned to evidence describing preferred educational activities, their relative value in the award of credit and the examination of outcomes associated with CME activities, physicians' competence and performance.
The AMA, owners of the trademark, AMA Physician's Recognition Award Category 1 Credit,™ recently determined that physicians should receive 20 Category 1 Credits for participating in performance improvement (PI) activities; “... structured, long-term processes by which a physician or group of physicians can learn about specific performance measures, retrospectively assess their practice, apply these measures prospectively over a useful interval and re-evaluate their performance.”22 Representing more than 90,000 members, in 2002, the AAFP established criteria for evaluating and categorizing the clinical content of CME, believing, “... an evidence-based approach to CME will help ensure the validity of CME clinical content and lead to improved medical practice and patient outcomes.” Physicians participating in AAFP approved “Evidence-based CME” earn double credit from AAFP.26
Credit for Recall and Application of Knowledge (Level 2)
Table 1 presents the framework of a system to more closely align physicians' knowledge, competence and performance. Measured outcomes may be used to assess the physician's response as a learner engaged in CME and to judge the value of a CME activity designed as an intervention to cause improvement.
Accreditation, Assessment, and Credit: A System to Improve Education and Quality of Care
Assessed outcomes may involve cognitive tests of the recall of relevant information delivered during a CME activity. This testing reflects some knowledge (or lack thereof) of the content spanned. Tests administered before and after a CME activity can be straightforward, but demonstrating that knowledge is acquired following CME does not mean that physicians will automatically apply such knowledge appropriately, or that such application necessarily benefits their patients. These assertions require additional measurement.
There are nascent attempts to gauge the application of knowledge and skills in clinical settings. The challenge is formidable, because success presupposes accurate knowledge of the individual physician's practice context and patient profile. A start can be made, however, by exploring clinical decision making.
Consider a primary care physician with a growing population of Type II Diabetes patients. Using recall interviews, records of three to six diabetic patients may be revisited to reveal the feelings, factors and strategies going through the physician's mind when the clinical decisions were made. Professional, social, personal and health system factors may be elaborated. Evaluators then may assess the cognitive processes of knowledge application, identify gaps in the physician's clinical behavior and suggest opportunities for practice based learning and improvement.
Competence (Level 3); Performance (Level 4)
For the putative physician who participates in CME on diabetes, relevant measures of competence might include the percentage of patients for whom the physician completes foot and eye examinations and Hemoglobin A1C (HgA1c) measurement. It also may be important to demonstrate in a controlled setting or with simulated patients that foot exams can be conducted correctly and that waist to hip ratios and retinal eye examinations can be done appropriately; however, it is widely expected that effective CME should be associated with better patient outcomes and enhanced safety.4 The proof of this assertion is difficult: it lies in demonstrating that the physician's patients have reduced rates of amputations and of blindness from retinopathy.
Some reports at the level of the individual physician and the diabetic patient have been developed for use by managed care organizations or other health plans initiating pay-for-performance (P4P) programs that give physicians financial incentives for offering high-quality care; however, there are major potential sources of error in the derivative analyses.27–29 For example, errors can relate to whether the decisions and actions that resulted in measurable patient outcomes may be attributed directly to the physician or another team member, such as a nurse, resident, or the cross-covering colleague. In addition, outcomes must be adjusted for such factors as case severity, case mix and number of patients seen. Finally, in evaluating for the effects of CME, a limitation is that related educational interventions, both planned and perfunctory, may occur during post-CME periods necessary to collect data on selected processes of care, leaving problematic the causal attribution of any improvement based on CME.
RECOMMENDATIONS
It is beyond the scope of this paper to summarize everything that is known about accreditation, certification, education, licensure, patient safety and improvement. Yet, recent advances in these fields suggest it is time to consider a more consolidated system for recognizing physicians, as they improve their knowledge, competence and performance, working to reduce harm.
Documented credit can bring personal gratification to physicians. It also may be used in partial fulfillment of requirements for maintenance of certification, and similar benefits seem probable for the emerging conceptual model of maintenance of licensure. Earned CME credit provides the opportunity to create a portfolio for tracking through one's personalized course of study, defined individually and assessed for progress, in relation to the six general competencies. CME providers are challenged to improve the quality of education through better evaluation and assessment tied to the physician's performance in organizations of care. (See Table 1.) A more robust system for awarding credit may drive ever more creative uses to improve the accreditation of education and health care organizations, as well as processes of credentialing and privileging.
There are three major recommendations:
A system of relative value credits should serve physicians and organizations interested in assuring patient safety and quality improvement through physicians' achievements in lifelong learning and self-assessment. The first level of credit might recognize attendance or participation of the physician-learner in educational activities that address medical practice. Assessed gains in recalled or applied knowledge should be recognized at the second level. Proficiency in simulation or improved clinical skills assessed should be granted higher value, at level 3. The highest value is recognized at level 4, rewarding the physician for his/her measured improvement in practice. For example, the National Committee for Quality Assurance (NCQA)28 reports, from physicians 'records, the proportion of enrollees in a health plan with hypertension who have their blood pressure controlled. A physician might progress from the 80th percentile to the 90th percentile on compliance with a select diabetes guideline, ordering tests for HgA1c and bringing a higher proportion of patients into compliance with acceptable HgA1c levels. Such performance already is valued highly for decisions involving certification, licensing and privileges to practice.
Accreditation of CME providers and health care organizations should: a) encourage assessment useful to the physician and to others interested in gauging the physician's progress toward improved patient safety and quality of care, and b) promote the evaluation of CME activities to determine more precisely their relative value in causing improved knowledge application, competence and performance. Better assessment of physicians' knowledge, competence and performance and better evaluation of CME activities should be rewarded with more favorable accreditation decisions in CME and in health care.
A more synchronized and aligned consortium of medical licensing boards, specialty boards, hospitals and other organizations granting practice privileges should determine the best mix of education and assessment for assuring patient safety and quality improvement. For example, a select medical board or licensing board might determine that over a 3-year period, 10 credits at level 2; 4 credits at level 3; and 10 credits at level 4 should be required for demonstrated success in maintenance of certification or for renewal of a medical license. Moving in this direction renders moot the question of credit awarded solely for participating in CME.
CHALLENGES
Health care is provided in systems that include relationships, groups, associations and communities ordered by institutional structures and interpreted by cultural patterns of meaning. Recent reviews and commentaries30–32 suggest physicians find themselves successful in self-assessment, not independently of other people, standards and institutions, but through them; yet, valid and reliable measures of knowledge, competence and performance are not widely available to assist in assessment and professional development. Resources in evaluation, study design and statistics must be brought to bear to aid the effort.
The positive side of a course of study privately assessed and individually executed may be the sense of dignity, worth and moral autonomy of the individual professional, but the performance of the physician ultimately depends on an agreement with society, a license to practice in a culture that is challenging medicine to measure its successes and reduce its failures. The expectations of patients must be discussed publicly with the successes and failures of physicians practicing in systems of care. The value of the discussion depends upon the trust, leadership and civic cooperation of medical professionals. Patient safety depends on the transparency of the discourse for improvement, and across the continuum of medical education, planners rely on the interaction and results of the conversation to enhance the practice and standing of medicine as a profession.
- Copyright 2008 Federation of State Medical Boards. All Rights Reserved.
REFERENCES
- 1.↵SullivanWM. Work and Integrity: The Crisis and Promise of Professionalism in America. Second Edition. Jossey-Bass, Inc.: San Francisco, CA. 2005.
- 2.↵Institute of Medicine . To err is human: building a safer health system.KohnLinda T.Corrigan,Janet M.DonaldsonMolla S. , eds.Washington, DC: National Academy Press, 2000.
- 3.↵Institute of Medicine . Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001. pages5– 6.
- 4.↵Institute of Medicine . Health professions education: a bridge to quality.GreinerAnn C.KnebelElisa, eds.Washington, DC: National Academy Press, 2003.
- 5.↵SimonFA AschenbrennerCA. Undergraduate medical education in accreditation as a driver of lifelong learning. J Contin Educ Health Prof.2005; 25( 3); 162– 167.
- 6.↵LeachDC. In search of coherence: A view from the accreditation council for graduate medical education. J Contin Educ Health Prof2005; 25( 3): 162– 167.
- 7.↵RegnierK KopelowM LaneD AldenE. Accreditation for learning change: quality and improvement as the outcome. J Contin Educ Health Prof2005; 25( 3): 174– 182.
- 8.↵Accreditation Council for Continuing Medical Education . The ACCME's Essential Areas and Their Elements. Available at: http//www.accme.org/dir_docs_upload/f4ee5075-9574-4231-8876-5e21723c0c82_uploaddocument.pdf. Accessed May 4, 2007.
- 9.↵LudmererKM. Time to Heal: American Medical Education from the Turn of the Century to the Era of Managed Care. Oxford University Press: New York, 1999. p. 87.
- 10.American Board of Medical Specialties . What We Are & What We Do. http://www.abms.org/about_ABMS/Who_We_Are.aspx. Accessed May 4, 2007.
- 11.↵MillerSH. American board of medical specialities and repositioning for excellence in lifelong learning; maintenance of certification. J Contin Educ Health Prof2005; 25( 3): 151– 156.
- 12.↵American Board of Medical Specialties . What is Maintenance of Certification © (MOC)?http://abms.org/About_Board_Certification/MOC.aspx Accessed May 4, 2007.
- 13.↵Liaison Committee on Medical Education: Accreditation Standards http://www./cme.org/standard.htm#ed1a Accessed Sept. 10, 2007
- 14.↵MooreDE PenningtonFC. Practice-based learning and improvement. J Contin Educ Health Prof2003; 23( S1): S73– S80.
- 15.↵Accreditation Council for Graduate Medical Education . The ACGME Outcome Project: An Introduction. http://www.acgme.org/outcome/opintrorev1_7_05.ppt#13.
- 16.↵Federation of State Medical Boards . http://www.fsmb.org/mission.html Accessed May 4, 2007.
- 17.↵JohnsonDA AustinDL ThompsonJN. Role of state medical boards in continuing medical education. J Contin Educ Health Prof. 2005; 25( 3); 183– 189.
- 18.↵Federation of State Medical Boards . http://www.innovationlabs.com/summit/Draft_TOC_Good_Medical_Practice.pdf. Accessed May 21, 2007.
- 19.↵Census of Physicians 2006 . AMA Professional Physician Data: Department of Census and Self-reported Data. American Medical Association. Chicago, Illinois60610.
- 20.↵http://www.jointcommission.org/AboutUs/joint_commission_facts.htm. Accessed May 21, 2007.
- 21.↵2007 Hospital Accreditation Standards . The Joint Commission. Oakbrook Terrace, Illinois2007, pp 371– 393.
- 22.↵ACCME Glossary of Terms and Abbreviations . http://www.accme.org/dir_docs/doc_upload/bd51dd7 Accessed May 6, 2007.
- 23.↵The Physician's Recognition Award and Credit System: Information for accredited providers and physicians. http://www.ama-assn.org/ama1/pub/upload/mm/455/pra2006.pdf Page 4. Accessed May 21, 2007.
- 24.
- 25.↵MooreDM BennettN KnoxA KristofcoR. Participation in formal CME: Factors affecting decision making. IN: DavisDA FoxRD , eds.The Physician as Learner: Linking Research to Practice. Chicago: American Medical Association; 1994: 221– 240.
- 26.↵American Academy of Family Physicians . http://www.aafp.org/online/en/home/cme/cmea/ebcme.printerview.html Accessed May 4, 2007.
- 27.↵http://www.Premierinc.com/qualitysafety/tools-services/p#4p/hqi/specification-by-focus-area.jsp Accessed Sept. 10, 2007.
- 28.↵National Committee for Quality Assurance, Diabetes Physician recognition Program. http://web.ncqa.org/tabid/139/default.aspx Accessed May 23, 2007.
- 29.↵McGlynnEA AdamsJL. Public release of information on quality. IN:AndersonRMRiceTHKominskiGF, eds.Changing the U.S. Health Care System: Key Issues in Health Services Policy and Management. 3rd Edition.San Francisco: Jossey-Bass; 2007: 245– 272.
- 30.↵DavisDA MazmanianPE FordisM HarrisonRV ThorpeKE PerrierL. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006; 296: 1094– 1102.
- 31.Epstein,RM. Assessment in medical education. N Engl J Med2007; 356: 387– 396.
- 32.↵KlassD. Assessing doctors at work – progress and challenges. N Engl J Med2007: 356: 414– 415
