Abstract
Risk from sub-optimal medical practice remains a perennial international problem. While regulatory efforts for improvement have been significant, new thinking and innovation is needed. In an ideal world, professional career paths would be enhanced, supported and successfully maintained from medical school to retirement. Regulatory outcomes would be made resilient to public and professional challenge. Professional development, with quality improvement at its heart, would be maximized, and concerns about medical competency would be highlighted and acted upon at an early stage -- before they become “a fitness to practice” matter. At this early stage, referred to in this paper as the “amber zone,” concerns about an individual's ability to practice medicine competently may emerge, but they are not considered of sufficient severity to warrant a referral to a fitness to practice inquiry by medical regulators. The introduction of a concept called Insightful Practice is one attempt to address the unmet challenge of the amber zone. A surrogate measure of professionalism, Insightful Practice is a method that assesses medical professionals' engagement with the system within which they work and with feedback on their performance for any given work role. In addition and crucially, the method considers medical professionals' insight into what they need to change and their plans for improvement. Potential problems are identified early, increasing the likelihood that remediation measures will be successful. An application using Insightful Practice is described here, examples of its use given, and a discussion is provided of the concept's advantages, limitations and potential to help regulatory authorities and other health care agencies address the challenge for regulatory systems to identify and remediate medical professionals who find themselves in an early amber zone of concern. The application described is based on humans' long understood struggle “to see ourselves as others see us,” and is an attempt to support and channel medical professionals' integrity and drive for improvement in order to protect patients. While the Insightful Practice concept is discussed in this paper in the context of the UK's regulatory system, its principles are applicable to other medical regulatory systems around the world.
Introduction
The practice of medicine is a privilege. Public trust, while it may be enduring, is rooted in the expectation of the best possible professional care available. Medical regulators are charged with ensuring standards are maintained, unprofessional practice dealt with and patients protected by encouraging the raising of overall standards of professional practice.1,2 “Putting patients first” is a key principle of the regulation of physicians that both the public and profession would agree is unarguable. While the goal seems simple, the process is complicated.
In addition to having adverse impact on patients and their families, failings in medical professionalism have significant adverse impact and costs for health care providers, professional organization bodies and medical defense agencies.* In preventing these costs, and to minimize risk, potential ambiguity around roles and responsibilities of these multi-agency interests may need clarification. For example, it is accepted that medical regulation needs to become involved when physicians do not engage with their organizations' systems of appraisal, or when a serious adverse event or complaint emerges. But should regulators also take a pro-active interest when tacit information within the system suggests a physician's performance may be sub-optimal? Or is that the responsibility of health care employers alone? Arguably, medical regulators should keep a wide sphere of influence and interest in order to ensure that different efforts dovetail to offer a robust and comprehensive system of oversight, with no weak links. Failure to do so creates the risk of unprofessional health care workers gaming their way around the system, or moving on before their behaviors are identified and acted upon. The ideal is for an integrated system, which would identify medical professionals who are at risk before patients are harmed — and the associated human and material costs are incurred.
Many agencies have an interest. Medical schools seek to recruit good students and to promote standards of professional behavior.3–5 Health care employers and professional and accreditation bodies seek high standards of quality improvement. Medical defense agencies seek methods by which potential litigation and risks can be identified early and their impact minimized.
In addition to taking responsibility for “fitness to practice” cases (i.e., formal inquiries into the competency of a physician to practice medicine), regulatory authorities in some countries play other important roles. These include: 1) participating in the monitoring of medical schools' efforts to support the development of professionalism and deal with problems, 2) enhancing established medical professionals' personal development and, importantly, 3) identifying those medical professionals who are starting to lose their way and becoming a risk to the system — before serious harm results.
THE IDEAL IS FOR AN INTEGRATED SYSTEM, WHICH WOULD IDENTIFY MEDICAL PROFESSIONALS WHO ARE AT RISK BEFORE PATIENTS ARE HARMED.
Current initiatives
While there has been considerable global effort by regulatory authorities to achieve quality improvement in their systems, and some of these efforts continue to progress, problems persist and opportunities for improvement remain.
In the United Kingdom (UK), practicing physicians are licensed and registered with the General Medical Council (GMC) after completing a lengthy educational process (in general, anywhere from four to nine years of total medical training). Continuing medical education is mandatory, and the GMC has the authority to sanction physicians who put patients at risk.
In recent years, a new, comprehensive system for continuous professional development has been introduced in an effort to strengthen the ongoing competency of physicians in the UK. The GMC's “revalidation” system requires all medical practitioners to undertake an annual appraisal by a trained peer appraiser with formal review every five years in order to remain on the GMC's specialist registers as being “fit for practice.”6 UK revalidation aims to reassure the public through positive affirmation of physicians' performance and skills, and by checking that physicians are making appropriate efforts to keep their knowledge and skills up to date. The system involves a collection of information based on a portfolio that includes specified categories: peer feedback, patient feedback, complaints, significant-event analysis and evidence of self-directed continuous professional development (CPD) activities. This evidence is discussed and learning is facilitated through the peer appraisal. Post appraisal, the health employers' Responsible Officers (ROs)** confirm satisfactory completion or flag concerns to the GMC, following which, the GMC decides on physicians' revalidation.6
While comprehensive, the UK's system still requires no formal knowledge-testing of physicians. After the tragedy caused by Harold Shipman — a general practitioner and the U.K.'s most prolific serial murderer, who killed his own patients — an inquiry was launched to establish what lessons should be learned and make recommendations for actions to prevent future risk to patients.7 One of the inquiry's recommendations was that an “open book” knowledge test should be a minimal requirement of doctors for their future revalidation.7 While this option has since been piloted with general practitioners as part of a research study, it has not been instituted by the current system.7–8
In 2014, the GMC commissioned a group of independent researchers to undertake a long-term independent evaluation of UK revalidation, based on an agreed evaluative framework.9,10 A final report is expected in 2018 and the evaluation's outcomes and recommendations are eagerly awaited. Two outcomes of the evaluation are of particular interest. First, the effectiveness of face-to-face appraisal as a reliable component of revalidation will be of key importance, as its reliability has been previously questioned.8 Second, whether physicians whose practice may be falling into an “amber zone” (i.e., tacit knowledge in the system has identified concerns with their medical practice, but these are not considered sufficiently severe to warrant a referral to a fitness to practice inquiry) are being identified by the process. If physicians in an early amber zone of concern are being successfully identified, it will then be important to establish if the future trajectory in standards of their practice is being effectively tracked and supported to protect patients before harm can occur. Last, the tracking of progress of any cohort of such amber-zone physicians will need to be based on a valid, reliable and feasible system which is clear around the roles, responsibilities and accountabilities of involved agencies. This raises a number of challenges.
‘THE ROUTE TO SELF-IMPROVEMENT IS NOT THROUGH BECOMING A MORE ACCURATE SELF ASSESSOR, BUT THROUGH SEEKING OUT FEEDBACK FROM RELIABLE AND VALID EXTERNAL SOURCES.’
Challenges
Self-assessment
While self-assessment has an important role in helping medical professionals' identify their learning needs, the accuracy of self-assessment, without the benefit of calibration by the comparison with individuals' peer assessment, is known to be poor.11,12 A systematic review of the physician literature found the majority of studies demonstrated little or even an inverse relationship between self and external assessments.11 Individuals' self-assessment skills seem related to levels of competence and the accuracy of self-assessment seems poorest at the extremes of performance.11,13 High-achievers, as individuals with high levels of competence, tend to be over-critical of themselves, while individuals with the lowest competence, and whose practice is likely of immediate concern, tend to overestimate their abilities and seem relatively unable to amend their opinion effectively, even when provided with independent external assessment.13 On the other hand, individuals in the mid-level of competence are most accurate in assessing their own performance and this insight is maintained following independent external assessment feedback.13
Reflective practice
In a review paper, Eva and Regehr argued that “the route to self-improvement is not through becoming a more accurate self-assessor, but through seeking out feedback from reliable and valid external sources, and then making a special effort to take the resulting feedback seriously rather than discounting it.”14 They argued that safe practice requires an on-going dynamic process of “reflection-in-practice,” aimed at continuously monitoring one's ability by addressing emerging difficulties.14 They recommended that external feedback be reflected upon to support effective self-awareness.14
Is there a “regulatory gap”?
High achievers' standards of practice are unlikely to be of concern to the public. Physicians whose practice is known to be of serious concern are likely to be already dealt with by regulatory systems' fitness-to-practice procedures. It is therefore professionals in the amber zone of performance, rather than the high achievers, or those of immediate concern that pose particular difficulty. Physicians within this amber zone risk becoming caught in a regulatory gap, where there may be tacit knowledge suggesting poor performance, but not enough concern to trigger current action by employers or regulators. In addition, those within this amber zone might challenge the validity and reliability of assessments which identify them and resist their classification as being in potential difficulty, so preventing engagement with monitoring or early remediation. Arguably, the development of a feasible system that is valid and reliable could act as a second tier of increased support and scrutiny. This would offer advantages to all parties. Enactment of a second tier process could be triggered by peer appraisers or other concerned bodies. The further clarification of a physician's progress, through an additional valid and reliable system, would help protect the status of both the physician and the physician's appraiser as a supporter and educational mentor, while providing the background and rationale for regulatory outcomes that are open to challenge by both medical professionals and public.
THE ACCURACY OF SELF-ASSESSMENT, WITHOUT THE BENEFIT OF CALIBRATION BY THE COMPARISON WITH INDIVIDUALS' PEER ASSESSMENT, IS KNOWN TO BE POOR.
Keeping professionalism “on-track”
If we are to meet the challenge of the regulatory gap, it is important to understand the risks that can threaten to derail medical professionalism. Maintaining professional standards throughout any career is a significant challenge, given the pressures generated by changing expectations of best practice. Over the lifetime of a medical career, changes in practice context, social circumstances, personal health issues and disaffection are risks that can impact any practitioner. Given that at the start of a career one is “on track” for a successful professional working life, there is a strong argument that monitoring satisfactory progress throughout a career is desirable and offers practitioners confirmation of their ongoing professionalism, self-worth and standing with their patients. Medical professionals need to be given a reliable early warning that they may be wandering “off track” and given support and opportunity, if needed, to correct and change direction. While human-factor science and error theory point to systems as the main cause of error, the quality improvement movement has recently observed that medically adverse events can be more than uniquely a system problem.15 Some individuals repeatedly display incompetent or grossly unprofessional behaviour. Evidence exists that a minority of physicians account for the vast majority of complaints.15 As a result, addressing the problem of those practitioners labelled as “bad apples”15 has been suggested to augment the quality improvement field's still crucial efforts to improve the design of organizational systems and human factors.15 While agreeing on the importance of identifying and dealing with clearly unprofessional behaviour, we would argue that this view is too simplistic. We would assert that some medical professionals in difficulty may not be “bad,” but unwell or poorly advised or supported — and still potentially remediable if identified and encouraged back into the professional fold before patients suffer harm. Rather than searching for a system to find the “bad apples,” should we view the problem from an alternative standpoint, where any professional, when challenged by life events, has the potential of losing his or her way and putting patients and colleagues at unnecessary risk?
CURRENT ASSESSMENT SYSTEMS ARE OFTEN VIEWED BY PHYSICIANS AS HOOPS TO JUMP THROUGH WITH NO CONNECTION TO THEIR EVERYDAY WORK-ROLES.
Assessment, performance and criminality
Criminality and serious complaints bring public concerns to the forefront and demand action from regulators.16 Many quality improvement systems in use by regulators rely on health professionals' self-assessment of their CPD activities. If not triangulated by other independent methods, this approach can be flawed and open to the manipulation of evidence. For example, Harold Shipman was complimented on his personal submission of audits just nine months before his arrest. His feedback at his medical appraisal read: “Great to see a single-handed enthusiastic GP with a rolling program of audit — keep up the good work.”16 Such cases of murder of patients by clinicians can be used to resist or criticize innovation in regulatory monitoring systems as unlikely to protect against criminal events.16
OVER THE LIFETIME OF A MEDICAL CAREER, CHANGES IN PRACTICE CONTEXT, SOCIAL CIRCUMSTANCES, PERSONAL HEALTH ISSUES AND DISAFFECTION ARE RISKS THAT CAN IMPACT ANY PRACTITIONER.
Why measuring professionalism could be the key
Despite various international efforts, current assessment systems are often viewed by physicians as hoops to jump through with no connection to their everyday work-roles.17 We believe systems should require all health professionals to engage in effective reflection to demonstrate their responsibilities and accountability to the public. New methods are needed, however, if current testing systems are to extend to the measurement of integrity and professionalism.17
Agreement of a definition of professionalism is of key importance. Given the multitude of personal qualities and skills and the different contexts in which medicine is practiced, this is extremely difficult.18 Currently, medical regulators generally consider medical professionals to be dedicated individuals who are engaged with their continuous professional development (CPD). Repeated evidence of poor performance over time, serious adverse events, and complaints will result in action by medical regulators. This simple standpoint is attractive. It proffers reassurance that the majority of professionals are of good standing and supports employment needs. But it offers little protection for patients by exposing them to the risk of experiencing random negative outcomes before action is taken and the harm has already been done (Figure 1).
Professionalism and Risk
The Concept of Insightful Practice: A Proposed Way Forward
In this paper, we put forward Insightful Practice as a new concept to act as a surrogate measure of physicians' professionalism in maintaining an appropriate response to independent feedback on their career progress.8,19 Insightful Practice is defined as professionals demonstrating appropriate levels of engagement, insight and action when presented with credible and independent feedback on their individual and/or team performance.8,19 This alternative system is based on the view that we begin our careers from a professional position on entry to medical school. The challenge is to help practitioners maintain this position throughout their careers in order to maximize their competence and performance to protect patients. Use of Insightful Practice is based on professionals' appropriate response to independent feedback on performance, testing professional integrity and offering a basis for monitoring and improvement. The method is designed to facilitate effective reflection on professional performance, give early warning of any problems and, where required, monitor successful remediation to maintain a professional career path. Insightful Practice requires physicians to engage with the system within which they work, show insight into the messages conveyed by credible and independent feedback on their performance and set meaningful personal objectives for improvement. Insightful Practice is cyclical and aimed at continuous improvement and maintenance of professional standards (Figure 2).8,19
INSIGHTFUL PRACTICE IS DEFINED AS PROFESSIONALS DEMONSTRATING APPROPRIATE LEVELS OF ENGAGEMENT, INSIGHT AND ACTION WHEN PRESENTED WITH CREDIBLE AND INDEPENDENT FEEDBACK.
INSIGHTFUL PRACTICE EXPLICITLY ACKNOWLEDGES THAT ALL PROFESSIONALS HAVE AREAS OF NEED, BEYOND THEIR SELF-ASSESSMENT.
Cycle of Insightful Practice
The proposed system of Insightful Practice requires further consideration with respect to the literature on physicians' reflection and self-assessment.
Insightful Practice can bolster individuals' self-assessment and reflection on their practice. We believe it meets the call described earlier, by Eva and Regehr, for a fresh approach that can monitor the dynamic and more forward-looking process of physicians' reflection-in-practice to make any required quality improvement. Insightful Practice explicitly acknowledges that all professionals have areas of need, beyond their self-assessment, that could, if addressed, benefit patients. For the minority of medical professionals in the amber zone who fail to engage, demonstrate insight, or alter course, despite support and facilitation, the system would provide regulators and employers an early warning of risk of sub-optimal care. Reassuringly, two previous and separate studies, conducted in Scotland, have shown the proposed concept of Insightful Practice to offer regulators and medical schools a valid and highly reliable measure to differentiate between family doctors and medical students on their levels of professionalism.8,19
Insightful Practice: Family practitioners' study
The original study investigating the concept of Insightful Practice involved 60 family practitioners and 12 peer appraisers in Tayside in Scotland. In this study, face-to-face assessment of physicians' levels of Insightful Practice following their statutory medical appraisal by a peer appraiser proved unreliable as a method of assessment. However, anonymous global assessment by three blinded web-based appraiser assessors of the physicians' levels of Insightful Practice was highly reliable (G=0.85), as were revalidation decisions using four anonymous assessors (G=0.83). Unlike face-to-face appraisal, anonymous assessment of Insightful Practice was shown to offer a valid and reliable method to decide recommendations on the revalidation of family practitioners.8
...TWO PREVIOUS AND SEPARATE STUDIES, CONDUCTED IN SCOTLAND, HAVE SHOWN THE PROPOSED CONCEPT OF INSIGHTFUL PRACTICE TO OFFER REGULATORS AND MEDICAL SCHOOLS A VALID AND HIGHLY RELIABLE MEASURE...
Insightful Practice: Medical students' study
Following the family practitioner study, a similar study was conducted at University of Dundee's Medical School. This study involved 28 fourth-year medical students divided into two equal groups (n=14). Both groups were assessed by medical school staff web-based assessors for levels of Insightful Practice in response to their available assessment feedback. One group was assessed by blinded and calibrated assessors, and the other group was assessed by blinded and uncalibrated assessors. Calibration of assessors involved a single one-hour meeting to discuss the concept and share experience of making assessments of “virtual examples” of possible students' portfolios. Insightful Practice offered a feasible and highly reliable global assessment for calibrated anonymous assessors, G (inter-rater reliability) > 0.8 (two assessors). Assessment by uncalibrated assessors, however, showed low reliability (G < 0.31). This study concluded that calibrated assessment proved an acceptable basis to enhance feedback and identify concern in students' professionalism. Students reported increased awareness in teamwork and in the importance of heeding advice. Students' staff coaches (appraisers) reported improvement in their feedback skills and commitment to improving the quality of student feedback.19
UNLIKE FACE-TO-FACE APPRAISAL, ANONYMOUS ASSESSMENT OF INSIGHTFUL PRACTICE WAS SHOWN TO OFFER A VALID AND RELIABLE METHOD TO DECIDE RECOMMENDATIONS ON THE REVALIDATION OF FAMILY PRACTITIONERS.
What about high achievers and amber-zone physicians who lack insight?
Insightful Practice appears to offer possibilities for high achievers and amber-zone physicians. For the context of Insightful Practice, physicians' self-assessment involves reflection on a valid, credible and independent suite of feedback rather than limiting its scope to physicians' own personal perceptions of their performance. In high-achiever cases, the patient benefits from the high achiever's continuous striving for excellence, and the high achiever, in turn, is reassured and validated. On the other hand, in the case of amber-zone or “bad apple” physicians, both would need careful support and recording of progress. Those who are unable to engage with feedback, make improvements and get back on track are arguably buying out of their professional responsibilities and, if a risk to patients, need to be referred to a fitness-to-practice hearing.
It is important to stress that all of the above scenarios concern insight, or its absence. Criminality is another matter entirely. It is unlikely that any fixed assessment system could be relied upon to address this.
How would a system based on Insightful Practice work?
We believe the measure of Insightful Practice could bolster and stabilize existing regulatory and monitoring systems. Figure 3, by the addition of a second triangle to the earlier Figure 1, illustrates how additional scrutiny could be provided by incorporating Insightful Practice into the overall system (Figure 3). This additional scrutiny could be used to cross-check the validity of outcomes for a sample of those assessed by current systems. It could also help to identify, monitor and support those showing early signs of difficulty. In addition, a system based on Insightful Practice could help support the current shift in the governance processes of medical regulatory systems.20 It could increase opportunities for lay public involvement in future developments by, for example, including the use of lay assessment.20 Insightful Practice is not proposed as a replacement for existing systems of professional appraisal or fitness-to-practice systems. It is proposed to fill the regulatory gap mentioned earlier. It could help address a difficult group of practitioners who are in the amber zone of early concern and in danger of being consigned to a “limbo status,” waiting for the calamity that triggers serious action after the avoidable harm has occurred. The system could be adopted and molded to any given work-role or context to give valid feedback. Robust and efficient, the system could help avoid escalating concern resulting in future fitness-to-practice cases. In doing so, the system would help regulators, higher educational institutions, deaneries, indemnity organizations, medical schools, and other groups to meet their shared aim of helping ensure robust outcomes to underpin safe and effective patient care.
THOSE WHO ARE UNABLE TO ENGAGE WITH FEEDBACK, MAKE IMPROVEMENTS AND GET BACK ON TRACK ARE ARGUABLY BUYING OUT OF THEIR PROFESSIONAL RESPONSIBILITIES...
Professionalism and Risk
Using Insightful Practice to Measure Professional Reflection: Four Steps
Blank templates of the four steps are provided in Figures 4–7. For examples, visit this link: www.tipportfolio.co.uk/InsightfulPractice.htm
Step 1: Time to Reflect on Your Feedback
Step 2: Now Set Your Objectives for Improvement
Step 3: Reflect on Your Insightful Practice
Step 4: Professional Regulation: Independent Anonymous Review Template
Step 1 (Figure 4)
Professional participants are asked to reflect on each component from their agreed suite of credible independent feedback data by replying on the web to four-question prompts to each component. Components making up the suite of feedback can be contextualized to the work-role being considered. For example, in the examples provided (www.tipport-folio.co.uk/InsightfulPractice.htm) different suites of data are used. In the undergraduate medical school application section, the feedback is based on identified examples of what are considered lapses in professionalism by the medical school. In this case, the purpose of the exercise is to assess and monitor longitudinally an appropriate response by the student to get their professionalism on track.
In the post-graduate section, feedback sources include peer and patient feedback tools as well as complaints. The purpose of this post-graduate application is to enhance the participants' response to feedback to improve the care they provide as well as identify those who are failing to address important opportunities for improvement, or worse, are possibly practicing in a manner that appears unprofessional. Participant responses to question (c) — “This feedback highlighted planned change” — are automatically harvested by the web system from each entry against each source of feedback to populate a “box of ideas” on the web-site for the participant to subsequently fashion into objectives for improvement at Step 2.
Step 2 (Figure 5)
Participants use their “box of ideas” for change to develop Specific, Measurable, Achievable, Relevant and Time-based (SMART) objectives for any areas of identified improvement or further development.21 A drop-down menu allows participants to allocate a relevant regulatory domain for each of their objectives. The drop-down template uses the UK General Medical Council's current domains for appraisal and revalidation, but this can clearly be specified appropriately by the system to any given context.10 The purpose is to underpin the importance of personal reflection, engaging with the system within which the participant professional works.
Step 3 (Figure 6)
In Step 3, participants are asked to rate the quality of their reflection and whether best use of it has been achieved. This is facilitated by asking them to self-assess whether they have shown engagement with all of their feedback, shown insight into its content and taken required appropriate action for any needed improvement/development. By asking them to do this, Step 3 acts as a final cross-check and opportunity to reconsider their reaction and plans.
Step 4 (Figure 7)
Step 4 is optional and repeats the same questions as at Step 3. On this occasion, however, the assessment of Insightful Practice can be made by an independent reviewer or multiple reviewers. The purpose of this step is to offer involved organizations and/or regulators opportunities to decide priorities for placing any additional resources. For example, as facilitation is known to be important to encourage insight and help deliver required change,22 Step 4 could allow input and support to be given, perhaps following an existing system's planned coaching or appraisal interview, to encourage participants' successful engagement, insight and action with the process. Facilitation by coaching or appraisal systems is expensive, however, and agencies may wish to integrate the application of the method into their existing systems or use it to help target unmet areas of interest. Importantly, in addition to helping facilitate reflection (Steps 1–3), an additional Step 4 can provide a highly reliable overall surrogate assessment of professionalism by independent web-based assessors to provide challenge-resistant high-stakes decisions on progress if it be required.8,19 Given the human basis of an independent assessment of Insightful Practice at Step 4, future research into the validity, reliability and acceptability of including lay assessors would be interesting and widen the potential role of patients in regulatory processes.
THE MEASUREMENT OF INSIGHTFUL PRACTICE HAS BEEN SUCCESSFULLY PILOTED IN TAYSIDE, SCOTLAND IN THE CONTEXT OF GENERAL (FAMILY) PRACTITIONERS AND MEDICAL STUDENTS.
Applications to date
The measurement of Insightful Practice has been successfully piloted in Tayside, Scotland in the context of general (family) practitioners and medical students.8,19 The system is used by the University to assess and longitudinally monitor the progress of medical students who have been otherwise identified as showing either lapses in professionalism or evidence of being in academic difficulty. In addition to its use with students who are having problems, it is also being adapted to help underpin the reflection and provide an audit trail for all medical students on their continuous professional development.
Potential applications
While accurate self-reflection can be difficult when we are challenged by adversity, stress, complaints, or health and social matters, it can be all the more difficult at a time when we need to see problems clearly and strategically to help get our careers back on track. Many of us will be able to recall “a word in your ear” by a trusted professional confidant that was important at some key time of our careers. The system formalizes the process and allows its use at a number of levels. It can be used to maximize the benefit of credible, independent feedback and monitor those who, while not “bad apples,” may be showing signs of becoming of concern and in need of support. This system can build on and add value to regulatory authorities and other agencies' current systems to support physicians in difficulty. In cases where all efforts have been exhausted, measurement of Insightful Practice by blinded and independent assessors offers robust additional evidence to help inform decisions on continued fitness to practice.8,19
Potential benefits
Professional reflection based on self-driven assessment of needs and current standing is not enough. It risks the high achievers' collecting ever more data to meet unrealistic targets and those with poor insight into their needs collecting confirmatory evidence of their perception of their standing. Reflection based on credible and independent evidence of performance is of key importance and offers physicians a cross-check on their self-assessment of progress. In addition, physicians' adequate and appropriate response to independent feedback to highlight opportunities for quality improvement may well highlight deficiencies unobtainable by other means. The system is adaptable to work context and roles and so can offer support to existing regulation methods and associated systems — including professional appraisal, examinations and professional accreditation — to maximize performance and quality of health care. Measurement of Insightful Practice offers a reliable means by which regulatory systems could cross-check the validity of their existing regulatory decisions. In addition, it offers a practical method to scrutinize individual medical professionals identified as being of some concern, provide an audit trail of their progress and help inform decisions on professional outcomes. Insightful Practice offers a basis for the surrogate monitoring of professionalism from outset to end of career. It offers medical schools a system to monitor and highlight professional behaviour problems upstream of adverse impact on patients. This is of particular importance, given increasing calls for medical schools to record an audit trail of their input to any decisions on licensure for practice at the end of training.
Potential limitations
Professionals are capable people and gaming a way around any system can be a problem for its durability. Ironically, from our studies to date with both family practitioners and medical students, lack of insight into problems can appear to preclude the capacity to recognize that it would be prudent to change course. Where insight is lacking and when feeling threatened by what they perceive as unwelcome and unfair scrutiny, physicians could “agree on the face of it” with their feedback without intending to engage with change in any meaningful way. If so, the longitudinal monitoring of progress would reveal future difficulties. Future experience will be important to establish if these early experiences are valid. The presented method of assessment is designed to encourage and focus on needed change and log progress in performance. It is not designed to facilitate an interactive one-to-one conversation with subjects for the purpose of wider personal support — for example, the discussion of health issues. This would have to be provided separately by alternative methods, such as the services of an occupational health organization. However, a twin-track approach may sometimes be needed, when both occupational health support and the early monitoring of provided standards of care are needed if risks are to be successfully managed and patients protected.
Implications for medical regulation
For medical regulation, suggesting to physicians that they “keep up the good work” is not enough. At the same time, suggesting that new regulatory systems should not be implemented because they cannot be guaranteed to catch potentially harmful physicians is also not good enough. Agencies need to work together and be clear of their agreed responsibilities and boundaries or health professionals who put patients at risk will be able to game their way onwards, unchecked, until their behavior finally catches up with them, patients are actually harmed and significant costs are incurred. An applied system based on the measurement of Insightful Practice offers a useful and simple basis to help national systems develop their future plans, meet their contextual needs and provide robust and reliable systems to keep health care professionals on track and to maintain public protection.
Conclusion
The measurement of Insightful Practice is a simple concept. It is based on a long-recognized human observation that it can be difficult to see ourselves as others see us. The authors assert that the measurement of Insightful Practice is practical, flexible, and can be adapted to any given context, or professional work role. The system can help support those amber-zone physicians whose practice falls into the regulatory gap of being neither good nor bad enough for either revalidation programs or fitness-to-practice systems. It offers a reliable surrogate assessment of professionalism that could hold valuable application for health care organizations, regulatory authorities, higher educational institutions, professional organizations and professional defense organizations, all of whom have a shared interest in achieving quality and safety in the provision of international health care. If it were adopted, agencies could develop their own system based on the concept of Insightful Practice, consider involvement with an external resource to meet their requirements, or develop an alternative robust system. The status quo is not an option. The challenge will be ensuring that risk within the current regulatory gap does not prevail by being allowed to fall between ill-defined areas of responsibility, for we are all accountable and pay the cost. We will all be patients one day.
Appendix
A workbook of illustrative examples of Insightful Practice is available at: https://www.tipportfolio.co.uk/InsightfulPractice.htm.
About the Authors
↵Douglas James Murphy, MD, FRCGP, is Senior Clinical Research Fellow, Quality, Safety and Informatics Research Group, University of Dundee.
↵Peter Davey, MD, FRCP, is Lead for Clinical Quality Improvement, Medical Education Institute, University of Dundee.
↵Eleanor Jane Hothersall, MD, FFPH, is Honorary Senior Clinical Lecturer and Consultant in Public Health, Ninewells Hospital and Medical School, Dundee.
↵Fiona Muir, EdD, MEd, is Senior Lecturer, Centre for Undergraduate Medicine, Medical Education Institute, University of Dundee.
↵David A. Bruce, MBChB, FRCGP, is Director Postgraduate General Practice Education, East of Scotland Deanery, Ninewells Hospital, Dundee.
↵ *In the UK, medical defense is provided by not-for-profit organizations that indemnify and provide legal representation for members for incidents arising from their clinical care of patients.
↵ **UK Responsible officers (ROs), usually a senior clinician, are accountable for the local clinical governance processes in their NHS health care organization with a focus on the conduct and performance of doctors. Duties include evaluating a doctor's fitness to practice, and liaising with the GMC over relevant procedures. The ROs' role is to make recommendations, but the decision on whether a doctor should be revali-dated belongs to the GMC, as the regulator.
Data Sharing Statement
The concept of Insightful Practice was invented and first described by Douglas J. Murphy, MD, FRCGP, as were all the materials and figures contained in this paper. Anyone wishing to use any of the materials in this paper should contact Dr. Murphy at [email protected] or [email protected].
Dr. Murphy was funded in the development of all the ideas and concepts described in this paper by a Primary Care Research Career Award provided by Chief Scientist Office, Scottish Government.
- Copyright 2015 Federation of State Medical Boards. All Rights Reserved.
References
- 1.↵Federation of State Medical Boards (FSMB) Mission Statement. http://library.fsmb.org/foundation2.html Accessed February 16, 2015.
- 2.↵General Medical Council (GMC) Statement of Strategy 2014 — 2018. https://www.medicalcouncil.ie/News-and-Publications/Publications/Strategy-/Statement-of-Strategy-2014-2018/Statement-of-Strategy-2014-2018-.pdf Accessed February 16, 2015
- 3.↵General Medical Council, Working with Doctors Working for Patients. http://www.gmc-uk.org/about/role.asp Accessed February 16, 2015.
- 4.Medical Schools Council, London. Selecting for Excellence, Final Report. http://www.medschools.ac.uk/SiteCollection-Documents/Selecting-for-Excellence-Final-Report.pdf Accessed February 16, 2015.
- 5.Medical Schools Council, London, Selecting for Excellence. http://www.medschools.ac.uk/AboutUs/Projects/Widening-Participation/Selecting-for-Excellence/Pages/Selecting-for-Excellence.aspx Accessed February 16, 2015.
- 6.↵GMC Revalidation Information. http://www.gmc-uk.org/doctors/revalidation.asp. Accessed February 16, 2015.
- 7.↵Smith J. The Shipman enquiry — Fifth Report: Safeguarding Patients: Lessons From the Past — Proposals for the Future. 2004. http://www.shipman-inquiry.org.uk/_fthreport.asp. Accessed February 16, 2015.
- 8.↵Murphy DJ , GuthrieB, SullivanFM, MercerSW, RussellA, BruceDA. Insightful practice: a reliable measure for medical revalidation. BMJ Qual Saf2012;21:649–656doi:10.1136/bmjqs-2011-000429.
- 9.↵General Medical Council. Monitoring and evaluating revalidation. http://www.gmc-uk.org/doctors/revalidation/9610.asp. Accessed November 8, 2015.
- 10.↵General Medical Council. GMP Framework for Appraisal and Revalidation. http://www.gmc-uk.org/doctors/revalidation/revalidation_gmp_framework.asp. Accessed February 16, 2015.
- 11.↵Davis DA , MazmanianPE, FordisM, Van HarrisonR, ThorpeKE, PerrierL. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094–1102. American Journal of Pharmaceutical Education 2010; 74 (5) Article 85.6.
- 12.↵Ward M , GruppenL, RegehrG. Measuring self-assessment: current state of the art. Adv Health Sci Educ. 2002;7:63–80.
- 13.↵Kruger J , DunningD. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77(6):1121–1134.
- 14.↵Eva KW , RegehrG. Self-assessment in the Health Professions: A Reformulation and Research Agenda. Acadmic Medicine, Vol 80, No 10/ October2005Supplement. S46–S54.
- 15.↵Shojania KG , Dixon-WoodsM. “Bad apples”: time to redefine as a type of systems problem? BMJ Qual Saf 2013;22:528–531doi:10.1136/bmjqs-2013-002138.
- 16.↵Kaplan R. The clinicide phenomenon: an exploration of medical murder. Forensic Psychiatry2007, Vol. 15, No. 4, 299–304doi:10.1080/10398560701383236
- 17.↵Wu J. A piece of my mind. Recertification. JAMA2010; 303:309–10.
- 18.↵The Scottish Government. Professionalism in nursing, midwifery and allied health professionals in Scotland: a report to the Coordinating Council for the NMAHP Contribution to the Healthcare Quality Strategy for NHS Scotland. 3July2012. http://www.gov.scot/Resource/0039/00396525.pdf Accessed March 31, 2015.
- 19.↵Murphy DJ , AitchisonP, Hernandez-SantiagoV, DaveyP, MiresG, NathwaniD. Insightful Practice: a robust measure of medical students' professional response to feedback on their performance. BMC Medical Education2015, 15:125doi:10.1186/s12909-015-0406-2.
- 20.↵Chamberlain JM. The sociology of medical regulation: an introduction. New York and Amsterdam: Springer, 2012.
- 21.↵University of Iowa: SMART Goal Setting. http://www.dso.iastate.edu/asc/academic/handouts/goal/smart.pdf. Accessed February 16, 2015.
- 22.↵Sargent JM , MannKV, van der VleutenCP, et al. Reflection: a link between receiving and using assessment feedback. Adv Health Sci Educ Theory Pract2009;3:399–410.
