Abstract
In response to the proliferation and abuse of opioids in the United States, the U.S. Food and Drug Administration (FDA) recently required manufacturers of extended-release and long-acting (ER/LA) opioid analgesics to provide education for prescribers of these medications through an FDA-approved risk management program. As a part of its Risk Evaluation and Mitigation Strategy (REMS), the FDA provided a blueprint for content to help accredited continuing medical education (CME) providers develop educational modules for prescribers. The University of Nebraska Medical Center (UNMC), the Federation of State Medical Boards (FSMB), the FSMB Foundation, The France Foundation, and CECity worked collaboratively to create “Extended-Release and Long-Acting Opioid Analgesics,” a prescriber education program aligned with the FDA blueprint, comprised of live meetings and web-based enduring activities that spanned April 2014 to February 2016. A total of 4,535 health care providers participated in the live and enduring REMS CME activities and completed evaluations of these activities. The evaluations were analyzed to learn more about the impact of the content on prescribers' knowledge and clinical decision-making. Results were similar for live and enduring activities. Participant-knowledge increased by 22 percentage points. Several competence measures suggest that learners returned to practice with more confidence, and 83% declared specific practice changes they planned to make to improve patient assessment and care. More than one-third of the evaluation respondents felt there were no barriers to making intended practice changes. More than 95% of the respondents said the activity increased their ability to apply knowledge, skills, and judgment in practice. Analysis suggests that live and enduring educational activities, following the FDA REMS blueprint, can impact prescriber attitudes and future care-decisions related to ER/LA opioid analgesics. These results justify additional efforts to involve more participants in these REM activities.
Introduction
What is REMS?
A Risk Evaluation and Mitigation Strategy (REMS) is a safety strategy to manage a known or potential serious risk associated with a medicine, and may be required as part of the FDA's approval of a new product, or for an approved product when new safety information arises. The Opioid Analgesics Risk Evaluation and Mitigation Strategy (REMS) is designed to ensure that the benefits of ER/LA opioid analgesics outweigh the risks. The FDA Blueprint for Prescriber Education (“FDA Blueprint”) provides a detailed content outline for accredited CE providers offering REMS education.1 In response to the recent proliferation and abuse of opioids in the U.S.,2,3 the FDA required manufacturers of extended-release and long-acting (ER/LA) opioid analgesics (collectively known as the REMS Program Companies) to provide education for prescribers of these medications through the FDA's approved risk management program.
According to the FDA REMS, education includes the following core goals4:
Understand how to assess patients for treatment with ER/LA opioid analgesics.
Be familiar with how to initiate therapy, modify dose, and discontinue use of ER/LA opioid analgesics.
Be knowledgeable about how to manage ongoing therapy with ER/LA opioid analgesics.
Know how to counsel patients and caregivers about the safe use of ER/LA opioid analgesics, including proper storage and disposal.
Be familiar with general and product-specific drug information concerning ER/LA opioid analgesics.
The University of Nebraska Medical Center (UNMC), the Federation of State Medical Boards (FSMB), the FSMB Foundation, The France Foundation, and CECity collaborated to develop live and enduring REMS educational activities for health care providers who prescribe ER/LA opioid analgesics. The education was funded by a grant from the REMS Program Companies.
To fulfill the FDA's requirements regarding REMS education and to meet the educational needs, UNMC, FSMB, and the collaborators on this project worked with a steering committee to develop a set of core content slides used in both the live and enduring activities to ensure a consistent educational message across the different formats. The slides covered the REMS-compliant content outlined in the blueprint. Several content updates were made throughout the activity implementation period to keep the education current as new ER/LA products and doses were approved and new warnings were added to labels.
Methods
To implement the live educational programming, the collaborators developed a program to award grants to 25 state medical boards. Eligibility for grants included commitment to recruit at least 250 ER/LA opioid prescribers* to participate in a two-hour live educational session.
THE OPIOID ANALGESICS RISK EVALUATION AND MITIGATION STRATEGY (REMS) IS DESIGNED TO ENSURE THAT THE BENEFITS OF ER/LA OPIOID ANALGESICS OUTWEIGH THE RISKS.
For the enduring activity, the collaborators developed six 30-minute online modules, following the FDA content outline. Online modules were accessible through FSMB's web site (www.fsmb.org/safeprescribing) and were hosted by CECity.
Learners who completed all six modules, scored 75% on the post-test, and completed an activity evaluation could obtain AMA PRA Category 1 Credit(s)™ or AOA Category 2B credits. Each module covered one section of the FDA blueprint, touching on these subject areas:
Assessing Patients for Treatment
Initiating Therapy, Modifying Dosing, and Discontinuing Use
Managing Therapy
Counseling Patients and Caregivers about Safe Use
General Drug Information
Specific Drug Information
The content was created by a Steering Committee chosen for their expertise in the area of responsible opioid prescribing and with strong leadership ties to the Federation of State Medical Boards. The Steering Committee members were Humayun J. Chaudhry, DO, MACP, CEO and President, Federation of State Medical Boards, Euless, Texas; Janelle A. Rhyne, MD, MACP, President, Federation of State Medical Boards Research and Education Foundation (FSMB Foundation), Past Chair, Federation of State Medical Boards, Wilmington, North Carolina; and William L. Harp, MD, Executive Director, Virginia Board of Medicine, Henrico, Virginia.
The collaborators recruited learners over the duration of the live meetings and web-based activities, spanning April 2014 to February 2016. Given the broad spectrum of health care providers who prescribe opioids, the educational activities targeted a multi-disciplinary, interprofessional audience of prescribers. However, the primary audience for the program was clinicians who are registered with the DEA, eligible to prescribe Schedule 2 and 3 drugs, and have written at least one ER/LA opioid prescription in the past year. All education was offered free to learners. For live activities, the state medical board hosting the meetings was responsible for recruiting learners. For online activities, the collaborators worked together to continuously raise learner awareness over the duration of the CME accreditation period. Tactics included emails, promotional articles for state medical board newsletters, articles in the FSMB biweekly e-newsletter and the CECity monthly e-newsletter, and announcements on collaborator web sites and in the FSMB Journal of Medical Regulation. FSMB sent electronic marketing kits to all 70 state medical boards as well as printed marketing kits to 70 board executive directors.
Participants in both the live and online activities answered knowledge and competence questions to assess the impact of the REMS activities. These assessments aligned with the REMS FDA Blueprint.
Results
Twenty-three (23) state medical boards applied for a total of 26 available grants but six boards withdrew their intent to participate, citing an inability to recruit the required number of ER/LA opioid prescribers, turnover in staff and/or other legislative priorities of the board. Several states applied for and were awarded multiple grants, based on their anticipated high number of learners and need to do additional activities to reach them all. In the end, 22 grants were issued to 17 state medical boards to deliver the live REMS education (Table 1).
Federation of State Medical Boards Awarded Grants
Forty-one (41) live activities were implemented from June 2014 through April 2015. Of the live activity learners, 3,104 submitted written evaluations for CME credit, but not all learners completed every question.
Of the 3,104 live learners, 3,016 identified their profession, with 82% identifying themselves as physicians (Table 2). Most of the remainder consisted of advance practice nurses (APNs) or physician assistants (PAs). There were 18 nurses and eight pharmacists in the “other” category. Evaluations submitted by enduring activity learners showed a similar distribution. Physicians represented almost 90% of prescribers in both activities.
PARTICIPANTS IN BOTH THE LIVE AND ONLINE ACTIVITIES ANSWERED KNOWLEDGE AND COMPETENCE QUESTIONS TO ASSESS THE IMPACT OF THE REMS ACTIVITIES.
Learner Professions
Nearly 89% of the live activity learners (2,665/3,005) were licensed by the FDA to prescribe Schedule 2 and/or Schedule 3 drugs. Of this group, 67% (1,791) wrote at least one ER/LA opioid prescription in the past year (prescribers) with physicians comprising 86% of prescribers in the live activity.
Learners completing the enduring activity submitted 1,431 evaluations. Almost 80% of learners who took the online activity and completed the evaluation were licensed ER/LA opioid prescribers, and 59% (646) wrote at least one ER/LA opioid prescription in the past year. Physicians comprised 88% of the prescribers in the enduring activity.
Table 3 shows that more than half of all live meeting attendees identified themselves as primary care providers. Only about 7% of the learners were pain specialists. Approximately 39% of the learners identified themselves as “other,” including psychiatry, surgery, emergency medicine, hospitalist, OB/GYN, anesthesiology, oncology, occupational health, palliative care, rheumatology, rehabilitation, geriatrics, hospice, and other professional categories.
Learners by Medical Specialty
Experienced practitioners with more than 25 years in practice accounted for approximately one third of the live activity learners, as seen in Figure 1. In contrast, the largest group of learners in enduring activities had five years or less in practice.
Number of Years in Practice Percent of Respondents
Approximately half the learners managed 15 or fewer patients for acute or chronic pain. Prescribers attending the live meeting showed the most uniform distribution among numbers of patients treated for pain, while enduring activity participants were skewed toward a low number of patients (almost half treating ?5 patients) (Figure 2). This analysis suggests that the live activities reached more patients receiving ER/LA opioids, since the patient number distributions among prescribers are similar, and in fact there were approximately two times as many prescribers at the live activities.
Patients Managed for Pain
Learners in both live and enduring activities reported high satisfaction with the REMS activities, with 97% agreeing that it met their educational needs. Almost half the live activity prescribers (42%) said the format was appropriate as presented, but 30% said more case-based presentations would improve the format, and nearly 30% said the activity could be improved by adding more learner interactivity through things such as audience response keypads and breakout sessions. More than 95% of the respondents said that the activity increased their ability to apply knowledge, skills, and judgment in practice.
LEARNERS IN BOTH LIVE AND ENDURING ACTIVITIES REPORTED HIGH SATISFACTION WITH THE REMS ACTIVITIES, WITH 97% AGREEING THAT IT MET THEIR EDUCATIONAL NEEDS.
Learners at the live activity completed a 12-question test before the activity and the same 12-question test after the activity. Learners in the enduring activity completed the same 12-question pre-test. After each module, they answered a post-test consisting of the two questions related to that module. Learners making incorrect responses received reinforcement in the form of teaching points of the module. Learners were given three opportunities to retake the test, but the post-test values reported here are the results of learners' first attempts.
ALMOST HALF THE LIVE ACTIVITY PRESCRIBERS (42%) SAID THE FORMAT WAS APPROPRIATE AS PRESENTED, BUT 30% SAID MORE CASE-BASED PRESENTATIONS WOULD IMPROVE THE FORMAT.
The average increase in knowledge was 20% for the live activity and 25% for the enduring activity. The question “Which is the strongest indicator of risk in the Opioid Risk Tool?” had a high pre-test value (86% live, 92% enduring) and the change in correct answers for this question was not significant for any of the groups. Correct responses increased significantly for the remaining questions as well as the whole test (t-test P < 0.001). Test results were very similar for prescribers at both the live and enduring activities.
Test results for live and enduring REMS activities
Numbers in the first column indicate the relevant FDA goal. Intensity of color in the change columns signifies the size of the change. “Prescriber-only” data was very similar to “all learner” data, so data is combined.
Nurses, PAs, physicians, and APNs scored between 73% and 77% correct on the post-test. This includes all participants who answered at least one post-test question. If only learners answering 10 or more questions are included, the percent correct was almost the same; there was an increase of 1% for nurses, physicians, PAs, and APNs; the percent of correct answers for dentists increased by 3%.
OVERALL, THE PRESCRIBERS IN THE LIVE ACTIVITY WHO RATED THEMSELVES AS CONFIDENT OR VERY CONFIDENT INCREASED FROM 25% BEFORE THE ACTIVITY TO 59%. LEARNERS IN THE ENDURING ACTIVITY ALSO REPORTED INCREASED CONFIDENCE.
Learners in the live activity used a seven-point scale to rate how confident they were before and after the activity (0 = not at all, 4 = moderately, 7 = very confident). There was an increase in confidence for each of the five skills (Chi-square P < 0.0001) as seen in Figure 3. Learners had the least confidence in managing pain with ER/LA opioid analgesics. Overall, the prescribers in the live activity who rated themselves as confident or very confident increased from 25% before the activity to 59%. Learners in the enduring activity also reported increased confidence, from 19% before the activity to 37% afterwards using the same Likert-type confidence scale. (Figure 4).
How confident are you ... (Live Activity)
How confident are you ... (Enduring Activity)
Intent-to-change was assessed in the live activity evaluation (see Table 5). Of the 2,953 learners who answered the question, “Do you intend to make changes or apply new knowledge to your practice as a result of this activity?” 71% answered “Yes”; 18% answered “I'm not sure, but I'm considering changes”; and 11% answered “No, I already practice these recommendations.” A list of specific changes was offered (including a free text option) and the learners chose an average of 11.5 intended actions each.
Intent-to-Change Selections by Management Category
More than 70% of the learners in the enduring activity also intended to make practice changes. Among prescribers, 81% (637) intended to make changes, while 61% (752) of the non-prescribers intended to make changes.
Several intended practice changes were selected more frequently by live activity participants than enduring activity participants. Those that differed by 10% or more included:
Use a validated questionnaire to assess patient risk for aberrant drug-related behavior (61% live vs. 51% enduring).
Use a patient counseling document to guide appropriate medication use (51% live vs. 40% enduring).
Refer patients for addiction and abuse treatment as needed (56% live vs. 43% enduring).
Share lessons from this REMS education with colleagues (63% live vs. 48% enduring).
Learners were asked what barriers they perceive in implementing changes. The most commonly chosen barrier was “time to assess/counsel patients” (42% live, 27% enduring). Approximately one-third (30% live, 39% enduring) said there were no barriers.
This REMS activity did not contain performance improvement or long-term survey components. However, in the activity evaluation more than 90% of the learners said that the activity would improve their performance, and almost 84% of them said the activity would improve their patient outcomes.
Discussion
REMS education is one facet of the response to widespread opioid abuse in the United States. The impact of REMS education is difficult to assess, but there are indications that the death rate from opi-oids, as well as the opioid prescription volume, decreased after REMS initiation.5
A total of 4,535 health care providers completed our live and enduring REMS CME activities. Offering both live and online activities provided different learning options. Data shows that the experienced practitioners appear to prefer live format, while younger physicians appear to prefer online. Recruitment of learners was a challenge and participation in both the live and enduring activities was much lower than anticipated. We aimed to educate 8,750 prescribers. This shortfall may be due to an abundance of education on opioids, the length of the activities, or confidence of practitioners in their ability to use opioids wisely. Given the nature of the opioid crisis, future activities should target everyone who prescribes any type of opioids, not just ER/LA prescribers.
The documented numbers of learners may underestimate the reach of the REMS initiative, as some learners may not have sought CME credit and others may have participated in selected portions of the activities. We observed significant attrition during the enduring activity: only 58% of those who started the activity continued to completion. This could have been due to the three-hour time commitment required to complete and an online audience consisting of 47% prescribers. However, this compares favorably to other REMS activities. The REMS Program Companies estimated that only 36% of the 438,000 participants who began a REMS activity completed it.6
The post-activity evaluations asked how many patients participants manage for chronic pain; 49% of the enduring activity participants manage five or fewer patients for chronic pain, compared with 31% of the live activity participants. Perhaps providers managing fewer patients for pain are deterred from live meetings by a larger perceived time burden. Although the content of the activities is the same, learners can participate in the enduring activity in segments, according to their own schedules. On the other hand, live activities allow for discussion, interaction with colleagues, and access to experts. We estimate that more patients were reached by the live activities than the enduring activity.
...IN THE ACTIVITY EVALUATION MORE THAN 90% OF THE LEARNERS SAID THAT THE ACTIVITY WOULD IMPROVE THEIR PERFORMANCE, AND ALMOST 84% OF THEM SAID THE ACTIVITY WOULD IMPROVE THEIR PATIENT OUTCOMES.
Pre-test and post-test values for prescribers and the entire participant group are very similar and learners had a large and significant average increase in knowledge, with an average increase of 20% for the live activity and 25% for the enduring activity. The slightly higher post-test values for the enduring format might be a result of the testing sequence. Since the enduring post-test questions were answered immediately after completion of the module, the education was probably more immediate than in the live activities.
Several post-activity measures suggest that learners from both the live and enduring activities returned to their practices with increased competence. Their confidence increased in the ability to assess and manage patients with ER/LA opioids and they identified multiple specific actions they intended to take to improve patient assessment and care. We did not conduct a follow-up study to assess whether participants implemented their intended changes or whether there were changes in patient outcomes subsequent to the ER/LA REMS activities.
The education was developed by a collaborative, with each entity bringing different strengths. The University of Nebraska Medical Center offered leadership and content expertise. CECity provided technical acumen and hosted the enduring modules. The France Foundation created the activity content and assessment tools; it also provided data analysis and project management. The Federation of State Medical Boards provided direct access to the target audience of health care providers. The greatest recruitment success occurred when learners heard about the activities from a trusted source, such as the FSMB. Additionally, we learned that participation is higher when the individual state boards had strong internal advocates encouraging providers to complete the education. Four state associations lacked a strong internal champion and returned their grants due to insufficient participation.
There were several challenges in implementing these activities. The curriculum outlined in the FDA Blueprint was overambitious for the allotted two to three hours and some of the live activity sites divided the presentation into multiple sessions. The extensive FDA outline required a presentation dominated by didactic material. The FDA Blueprint did not permit CME providers the flexibility to present the content using more engaging, interactive formats. Activities were audited to ensure the content mapped to the strict blueprint. More cases and more focus on non-pharmacologic and non-opioid management of chronic pain may be more compelling than specifics of ER/LA drugs that occupy approximately one-third of the current FDA Blueprint (also noted by Traynor, et al.).6
Conclusion
The continued rise of opioid abuse warrants continued REMS education. This education should include prescribing of both short-acting and ER/LA opioids. Analysis suggests that live and enduring educational activities, following the FDA REMS blueprint, can impact prescriber attitudes and future care-decisions related to ER/LA opioid analgesics. Education should extend beyond prescribers to reach the entire care team and merits reorganizing the format to break the education into smaller, more digestible segments and including more learner engagement.
While this study had a limited scope, it demonstrates that gathering data about REMS training is an important element of course construction. Future studies could focus more specifically on topics such as the attrition rate of participants, correlations between demographic factors and knowledge retention, and reliability and validity testing of live and online processes.
About the Authors
↵Daniel U. Rabin, PhD, is Medical Director at The France Foundation.
↵Heather K. Tarbox, MPH, is Director of Education at The France Foundation.
↵Lois Colburn is Executive Director at the University of Nebraska Medical Center for Continuing Education.
↵Kelly C. Alfred, MS, is Senior Director of Education Services at the Federation of State Medical Boards.
↵ *The FDA definition of an “ER/LA opioid prescriber” is “an individual who is registered with the DEA, eligible to prescribe Schedule 2 and/or Schedule 3 controlled substances, and has written at least one ER/LA opioid script in the past year.”
- Copyright 2017 Federation of State Medical Boards. All Rights Reserved.
References
- 1.↵U.S. Food and Drug Administration. Extended-Release (ER) And Long-Acting (LA) Opioid Analgesics Risk Evaluation And Mitigation Strategy (REMS). Accessed at: http://www.fda.gov/downloads/Drugs/DrugSafety/PostmarketDrugSafetyInformationforPatientsandProviders/UCM311290.pdf.
- 2.↵Substance Abuse and Mental Health Services Administration. 2011. Results from the 2010 National Survey on Drug Use and Health: Detailed Table, Table 7.1.a. Rockville, MD.
- 3.↵Warner M , ChenLH, MakucDM, AndersonRN, MiniñoAM. Drug poisoning deaths in the United States, 1980–2008. NCHS data brief, no 81. Hyattsville, MD: National Center for Health Statistics. 2011.
- 4.↵ER-LA-opioidrems.com. ER/LA Opioid Analgesics REMS: The extended-release and long-acting opioid analgesics risk evaluation and mitigation strategy. Accessed May 2017 at http://www.er-la-opioidrems.com/IwgUI/rems/training.action.
- 5.↵Cepeda MS , CoplanPM, KopperNW, MaziereJY, WedinGP, WallaceLE. ER/LA opioid analgesics REMS: Overview of ongoing assessments of its progress and its impact on health outcomes. Pain Med2017; 18: 78–85doi: 10.1093/pm/pnw129.
- 6.↵Traynor K. Advisers say FDA's opioid REMS program needs improvement. Am J Health Syst Pharm. 2016;73(13):940–942.
