Principles-based Approach to Regulation of Artificial Intelligence in Professional Work: Perspectives of Health Professionals' Regulators

Implications of AI for Regulators

The implications of this rapid shift for regulators of health professionals has been discussed but, in most cases, there has been limited specific action by regulatory bodies to manage current realities of AI and anticipate future trends. At the time of this research, no regulatory bodies have promulgated formal regulations to safeguard public interests as AI becomes more prevalent in professional work. Technology entrepreneurs designing AI systems for use in health care are currently working in a regulatory vacuum, and as a result "what is possible" is driving innovation, as opposed to "what is desirable". Regulators themselves are contemplating ways in which AI could be integrated in their own workplaces to optimize cost-effective and time-efficient regulatory processes (for example, triaging of complaints or assessing qualifications at entry-to-practice using AI driven systems). ⁸ Thus far, in a time where HiL-AI continues to be the dominant mode of AI being deployed in health care, the presence of an identifiable human overseer or decision-maker appears to have attenuated the urgency of generating a regulatory framework to guide responsible adoption of AI in the professions. As HoL-AI becomes more prevalent and acceptable, and no single identifiable human decision-maker is necessarily involved, the complexity of regulatory practice may increase significantly.

Optimism regarding the potential role of AI in professional work and daily life abounds. Technology continues to evolve rapidly, and the costs continue to drop as computational power and efficiency improve. In the recent past, "hallucinations" have plagued AI-driven systems, a function of the large-language modeling processes by which AI systems are trained. ⁹ Hallucinations are usually described as incorrect of misleading information presented as certain fact by AI, usually in response to faulty or incomplete training of the language model. ⁹ For example, an ambiguous tumour may be diagnosed as "benign" by AI due to insufficient training/data input to recognize borderline cases. Innovations such as closed system training have reduced frequency and risk of hallucinations, though not entirely removed them. ¹⁰ Newer AI models have begun to address issues such as algorithmic bias that have been concerns for those opposed to HoL models. ¹¹ Importantly, the rapid pace of AI evolution means that AI itself is improving and self-addressing concerns regarding its quality. Concerns regarding AI continue to grow. ¹² Even those deeply invested in the technology and responsible for its success have expressed grave reservations regarding its rapid evolution and unquestioned uptake, noting that dystopic futures of human enslavement by AI-powered robots and tools is neither fanciful nor far-fetched. ¹³

More pragmatically, health professionals themselves have highlighted significant concerns regarding AI uptake in their day-to-day work. First and foremost is the risk of de-skilling of the workforce. ¹⁴ As both HiL-AI and HoL-AI become more prevalent in health care work, professionals themselves start to forget how to perform foundational tasks. Like the experience of children who learn to do math using calculators (and who then forever after have difficulty doing mental calculations like tallying up how much a cart of groceries may cost), professionals who were once skilled and competent come to rely upon AI-powered tools and relatively quickly become deskilled. Even where HiL-AI systems are used, blind faith in the technology reduces the need for continuous self-assessment of competence. Further, in resource constrained environments where professionals are stressed and overworked, reliance on AI systems becomes the only way to get through a workday. As outcomes of HiL-AI and HoL-AI supported patient care approach and eclipse those of human-driven care, self-confidence in human clinical decision making and activities also declines. ¹⁵ Similarly, as wages and practical labour relations issues associated with a human workforce continue to become more complex, an AI-driven workplace becomes more attractive for employers, governments, insurers, and ultimately perhaps, the public.

The complexity of the current situation is significant. Despite this reality, few regulatory bodies have advanced significant work to address public, professional, governmental, employer, educator, and other concerns regarding how HiL-AI and HoL-AI can and should "fit" into the daily work of professionals. In the absence of clear regulatory guidance, a "wild west" philosophy dominates, one in which unencumbered technological innovation drives professional practice in a way that may (or may not) be beneficial for the public's safety and interests. Understanding regulators' perspectives on the regulation of AI in professional practice is essential to support mindful, proportionate, effective, and acceptable regulatory practice.

Research Objective

The objective of this research was to understand regulators' perspectives on the need for regulation of AI in professional work and to characterize the approaches being taken to support public safety and interest. For this research, both HiL-AI and HoL-AI were considered. Only AI used by professionals in the delivery of patient care and services was considered; the use of AI by regulators themselves to increase organizational efficiency and effectiveness was not a focus of this work. Similarly, the use of AI by health professionals for non-patient care activities (eg: to manage certain organizational or administrative tasks (for example, using AI to track inventory of supplies in a practice to then trigger automatic ordering from vendors) was not a focus. Instead, the objective of this research was to understand the ways regulators conceptualized "risks" and "benefits" of HiL-AI and HoL AI when it was used by health professionals for the delivery of patient-focused care and services formerly or traditionally delivered solely by humans with or without support from non-AI driven technologies (eg: calculators, word processing, spreadsheets, or the internet).

Research Method

Given the paucity of published protocols or regulations regarding adoption of AI by health professionals in their practice, an exploratory research method was identified as most appropriate to begin to examine this topic. An environmental scan of regulatory body websites across Canada, the UK, and the US was undertaken to identify regulatory bodies who had begun to consider, discuss or address this issue in their public-facing documentation. Regulated health professions selected for this scan were medicine, nursing, pharmacy, physical therapy, dentistry, and occupational therapy—each of these professions exist and are regulated in the three jurisdictions examined in this study. Other professions—for example, midwifery, paramedicine, or naturopathy—may not be regulated or recognized in the same manner across all three countries and were consequently not included in this study.

Based on the results of the environmental scan, individual regulatory bodies in specified jurisdictions (countries) or sub-national jurisdictions (provinces or states) were identified as having undertaken some kind of publicly identifiable work with respect to regulation of AI in professional work. Examples of such work included convening a forum for professionals to discuss AI in their practice, establishing a working group within the regulatory body to discuss regulation, or crafting a position paper on the topic for comment by the profession. Based on this, a purposive sampling method was undertaken in which different regulatory bodies were invited to participate in semi-structured interviews to discuss the evolution of their thinking and work. Identifying the most appropriate individual to invite to participate in this research was based on publicly available records—for example where a regulatory body had crafted a position paper regarding AI in professional work and invited comments, the person to whom the comments were to be directed was initially contacted to participate in this study.

A semi-structured interview guide was created for the interviews (see Appendix 1). Given the breadth of professions involved, the diverse geographic regions included, the different health systems represented, and the asymmetric progress of different regulators, there was considerable flexibility in the application of the semi-structured interview guide. This research was guided by a protocol (46079) approved by the Research Ethics Board (REB) at the University of Toronto, Canada. Given the relatively small size of the regulatory community, and the sensitive nature of regulatory work, the approved protocol included strong provisions to safeguard confidentiality of research participants, including provisions that prevent identification of participants by profession or geographic location in publications such as this. The research team recognized the potentially controversial nature of this work and to encourage full and frank discussion during interviews agreed to minimize risk of disclosure of participant identity by not disclosing any form of identifying information in public-facing documents.

Results and Discussion

In the initial AI-driven search of regulatory body websites in Canada, the US, and the UK, 316 regulatory websites were scanned for information regarding regulatory work focused on AI in professional practice (250 in the US, 60 in Canada, and 6 in the UK). An AI-enabled search tool was built and used to optimize efficiency of this search, built on the Perplexity© AI platform. A keyword searching strategy approach was used to filter content from regulatory websites: key words included artificial intelligence, regulation, AI, decision support, algorithm, machine learning, large language model, natural language processing, artificial general intelligence (AGI), and large-language model (LLM). Though not an exhaustive or comprehensive search list, these keywords provided a sufficient overview of activity within a public-facing regulatory website to facilitate identification of regulators who had begun to engage in work in this area.

Of the 316 websites scanned, 196 had publicly available documents indicating some degree of work, interest, or involvement in the issue of AI in professional practice. Of these 196 regulatory bodies, 36 were identified as having undertaken some form of meaningful consultation with external stakeholders (eg: members of the profession, employers, members of the public, educators, or others) focused on the formulation of some kind of framework for regulation of AI in professional work. These 36 regulatory bodies were defined as the sample frame for the study, based on their (relatively) advanced work in this area. Of these 36 regulatory bodies, an identifiable contact individual within the organization was available for 20 of them; for the remaining 16 a generic, non-individualized/non-specific email address was provided for those in the public interested in following up. Initial recruitment for interview participants was limited to these 20 organizations. Of these 20 organizations, 11 were in Canada, 5 were in the US and 4 were in the UK. Professions represented were dentistry (6/20), pharmacy (3/20), physical therapy (3/20), medicine (5/20) and nursing (3/20). Of the 20 individuals invited to participate in this research, 18 responded positively and were engaged in interviews: dentistry (6/18), pharmacy (2/18), physical therapy (3/18), medicine (5/18), and nursing (2/18).

All interviews were undertaken via Zoom or Microsoft Teams and were recorded and transcribed with permission of participants. No substantive conflicts of interest or previous relationships were disclosed or existed between research team members and interview participants. NVivo v15.1® software was used for qualitative analysis of transcripts. A constant-comparative analytical framework ¹⁶ was used to identify common themes across professions and jurisdictions to highlight evolving promising practices with respect to regulation of AI in health care professional work, mindful of the significant differences in national cultures, health systems, and professional cultures. All transcripts were reviewed independently by two reviewers who read and coded to consensus. All themes identified achieved consensus without recourse to a third reviewer.

Key themes that emerged from this research included:

All 18 participants in this research noted that, within their respective professions, the rapid emergence and deployment of AI-powered practice-focused tools was a concern. In all cases, practitioners themselves were asking the regulator for guidance, rules, or support in understanding how best to responsibly adopt AI in professional work, and whether regulators were involved in "authorizing" or "validating" technology entrepreneurs who were developing and selling products. All participants described concerns regarding the lack of capacity within regulatory bodies to actually understand AI technologies themselves—those involved in this work in regulation rarely had any particular technical or technological knowledge or skill with respect to the actual mechanics of AI itself, and most of them understood AI in professional practice to simply be a "black box" defying human understanding. Further, most participants in this research highlighted the reality that it was simply not clear who within a regulatory body actually had the expertise to lead regulatory change with respect to AI. Participants noted that it may be advantageous for practitioner-regulators to lead these discussions but that it may also be necessary for those with advanced legal training/skills to lead discussions, thought simultaneously those who worked in policy areas in regulatory bodies needed to take charge of the process, while it was also important for those with educational/competency assessment skills to lead discussions within regulatory bodies and with external stakeholders. The lack of clarity over who should be leading the regulatory response to AI resulted in delayed responses and inaction within regulatory bodies as the necessary expertise to knowledgably engage in leadership in this area was lacking. Cooperative leadership within a regulatory body (in which representatives from different departments or areas within the organization would co-lead development of regulations) was described as cumbersome, time-consuming and unlikely to succeed as the pace of change was so rapid. All participants in the study described concerns that the current structure of most regulatory bodies made it unclear as to who should lead regulatory responses to AI, how cooperation within regulatory bodies themselves could be hastened and nimbler, and how coalitions across the profession and across professions could be mobilized. As a result of these silos, regulation was unfortunately slow, cumbersome, and not keeping pace with technological advances—and as a result, technology innovators were leading the process with regulators, professionals, employers and educators all scrambling to keep up as best as they could.

All 18 participants in this research noted concerns within the regulatory community that regulation of AI could be seen as "scope creep," unnecessarily impeding technological innovation and slowing its adoption. They noted that, in this current era, skepticism regarding regulation of all sorts and concerns that is unnecessarily bureaucratic and expensive has heightened concerns by professional regulators about taking on new and challenging tasks such as regulation of AI. As a result, a prevailing sentiment across all participants in this study focused on the notion that, so long as AI remained as a "human-in-the-loop" form, there was sufficient regulation in place for the human decision-makers and overseers to obviate the need for additional regulation of the technology itself. Attempting to introduce new regulation focused on HiL-AI would be seen by the profession and the public as unnecessarily intrusive and potentially detrimental to public interest as it may slow the pace of adoption and innovation. The undesirability for regulation of HiL-AI was contrasted with the desirability to provide some kind of guidance for professionals themselves—the blunt instrument of regulation may not be appropriate or necessary, but the use of other approaches (eg: continuing education or guidance documents to enhance the knowledge and skills of professionals to better self-assess and self-manage responsible adoption of AI in their individual practices) was highlighted as both appropriate and preferable to introduction of new rules. Several participants expressed faith in existing legal systems (for example, tort law or civil lawsuits, or industry standards associations) to ensure the technology was developing appropriately and safety and explained that no regulatory body could possibly keep pace or supplant the risk of litigation and the power of industry standards associations to ensure safe and effective development of AI technologies.

Most participants noted regulatory concerns associated with deskilling of the professional workforce, even with HiL AI. They noted that conceptualization and measurement of professional "competency" (both at entry to practice and during a professional's lifetime) was becoming increasingly complicated and difficult to rationalize as HiL-AI proliferates. While noting this reality, no participant was able to describe any plausible reframing of understanding of "competency" within an HiL-AI driven workplace, highlighting instead the need for profession-specific research to help better understand the implications of this for skills, testing, and measurement of outcomes. Instead, participants in this study focused their attention on risk of harm to patients due to erroneous HiL-AI recommendations or hallucinations. So long as an identifiable professional made the final decision, that individual would be the focus of regulatory investigation and interest and would assume ultimate responsibility and/or liability for negative outcomes triggered by HiL-AI. Participants expressed confidence that existing systems of complaints, investigation, and discipline were sufficient to accommodate the proliferation of Hi- AI and that no significant modification to existing regulation was needed at this time.

All 18 participants in this study recognized that the era of solely HiL-AI in professional work was rapidly ending (or had already ended) and that going forward, HoL-AI would become more prevalent. Making the distinction that HoL-AI represents a more significant threat to public safety and interests than HiL-AI did not facilitate straightforward or practical solutions to the question of how regulatory bodies ought to respond. While simultaneously recognizing the potential risks associated with HoL-AI, participants also noted the infeasibility of regulatory bodies starting to regulate "machines" in addition to people—even if these machines were performing controlled and regulated acts that legislatively are the sole purview of regulated health professionals.

Described variously as a "conundrum," a "wicked problem," and "an impossible situation," participants expressed their concern that widespread uptake of HoL-AI in professional work would transcend any regulator's ability to regulate it effectively, transparently, and objectively. Regulators from the dental profession in particular noted, for example, that the use of AI to guide interpretation of radiographs had grown so prevalent, had resulted in such rapid deskilling of the workforce, and had such demonstrably superior outcomes than human-led interpretation that it was producing significant pushback to the idea that regulation of HoL-AI was necessary or desirable at all. Some participants noted that the most feasible path forward would likely be to continue to regulate human professionals who may or may not use AI in their daily practice, but where HoL-AI was providing substantially similar service or care to humans, that other systems—tort law, national standards, etc.—would have to be relied upon to safeguard public interests.

All participants in this study agreed that a rules-based regulatory approach to HiL-AI or HoL-AI was neither feasible nor desirable, given the rapid pace of evolution and the inherent limitations of regulatory bodies with respect to their processes and their remit. Where participants expressed greatest belief of valuable impact was in the development and articulation of guidance documents and principles to support practitioners in making better and more informed decisions as to how best to integrate HiL-AI and HoL-AI in their practices. This shift away from rules to guidance—from a regulatory to an educational philosophy—was universally seen as the most viable and practical path forward at the current time. Most of the participants in this study were already involved in generating profession—and jurisdiction-specific principles that could be used as the basis for education or self-reflection by practitioners to guide their own decision making.

Common principles that emerged across professions and jurisdictions included:

Disclosure

Ensuring that patients were aware when HoL-AI was being used was described by most participants as essential. There was less agreement over whether disclosure of HiL-AI was necessary or desirable, with some participants expressing concerns that such disclosure would be unhelpfully cumbersome and time consuming and would not necessarily provide patients with any additional context or helpful information.

Consent

All participants agreed that where HoL-AI was being used, some form of informed consent procedure should be utilized, one that went beyond simple disclosure. Several participants noted that this may become problematic over time: while the general value of informed consent was not contested, there were pragmatic concerns that in situations where there the only alternative to HoL-AI driven health care is no care at all, the value of consent may be called into question.

Transparency/Openness

Both disclosure and consent represent a practical form of transparency, a regulatory principle of importance with respect to AI that emerged from this research. The general understanding of transparency involves clarity in communication, and confirmation of understanding by those who may be receiving care that relies partially or wholly upon AI. The importance of level-appropriate communication and providing patients with opportunities to ask questions without judgment was identified as an important tool for safeguarding public interests.

Choice

Participants in this study highlighted challenging discussions they had engaged with regarding the inclusion of choice (or options) as a principle. The notion of choice speaks to core ethical beliefs in health professions regarding autonomy, non-maleficence, and justice, but participants noted the practical problems associated with foregrounding the principle of choice (or providing patients with the option to select AI-enabled care or chose human-led care instead). The reality of most healthcare workplaces today is somewhat more complex: participants noted that in some cases, health professionals themselves may not be aware of embedded AI (usually HiL-AI) in technologies and tools they use every day so may not be able to present choice as an option to patients. In other workplaces, there may simply be no other option that is feasible or available, other that rejecting care altogether. Particularly during this time of transition from HiL-AI to HoL-AI the inclusion of choice as a regulatory principle may be challenging to operationalize and difficult to justify.

Privacy

Across the multiple national and sub-national jurisdictions examined in this study, there are different approaches to safeguarding public interests with respect to privacy. In all these jurisdictions, some form of legislative approach exists in which data collected by health professionals in the course of their work has limitations placed on it with respect to sale, sharing, or transmission to those outside a circle of care—though the level of these safeguards may vary considerably. Participants in this study noted that, as a general approach, safeguarding and respect for privacy of patients' data was an important regulatory principle. The nature of AI-enabled systems means that abundant data is frequently collected and stored and in many cases these data may not necessarily be directly related to the specific health issue being addressed at that time. Further, the nature of large language models and AI-training requires more data to support better outcomes; thus, the data being collected and used in the delivery of AI-enabled care becomes the seedbank for future improvement in the AI technologies themselves. The ethical complexities this introduces with respect to ethical adoption of AI by practitioners is significant. While other principles such as transparency, disclosure, and consent can partially mitigate some of these ethical concerns with respect to privacy, the complexity of the technology itself suggests further work and greater clarity around reasonable privacy expectations is required. Participants in this study indicated this privacy principle was a particularly challenging one to navigate given its importance to patience, existing legislative requirements, and complexity of the AI technologies themselves.

Redundancy

Participants in this study generally recognized that HiL-AI was already proliferating in most professions' practices, and that HoL-AI was rapidly becoming both prevalent and accepted. From a regulatory perspective, this raised some concerns regarding the continuity of care in the event of technological failure. Electrical blackouts, natural disasters, system failures, and crypto-hacking are all significant risks for any technologically reliant industry; as health care becomes more reliant on AI, and as that reliance breeds deskilling of the workforce, there are concerns that in the event of technology breakdown problematic interruptions in care delivery may occur. Regulators in this study expressed their belief that, rather than focusing on the unrealistic goal of preventing deskilling, it was more effective from a public protection perspective to include redundancy/back-up system approaches as a principle, for both HiL-AI and HoL-AI.

Audit and Feedback

Human health professionals are accustomed to quality assurance and maintenance of competence systems that provide them with some form of external assessment and appraisal based on objective criterion regarding their performance in practice. Such assessments are the foundation for quality improvement. AI-enabled technologies are usually built in a manner that continuously draws upon a growing pool of data that allows it to "learn," develop, and improve; arguably these systems have audit and feedback loops embedded in their design. Nonetheless, regulators in this study noted that potential value of a regulatory principle emphasizing the importance of this process, in ways that were respectful of patients' privacy concerns, but which allowed for external benchmarking and comparisons to facilitate improvement and to ensure attainment of minimal performance expectations and standards expected of human-led health care.

Code of Ethics

All health professions represented in this study function within a regulatory environment that identifies both standards of practice and rely upon a code of ethics to guide practitioner conduct. The intersection of standards and the code has long been a feature of regulated health professions and helps practitioners and the public understand the "what," the "how," and the "why" behind professional work and decision making. Regulators in this study noted that much of the AI-enabled technology in health professions work appears to align with professional standards of practice but it is more challenging to determine alignment with existing codes of ethics. Ethical principles such as autonomy, non-maleficence, beneficence, fiduciary responsibilities, etc. may appear abstract and intangible but healthcare professionals are guided by these ideals in their daily interactions with patients. Currently, where the majority of AI in health care is of the human-in-the-loop variety, there may be a complementarity between the superior computational power of AI and the ethical reasoning of humans. As HoL-AI becomes more prevalent, regulators noted that it would be essential that these technologies also embedded ethical reasoning and problem-solving process/algorithms that aligned with professional expectations.

Adversarial Interoperability

This term refers to competition amongst technology providers and entrepreneurs to ensure no single technology or system establishes a monopoly. The risk of a monopolistic AI provider or developer was deemed significant by participants in this study. Options such as open-source coding or multiple vendor platforms were identified as an essential safeguard to prevent monopolistic practices that could adversely impact both patients and the professions themselves.

This study identified the 9 themes described above as the most discussed and explored topics for regulators interested in a principles-based, rather than rules-based, approach to responsible adoption of AI by healthcare professionals in practice. Not all principles may apply equally to all professions, and some professions may consider additional or other principles that are less relevant in some contexts. The challenge of a principles-based approach to regulation lies in making it applicable and relevant to an entire profession without diluting its impact precisely because it is so wide-spread. The consensus amongst participants in this study was that a rules-based approach would not be feasible given the rapidity of evolution of AI, its current ubiquity and anticipated growth, and the complexity of the technology itself. Some participants also noted the futility associated with a regulatory body even attempting to "take on" powerful technology companies and entrepreneurs using available regulatory tools. All participants noted the current socio-political climate in which public faith in regulation may be tenuous, leading these participants to take a more incremental, cautious, and less combative/antagonistic approach.

The regulatory challenge associated with AI could be significant, if regulators decide this is an object of regulatory interest and a priority. The initial AI-enabled environmental scan that led to identification of research participants suggests that many regulators have not yet reached the point where regulation of AI is seen as desirable, necessary, or a priority. Those that have appear to have opted for incremental, principles-based approaches that favour guidance, education, and support for practitioners to make their own decisions (rather than imposing regulation on them). Within the current context of human-in-the-loop AI systems predominating, this may be appropriate, since in most situations there will still be an identifiable human practitioner who is licensed/regulated who can be the object of regulatory scrutiny in case of problems. This may change rapidly as HoL-AI becomes more prevalent—but until that time arrives, regulators continue to evolve their thinking, processes, and approaches to this issue.

While this research represents a preliminary and exploratory examination of this issue, there are important limitations to consider. For efficiency purposes, an AI-powered initial environment scan was undertaken, and given limitations of AI itself that have been described above, the accuracy and thoroughness of this scan may be questioned. The objective was to identify individuals who had expertise in both regulation/regulatory work and AI; it is not clear if the screening method used was able to fully achieve this objective. Further, it is not clear that such expertise in both regulation and AI technologies exists within or outside regulatory bodies. The semi-structured interview method described was useful for eliciting perspectives from individual regulators, but caution must be exercised in framing this data as any kind of "consensus" or "answer" to regulatory questions. Finally, the rapidity of the evolution of AI in health care is such that perspectives will shift rapidly, and this data may already be out of date.

Supplemental Material

• Download figure
• Open in new tab
• Download PowerPoint

Appendix 1.

Appendix 1