Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
Change search
ExportLink to record
Permanent link

Direct link
BETA

Project

Project type/Form of grant
Other
Title [sv]
AI i hälso- och sjukvården: Ansvarsfull och etisk implementering av Large Language Model-chatbottar i kliniska flöden och vård
Title [en]
AI in Healthcare Unleashed: Responsible and Ethical Implementation of Large Language Model Chatbots in Clinical Workflows and Patient Care
Abstract [sv]
FrågeställningSedan OpenAI:s ChatGPT lanserades i november 2022 har denna artificiella intelligens (AI) blivit enormt omskriven. ChatGPT är ett exempel på en Large Language Model (LLM) som kan generera innehåll baserat på träning av stora datamängder samt algoritmer. LLM:er kan sammanfatta komplexa data och anpassa sina svar till önskad stil eller läskunnighetsnivå. Denna teknik är särskilt relevant i vårdsammanhang, i synnerhet när det gäller patienters tillgång till sin journal via nätet, en innovation där Sverige varit ledande globalt. Detta skulle kunna underlätta för överbelastad vårdpersonal som önskar hjälp med dokumentation. Men LLM kan också föra vidare fel och fördomar. Syftet med detta tvärvetenskapliga projekt är att undersöka om LLM kan användas för att minska dokumentationsbördan i vården och hjälpa patienter. Projektet har 3 mål:MÅL 1: Att undersöka vårdpersonalens erfarenheter och förståelse av LLM i klinisk dokumentationMÅL 2: Att undersöka patienters erfarenheter av och förståelse för LLM när det gäller att tolka sina hälsodata och journalinformationMÅL 3: Att undersöka hur vården på ett etiskt och säkert sätt kan använda dessa verktyg för att gynna personal och patienter Data och metodProjektet kommer att undersöka deltagarnas erfarenheter och åsikter med hjälp av enkäter som analyseras kvantitativt och kvalitativt (mål 1&2). Data för mål 1 kommer att samlas in genom en rikstäckande enkätundersökning till vårdpersonal; för Mål 2 kommer patienters åsikter av inhämtas genom en rikstäckande enkätundersökning via den nationella patientportalen 1177.se. En triangulering av resultaten och etisk analys genomförs för att nå Mål 3, följt av en 3-rundors Delphi-studie online med experter för att nå konsensus om rekommendationer för ett etiskt och säkert införande av LLM i hälsodokumentation. Samhällsrelevans och nyttiggörandeI december 2023 föreslog EU världens första regler för LLM-chattbottar och generativ AI, vilken innebär att konsumenterna måste få fullständig information om att de interagerar med en AI. Ett grundläggande problem är att vi inte vet när eller hur vårdpersonal och patienter använder LLM eller hur LLM kan integreras etiskt, säkert och effektivt i vården. GenomförandeProjektet kommer att delas in i 5 arbetspaket, utformade för att effektivt kunna hanteras av 7 tvärvetenskapliga forskare plus 1 doktorand och stödjas av en internationell rådgivande patientgrupp och en rådgivande LLM-grupp.
Abstract [en]
Research problem and specific questionsFollowing the release of OpenAI’s ChatGPT in November 2022, this artificial intelligence (AI) has never been out of the headlines. ChatGPT is an example of a Large Language Model (LLM) that can generate content based on training on large data sets & algorithms. LLMs can summarize complex data and write responses in a requested conversational style or literacy level. This technology is particularly relevant in a healthcare context especially the era of patient online record access (ORA), an innovation Sweden has been at the forefront of worldwide & where overburdened healthcare professionals (HCPs) desire assistance with documentation. However, LLMs also embed errors & biases. The purpose of this timely interdisciplinary project is to investigate whether LLMs might be used to alleviate documentation burdens & assist patients. The project has 3 aims:AIM 1: To investigate Swedish HCPs’ experiences & understanding of LLMs in documentation in clinical practiceAIM 2: To investigate patients’ experiences & understanding of LLMs in interpreting their health dataAIM 3: To investigate how healthcare might ethically & safely adopt these tools to benefit HCPs & patientsData and method: The project will survey the experiences & opinions of participants using mixed-methods online surveys for AIM1&2. AIM1 will leverage a nationwide survey of HCPs that is already being conducted; AIM2 will use a nationwide survey through the national patient portal 1177.se, managed by Inera AB, to solicit patients’ opinions. AIM3 will triangulate results and involve ethical analysis. AIM3 will also use an online 3-round Delphi methodology to poll experts for recommendations on the ethical, safe adoption of LLMs in health documentation.Societal relevance and utilisation: In December 2023 the EU proposed the world’s first rules for LLM chatbots with makers of generative AI stipulating consumers must be fully informed that they are engaging with AI. A fundamental problem is we do not know when or how HCPs & patients are using LLMs nor how LLMs can be ethically, safely & effectively integrated into healthcare.Plan for project realisation: The project will be divided into 5 work packages designed to be efficiently & effectively managed by a complement of 7 interdisciplinary research personnel plus 1 Ph.D. student & supported by an international Patient Advisory Group & an LLM Advisory Group.
Publications (3 of 3) Show all publications
Blease, C., Jones, J., Blease, C. E., Kharko, A., Garcia Sanchez, C. & D. Mandl, K. (2026). Patient and Family Perspectives on Generative AI Tools in Rare Diseases: Exploratory Mixed Methods Online Survey. Journal of Participatory Medicine, 18, Article ID e93720.
Open this publication in new window or tab >>Patient and Family Perspectives on Generative AI Tools in Rare Diseases: Exploratory Mixed Methods Online Survey
Show others...
2026 (English)In: Journal of Participatory Medicine, E-ISSN 2152-7202, Vol. 18, article id e93720Article in journal (Refereed) Published
Abstract [en]

Background: Generative artificial intelligence (GenAI) tools are widely accessible to the public, who are engaging with them for a wide range of health care applications. Existing research has focused predominantly on clinician-facing adoption. Far less is known about how patients and family members use GenAI tools, particularly in rare disease contexts, where diagnostic delay, limited specialist access, and unmet informational needs are common.

Objective: This study aimed to examine the experiences and opinions of adult patients with rare diseases and parents or guardians of children with rare diseases regarding the use of GenAI tools.

Methods: Between November 2025 and January 2026, we conducted an exploratory mixed methods web-based survey using convenience sampling through rare disease community organizations in the United States. The survey included closed-ended items assessing prior GenAI use, purposes of use, perceived influence on medical decisions and diagnoses, trust, concerns, communication with clinicians, and experiences of harm, alongside open-text questions capturing qualitative reflections. Descriptive statistics were used to summarize quantitative data. Inductive qualitative analysis was applied to the open-text responses.

Results: A total of 115 respondents completed the survey. A majority of respondents were parents or guardians of a child with a rare disease (n=74, 64.3%), and the remaining respondents were patients with a rare disease (n=41, 35.7%). Slightly more than half of respondents (n=63, 54.8%) reported prior use of GenAI tools in the context of rare disease. Common purposes included exploring new treatments or clinical trials (n=53, 46.1%), interpreting medical tests or clinical notes (n=37, 32.2%), locating specialists or care centers (n=29, 25.2%), and suggesting possible diagnoses (n=28, 24.3%). Nearly one-third of respondents (n=37, 32%) reported some degree of influence of GenAI on their medical decisions. Nearly 10% (n=12) reported contributions of GenAI to a formal diagnosis. Concern about GenAI accuracy was widespread; 71 of 115 (61.8%) respondents reported moderate to extreme concern. Most respondents (n=90, 78.3%) had not discussed AI-generated information with a clinician. Few respondents (n=7, 6.1%) reported experiencing harm. Qualitative analysis identified 3 themes: (1) GenAI as a practical tool for augmenting patient and caregiver expertise and advocacy, (2) conditional trust and bounded use of GenAI with an emphasis on verification and human oversight, and (3) perceived risks, harms, and structural concerns, including inaccuracies, genetic misinterpretation, and privacy and commercialization issues.

Conclusions: In this exploratory study, patients and families affected by rare diseases were actively experimenting with GenAI tools to support information seeking, preparation, and advocacy while simultaneously expressing substantial caution and concern about the reliability, safety, and appropriate boundaries of use. Our findings contrast sharply with clinician concerns that patients lack the capacity to use GenAI tools judiciously. Notwithstanding, the sample was skewed toward highly educated participants. Future research should prioritize more representative samples to better capture the range of patient and caregiver experiences with GenAI in rare disease care.

Place, publisher, year, edition, pages
JMIR Publications, 2026
Keywords
generative AI, general practice, primary care, large language models, education, training, online survey question-naire, qualitative research
National Category
Nursing Artificial Intelligence
Identifiers
urn:nbn:se:uu:diva-596846 (URN)10.2196/93720 (DOI)001853583100001 ()42497841 (PubMedID)
Funder
Forte, Swedish Research Council for Health, Working Life and Welfare, 2024-00039
Available from: 2026-08-31 Created: 2026-08-31 Last updated: 2026-08-31Bibliographically approved
Blease, C., Hagström, J., Garcia Sanchez, C., Kharko, A., McMillan,  ., Gaab,  ., . . . Mandl,  . D. (2025). General practitioners’ adoption of generative artificial intelligence in clinical practice in the UK: An updated online survey. Paper presented at 2025/11/25. Digital Health, 11
Open this publication in new window or tab >>General practitioners’ adoption of generative artificial intelligence in clinical practice in the UK: An updated online survey
Show others...
2025 (English)In: Digital Health, E-ISSN 2055-2076, Vol. 11Article in journal (Refereed) Published
Abstract [en]

Background: Following the launch of ChatGPT in November 2022, interest in large language model-powered chatbots has soared with increasing focus on the clinical potential of these tools. Building on a previous survey conducted in 2024, we sought to gauge general practitioners? (GPs) adoption of this new generation of chatbots to assist with any aspect of clinical practice in the UK.

Methods: An online survey was disseminated in January 2025 to a stratified convenience sample of GPs registered with the clinician marketing platform Doctors.net.uk. The research was conducted as part of a scheduled monthly ?omnibus survey,? designed to achieve a fixed sample size of 1000 participants.

Results: Of the 1005 respondents, 50% respondents were men, 54% were 46 years or older. 25% reported using generative artificial intelligence (GenAI) tools in clinical practice; of these, 35% reported using these tools to generate documentation after patient appointments, 27% to suggest a differential diagnosis, 24% for treatment options, and 24% for referrals. Of the 249 GPs who used generative AI for clinical tasks, 71% said that, in general, these tools reduced work burdens. In the last 12 months, 85% reported that their employer had not encouraged them to use GenAI tools, but only 3% said their employer had prohibited them from using GenAI tools in their work; 95% reported they had no professional training in using GenAI tools in their work.

Conclusions: This survey suggests that doctors? use of GenAI in clinical practice may be growing in the UK. Findings suggest that UK GPs may benefit from these tools, especially for administrative tasks and clinical reasoning support, and after adopting them, most users reported a decrease in work burdens. Continued absence of reported training remains a concern.

Place, publisher, year, edition, pages
Sage Publications, 2025
National Category
Health Care Service and Management, Health Policy and Services and Health Economy
Identifiers
urn:nbn:se:uu:diva-572030 (URN)10.1177/20552076251394287 (DOI)001622615600001 ()41312147 (PubMedID)
Conference
2025/11/25
Available from: 2025-11-25 Created: 2025-11-25 Last updated: 2025-12-15Bibliographically approved
Garcia Sanchez, C., Kharko, A., Hägglund, M., Riggare, S. & Blease, C. (2025). Health Care Professionals' Experiences and Opinions About Generative AI and Ambient Scribes in Clinical Documentation: Protocol for a Scoping Review. JMIR Research Protocols, 14, Article ID e73602.
Open this publication in new window or tab >>Health Care Professionals' Experiences and Opinions About Generative AI and Ambient Scribes in Clinical Documentation: Protocol for a Scoping Review
Show others...
2025 (English)In: JMIR Research Protocols, E-ISSN 1929-0748, Vol. 14, article id e73602Article, review/survey (Refereed) Published
Abstract [en]

BACKGROUND: Generative artificial intelligence (GenAI) leverages large language models (LLMs) that are transforming health care. Specialized ambient GenAI tools, like Nuance Dax, Speke, and Tandem Health, "listen" to consultations and generate clinical notes. Medical-focused models, like Med-PaLM, provide tailored health care insights. GenAI's capability to summarize complex data and generate responses in various conversational styles or literacy levels makes it particularly valuable since it has the potential to alleviate the burden of clinical documentation on health care professionals (HCPs). While GenAI may prove to be helpful, offering novel benefits, it comes with its own set of challenges. The quality of the source data can introduce biases, leading to skewed recommendations or outright false information (so-called hallucinations). In addition, due to the conversational nature of chatbot responses, users may be susceptible to misinformation, posing risks to both safety and privacy. Therefore, careful implementation and rigorous oversight are essential to ensure accuracy, ethical integrity, and alignment with clinical standards. Despite these advances, currently, no review has investigated HCPs' experiences and opinions about GenAI in clinical documentation. Yet, such a perspective is crucial to better understand how these technologies can be safely and ethically adopted and implemented in clinical practice.

OBJECTIVE: We aim to present the protocol for a scoping review exploring HCPs' experiences and opinions about GenAI and ambient scribes in clinical documentation.

METHODS: This scoping review will be carried out following the methodological framework of Arksey and O'Malley and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Scoping Reviews) checklist. Relevant papers will be searched for in PubMed, IEEE Xplore, APA PsycInfo, CINAHL, and Web of Science. The review will include studies published between January 2023 and September 2025. Studies will be included that represent original peer-reviewed work that explores HCPs' experiences and opinions about the use of GenAI or ambient scribes for clinical documentation. Data extraction will include publication type, country, sample characteristics, clinical setting, study aim, study design, research question, and key findings. Study quality will be assessed using the Mixed Methods Appraisal Tool.

RESULTS: The results will be presented as a narrative synthesis structured along the key themes of the evidence mapped. Data will be collated and presented in charts and tabular format. Findings will be reported in a peer-reviewed scoping review.

CONCLUSIONS: This will be the first scoping review that considers HCPs' experiences and opinions about GenAI and ambient scribes in clinical documentation. The results will clarify how HCPs use-or avoid using-GenAI in daily health care work. This insight will help address perceived benefits, risks, expectations, and uncertainties. It may also reveal key research gaps in the field.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/73602.

Place, publisher, year, edition, pages
JMIR Publications, 2025
Keywords
AI, GenAI, ambient scribes, artificial intelligence, attitude, clinical documentation, generative artificial intelligence, health care professionals, scoping review
National Category
Health Care Service and Management, Health Policy and Services and Health Economy
Identifiers
urn:nbn:se:uu:diva-564723 (URN)10.2196/73602 (DOI)001549480500001 ()40779760 (PubMedID)
Available from: 2025-08-10 Created: 2025-08-10 Last updated: 2025-09-01Bibliographically approved
Riggare, Sara
Brulin, Emma
Hägglund, Maria
Kharko, Anna
Hagström, Josefin
Wohlin, Martin
Principal InvestigatorBlease, Charlotte
Coordinating organisation
Uppsala University
Funder
Period
2024-07-01 - 2027-06-30
National Category
Other Health Sciences
Identifiers
DiVA, id: project:9109Project, id: 2024-00039_Forte

Search in DiVA

Other Health Sciences

Search outside of DiVA

GoogleGoogle Scholar