Addressing Patient Questions on Spinal Muscular Atrophy: Performance of Large Language Models in a Genetic Context
Performance of Large Language Models for Counselling on Spinal Muscular Atrophy
Keywords:
Spinal Muscular Atrophy, Genetic Counseling, Artificial Intelligence, Health Information Quality, Large Language Models, Patient EducationAbstract
Objective: Large language models (LLMs) are increasingly used by the public to obtain medical and genetic information. Given the genetic complexity and public health relevance of spinal muscular atrophy (SMA), this study aimed to evaluate the quality, readability, and actionability of LLM-generated responses to SMA-related frequently asked questions (FAQs).
Methods: Fifteen SMA-related FAQs were identified in Turkish using Google’s “People Also Ask” feature and categorized into disease definition, genetic screening, and genetic diagnosis and treatment. Each question was submitted to the free versions of ChatGPT, Gemini, and DeepSeek. Responses were evaluated using the modified DISCERN instrument and a 5-point Likert scale for information quality; the Flesch–Kincaid reading ease and grade level for readability; and the Patient Education Materials Assessment Tool (PEMAT) for understandability and actionability.
Results: Median DISCERN scores were 3.00 across all LLMs, indicating moderate information quality, and there was no significant difference among models (p = 0.069). Readability differed significantly, with ChatGPT producing responses at a lower Flesch–Kincaid grade level than that of Gemini and DeepSeek (p = 0.001). PEMAT understandability and actionability scores varied by question category, with significant differences observed for questions on disease definition and genetic screening (p < 0.05).
Conclusion: LLMs generate SMA-related information with moderate quality and understandability; however, variability in readability, actionability, and topic-specific performance limits their suitability for standalone use in genetic counseling. While these tools may serve as supplementary educational resources, they should not replace clinician-led genetic counseling, particularly in contexts requiring individualized risk assessment and decision-making.
