Artificial Intelligence in Emergency Toxicology: Comparative Readability Assessment of Methanol Poisoning Information Generated by ChatGPT-5 and Gemini 2.5 Pro
Background: Large language models (LLMs) are increasingly used to generate patient education materials and provide accessible medical information. However, the readability of AI-generated health information remains a critical concern, particularly for time-sensitive toxicological emergencies such as methanol poisoning, where delayed recognition and treatment can result in severe morbidity or death.
Objective: This study aimed to compare the readability of patient education materials on methanol poisoning generated by ChatGPT-5 and Gemini 2.5 Pro using multiple validated readability indices.
Methods: A cross-sectional comparative study was conducted using standardized prompts submitted independently to ChatGPT-5 and Gemini 2.5 Pro. Fifty responses were generated by each model, yielding a total of 100 patient education texts. Readability was evaluated using the Flesch Reading Ease Score (FRES), Flesch–Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Simple Measure of Gobbledygook (SMOG), Coleman–Liau Index (CLI), Automated Readability Index (ARI), Dale–Chall Readability Score (DCRS), Linsear Write Formula (LWF), and Spache Readability Formula (SRF). Statistical analyses were performed using independent-samples t-tests or Mann–Whitney U tests, with statistical significance set at p < 0.05.
Results: ChatGPT-5 generated significantly higher FRES values than Gemini 2.5 Pro (48.64 ± 7.72 vs. 38.84 ± 6.62; p < 0.001), indicating greater reading ease. Conversely, Gemini 2.5 Pro demonstrated significantly lower FKGL scores (5.74 ± 3.02 vs. 8.94 ± 2.64; p < 0.001), suggesting that its texts required a lower educational level for comprehension. No statistically significant differences were observed between the two models for ARI, GFI, CLI, SMOG, DCRS, LWF, or SRF (all p > 0.05), indicating broadly comparable overall linguistic complexity.
Conclusion: ChatGPT-5 and Gemini 2.5 Pro produced patient education materials with similar overall readability profiles but differed in specific readability characteristics. ChatGPT-5 generated texts with greater reading ease, whereas Gemini 2.5 Pro produced materials more closely aligned with the recommended educational reading level for patient education. Despite these advantages, neither model consistently achieved optimal readability across all indices. AI-generated educational materials should therefore undergo expert review before clinical use to ensure accessibility, clarity, and scientific accuracy, particularly for emergency conditions such as methanol poisoning.
Keywords: Methanol poisoning; Large language models; ChatGPT-5; Gemini 2.5 Pro; Artificial intelligence; Patient education; Readability; Health literacy; Emergency toxicology.
