A new study indicates large language models may detect dimensions of chronic pain that conventional severity scales fail to capture. The research team analyzed patient accounts from interviews, uncovering connections between physical discomfort, emotional suffering, and psychological factors that numeric ratings alone cannot reveal.
How LLMs Examine Pain Beyond Traditional Ratings
The findings appeared in npj Digital Medicine, based on interviews with 131 individuals—67 suffering from chronic low back pain, 33 diagnosed with major depressive disorder, and 31 healthy volunteers. They employed OpenAI’s Whisper for transcription and Meta’s Llama-3-405B-Instruct to evaluate responses across nine dimensions: physical pain intensity, emotional distress, daily functioning, catastrophizing thoughts, depressive symptoms, anxiety levels, repetitive negative thinking, perceived helplessness, and narrative incoherence.
All participants, aged over 40, took part in semi-structured Zoom discussions where they described their experiences in open-ended terms. The LLM-generated assessments aligned with established clinical evaluations, offering clearer distinctions between chronic pain, depression, and healthy controls than simple word-count analysis.
For instance, those with chronic low back pain registered the highest scores in physical pain categories, while individuals with major depressive disorder exhibited greater impairment in life quality, emotional pain, and perceived helplessness. The model’s capacity to distinguish these subtleties implies it could improve pain assessment beyond basic severity measurements.
Statistical analysis using Spearman’s rho showed 73% of comparisons between LLM scores and patient self-reports were significant. Machine learning models combining perceived helplessness and repetitive thinking achieved an F1 score of 0.851, demonstrating strong group differentiation, while models using perceived helplessness and narrative fragmentation reached 0.71.
Read Also: New Medical Codes Take Effect January 1
A separate review of Reddit discussions replicated these trends: posts about depression revealed raised psychological distress, whereas chronic pain discussions emphasized physical symptoms. However, online data produced systematically higher scores, suggesting that writing style and unverified self-reports may introduce bias.
Challenges and Clinical Concerns
The study acknowledges significant hurdles. Current LLMs lack extensive, varied datasets of pain narratives, and differences in education, language background, or emotional state could distort results. Dr. Hofer noted that large language models do well when trained on a large volume of well-labeled text, but these do not exist today. Few large, diverse datasets contain narrative descriptions of a patient’s pain, and how a patient describes their pain likely varies based on factors such as their primary language, level of education, and emotional state. Thus, while in theory an LLM can create a much more robust and standardized way of measuring a patient’s pain, the underlying data for training are currently missing.
Dr. Jotwani, who was not part of this study, also added that if clinicians routinely defer to LLM narrative summaries, “we risk eroding the deep listening and interpretive skills that create what is known as the ‘therapeutic alliance.’ We know from the literature that physician empathy directly influences chronic pain outcomes, and that relationship cannot be outsourced without cost. If used to guide treatment decisions, these tools function as clinical decision support and should be evaluated with the same rigor we would demand of any intervention we bring into the room.”
The research also warns of unintended effects: clinicians who depend on LLM-generated insights might overlook subtle patient signals that human judgment captures.
Potential Impact on Pain Evaluation
Traditional pain scales, such as the 0-to-10 numeric rating system, have faced criticism for oversimplifying the experience. They do not account for how pain interferes with daily activities, co-occurs with mental health issues, or responds to interventions. In contrast, LLMs could process unstructured patient language to uncover these complexities—provided they are trained on high-quality, representative data.
