When Machines Learn to Feel

Changes in heart rate can provide information about physical and emotional well-being. 
Photo Credit: © RUB, Kramer

In addition to linguistic prompts, large language models can also understand, interpret, and adapt their responses to heart frequency data. Dr. Morris Gellisch, previously of Ruhr University Bochum, Germany, and now at University of Zurich, Switzerland, and Boris Burr from Ruhr University Bochum verified this in an experiment. They developed a technical interface through which the physiological data can be transmitted to the language model in real time. The AI can also account for subtle physiological signals such as changes in heart activity. This opens new doors for use in medical and care applications. The work was published in the technical journal Frontiers in Digital Health.

Gellisch and Burr have developed a special interface through which heart data can be fed into the AI. 
Photo Credit: © RUB, Kramer

Table and visualization of the data – no problem

For their experiment, the two researchers used a common device that measures heart rate variability via a chest strap. The data acquired from this were decoded, filtered, and condensed. In real time, the processed heart data was fed into the large language model GPT-4. In response to a corresponding prompt, the AI was able to correctly display the transmitted heart data in a table containing average values, minimum, maximum, and other information. “Evaluation and visualization occur directly in the LLM environment, without any external statistical or plotting software,” says Burr. On request, the AI also visualized the measured heart data.

“We also tested our system in a range of real-time interaction scenarios, including an experiment on cognitive stress, in which the language model adapted its output based on heart frequency reactions to minor and major demands,” reports Gellisch. The system was able to identify differences in the heart frequency patterns between tasks of low and high cognitive demand, and then react to these in the AI output. “Our experiment is not a validated psycho-physiological study, but rather a proof-of-concept scenario,” Gellisch emphasizes. The goal was to show that the language model can respond to physiological parameters in real time when they are input via the developed interface.

The Dean of Studies for Medicine, Prof. Dr. Thorsten Schäfer, Morris Gellisch, Boris Burr, and Prof. Dr. Eckart Förster, Head of the Department of Neuroanatomy and Molecular Brain Research (from left to right)
Photo Credit: © RUB, Kramer

Added value for various tasks

“The integration of physiological signals in AI offers clear added value,” the two authors' state. Language models can thus consider text inputs as well as real-time indicators of autonomous condition, which could lead to more adaptive and context-sensitive interactions when learning and making decisions, and in healthcare. “The interface is not only interesting for education and research scenarios, but also for medical and healthcare applications, such as identifying stress, exhaustion, or emotional dysregulation in real time,” says Gellisch.

The project was developed with assistance from Professor Thorsten Schäfer, Dean of Studies of the Faculty of Medicine, and Professor Eckart Förster, Director of the Chair for Neuroanatomy and Molecular Brain Research at Ruhr University Bochum.

Published in journal: Frontiers in Digital Health

Title: Establishing a real-time biomarker-to-LLM interface: a modular pipeline for HRV signal acquisition, processing, and physiological state interpretation via generative AI

Authors: Morris Gellisch, and Boris Burr

Source/Credit: Ruhr University Bochum

Reference Number: ai101725_01

Privacy Policy | Terms of Service | Contact Us