AI model successfully detects patients’ emotions during therapy sessions, even the brief ones.

Researchers have discovered that an AI-powered system has the capability to identify facial expressions that convey various emotions, including momentary ones like happiness and sadness. This finding holds potential in assisting psychotherapy treatments, according to the researchers.

The AI system the researchers used in the study was a freely available artificial neural network trained to detect six basic emotions – happiness, surprise, anger, disgust, sadness and fear – using more than 30,000 facial images. Artificial neural networks are a type of machine learning, a subfield of artificial intelligence, and are built on the principles of the connections of biological neural networks in animal brains.

The researchers then created an AI model process and analyzed more than 950 hours of video recordings of therapy sessions with 23 patients with borderline personality at the Center for Scientific Computing at the University of Basel in Switzerland.

An international team compared the analyzes generated by the model with those of three trained therapists and found “significant agreement.”

They said that in addition to being able to measure patients’ facial expressions as reliably as a trained therapist, the model was also able to detect more fleeting emotions that occurred for less than a millisecond, such as a brief smile or disgust.

The AI was thus more sensitive than the therapists to momentary expressions of emotion that the therapists might have missed or that could only be detected subconsciously, the team said. They have published their findings in the journal Psychopathology.

“We wanted to find out whether artificial intelligence systems can reliably determine patients’ emotional states from video recordings,” said Martin Steppan, a psychologist at the Faculty of Psychology at the University of Basel and lead author of the study.

The researchers also found that analysis of the pattern revealed another trend—patients who showed emotional involvement by smiling at the beginning of a therapy session were more committed to psychotherapy than those who appeared emotionally indifferent to their therapist.

Smiling may therefore be a “good predictor” of the success of therapy sessions in people with borderline personality disorder, they said.

“We were really surprised to find that relatively simple AI systems can match facial expressions to their emotional states so reliably,” Steppan said.

AI could therefore become an important tool in therapy and research and could help support supervision of psychotherapists, the team said, although they added that currently “therapeutic work is still primarily relational and remains a human domain”.