3IA PhD/Postdoc Seminar #29

Flash presentation

AI for e-health: Stress Identification from Multimodal Data

We present StressID, a new dataset specifically designed to support learning pipelines for stress identification from unimodal and multimodal data. It contains videos of facial expressions, audio recordings, and physiological signals. Its experimental setup ensures a synchronized and high-quality multimodal data collection. Different stress-inducing stimuli, such as emotional video clips, cognitive tasks including mathematical or comprehension exercises, and public speaking scenarios, are designed to trigger a diverse range of emotional responses. The final dataset consists of recordings from 65 participants, representing more than 39 hours of annotated data in total, and is one of the largest datasets for stress identification. In addition to the dataset, we provide several baseline models for stress recognition, including multimodal predictive baselines that combine video, audio, and physiological inputs and highlight the significant advantage of multimodal learning. We investigate how to build robust and reliable models for stress identification using StressID, by focusing on aspects such as learning with unevenly represented modalities.

Generalised mutual information (GEMINI) - A constellation of discriminative clustering models

Clustering is a fundamental learning task which consists in separating data samples into several categories, each named cluster. In the last decade, recent successes displayed examples of clustering using neural networks. These works can be considered through the broad spectrum of discriminative models involving mutual information (MI) as an objective function. However, MI has shortcomings that we address by proposing a novel definition: the generalised mutual information (GEMINI). By incorporating distances or kernel between data samples for guidance, GEMINI can be used for training various discriminative models ranging from neural networks to decision trees for clustering. These models can be applied to different data types: tabular, images, graphs... using the associated package GemClus.

Open discussion on the two contributions