3IA PhD/Postdoc Seminar #47

Flash presentation - Comprehensive Emotion Understanding and Applications

Abstract: This presentation marks the beginning of my PhD journey under the 3IA program and introduces the research project I will pursue over the next three years: Comprehensive Emotion Understanding and Applications. The goal of this project is to explore and develop computational approaches for analyzing human emotional and psychological states using artificial intelligence. In particular, the research will investigate how multi-modal signals—such as facial video (computer vision) and physiological responses (e.g., EEG)—can be integrated to enable robust and context-aware emotion recognition. I will outline the motivations behind this work, its potential applications in AI-driven healthcare and human-computer interaction, and the technical roadmap for building models that go beyond surface-level cues to capture the deeper dynamics of human affect.

Multi-stage CNN for fast registration of 3D preoperative CTs to 2D intraoperative X-rays

Abstract: Minimally invasive interventions often rely on live 2D X-rays for image guidance. Yet, anatomical localization and procedural accuracy can be enhanced by spatial alignment of these intraoperative X-rays with 3D preoperative computed tomographies (CTs). This 3D/2D registration problem is typically formulated as pose estimation of the X-ray source relatively to the CT, which is done by simulating synthetic X-rays from 2D projections of CT volumes. However, the optimization-based refinement used by the state-of-the-art deep learning approach takes several seconds, thus exceeding the allowed time budget in live image guidance. We propose LXPose (Live X-ray Pose estimation), a self-supervised multi-stage 3D/2D registration framework for real-time image guidance. LXpose removes the dependency on optimization and leverages a two-stage CNN trained with a projection loss to ensure high accuracy and computational efficiency. Moreover, we apply extensive data augmentation to mitigate the domain gap between simulated and real X-rays. Overall, LXPose yields comparable 2D registration error to the state-of-the-art method, while reducing inference time to 20ms, which demonstrates the potential of LXPose for real-time clinical deployment.

Diffusion-Based Quality Control of Medical Image Segmentations across Multiple Organs

Abstract: Medical image segmentation, powered by deep learning, has revolutionized automated analysis pipelines for large-scale population studies. However, state-of-the-art methods are prone to hallucinations that lead to anatomically implausible segmentations. With manual correction impractical at scale, automated quality control (QC) techniques have emerged to address the challenge. While promising, existing QC methods are designed for handling a single organ, which restricts their generalizability across different applications. To overcome this limitation, we propose no-new Quality Control (nnQC), a robust QC framework based on a diffusion-generative paradigm that self-adapts to any input organ dataset. Central to nnQC is a novel Team of Experts (ToE) architecture, where two specialized experts independently process an image and its predicted segmentation, generating a pair of independent embeddings, or opinions. A weighted conditional module combines the opinions to guide the sampling within a diffusion process, enabling the accurate generation of a spatially-aware pseudo-ground truth (pGT) used to predict QC scores. We evaluated nnQC on seven organs using publicly available datasets. By adapting the network through extracted dataset information, or fingerprints, and leveraging the proposed ToE framework, our results demonstrate that nnQC consistently outperforms state-of-the-art methods across all experiments, including cases where segmentation masks are highly degraded or completely missing, confirming it as a versatile and off-the-shelf QC solution across different organs.

Open discussion about the three contributions