Published on September 19, 2025
–
Updated on September 19, 2025
Dates
on the September 2, 2025
Location
Suzhou, China
The institute takes pride in announcing that two articles cosigned by Paolo Papotti have been accepted at EMNLP25.
➊ “SQUAB: Evaluating LLM's robustness to Ambiguous and Unanswerable Questions in Semantic Parsing”
By Luca Cagliero (associate professor at Politecnico di Torino), Paolo Papotti (3IA Chairholder and associate professor in the Data Science department at EURECOM), and Simone Papicchio (Ph.D. student at EURECOM)
Abstract: Large Language Models (LLMs) have demonstrated robust performance in Semantic Parsing (SP) for well-defined queries with unambiguous intent and answerable responses. However, practical user questions frequently deviate from these ideal conditions, challenging the applicability of existing benchmarks. To address this issue, we introduce SQUAB, an automatic dataset generator of Ambiguous and Unanswerable questions. SQUAB generates complex, annotated SP tests using a blend of SQL and LLM capabilities. Results show that SQUAB reduces test generation costs by up to 99% compared to human-based solutions while aligning with real-world question patterns. Furthermore, these tests challenge LLM performance while revealing disparities between public and proprietary datasets. This highlights the need for a dynamic, automatic dataset generator as SQUAB. The code is designed for user extension to accommodate new ambiguous and unanswerable patterns.
➋ “Refining Attention for Explainable and Noise-Robust Fact-Checking with Transformers”
By Jean-Flavien Bussotti (Research Associate at Megagon Labs) and Paolo Papotti
Abstract: In tasks like question answering and fact-checking, models must discern relevant information from extensive corpora in an "open-book" setting. Conventional transformer-based models excel at classifying input data, but (i) often falter due to sensitivity to noise and (ii) lack explainability regarding their decision process. To address these challenges, we introduce ATTUN, a novel transformer architecture designed to enhance model transparency and resilience to noise by refining the attention mechanisms. Our approach involves a dedicated module that directly modifies attention weights, allowing the model to both improve predictions and identify the most relevant sections of input data. We validate our methodology using fact-checking datasets and show promising results in question answering. Experiments show up to a 51% improvement in F1 score over state-of-the-art systems for detecting relevant context, and up to an 18% gain in task accuracy when integrating ATTUN into a model.
Learn more about EMNLP25
By Luca Cagliero (associate professor at Politecnico di Torino), Paolo Papotti (3IA Chairholder and associate professor in the Data Science department at EURECOM), and Simone Papicchio (Ph.D. student at EURECOM)
Abstract: Large Language Models (LLMs) have demonstrated robust performance in Semantic Parsing (SP) for well-defined queries with unambiguous intent and answerable responses. However, practical user questions frequently deviate from these ideal conditions, challenging the applicability of existing benchmarks. To address this issue, we introduce SQUAB, an automatic dataset generator of Ambiguous and Unanswerable questions. SQUAB generates complex, annotated SP tests using a blend of SQL and LLM capabilities. Results show that SQUAB reduces test generation costs by up to 99% compared to human-based solutions while aligning with real-world question patterns. Furthermore, these tests challenge LLM performance while revealing disparities between public and proprietary datasets. This highlights the need for a dynamic, automatic dataset generator as SQUAB. The code is designed for user extension to accommodate new ambiguous and unanswerable patterns.
➋ “Refining Attention for Explainable and Noise-Robust Fact-Checking with Transformers”
By Jean-Flavien Bussotti (Research Associate at Megagon Labs) and Paolo Papotti
Abstract: In tasks like question answering and fact-checking, models must discern relevant information from extensive corpora in an "open-book" setting. Conventional transformer-based models excel at classifying input data, but (i) often falter due to sensitivity to noise and (ii) lack explainability regarding their decision process. To address these challenges, we introduce ATTUN, a novel transformer architecture designed to enhance model transparency and resilience to noise by refining the attention mechanisms. Our approach involves a dedicated module that directly modifies attention weights, allowing the model to both improve predictions and identify the most relevant sections of input data. We validate our methodology using fact-checking datasets and show promising results in question answering. Experiments show up to a 51% improvement in F1 score over state-of-the-art systems for detecting relevant context, and up to an 18% gain in task accuracy when integrating ATTUN into a model.
Learn more about EMNLP25
