Identifying Contradicted Findings in ACL Anthology Papers

Lab: Ubiquitous Knowledge Processing Lab (UKP), TU Darmstadt

Overview

The main objective of this project is to identify contradictions among different papers on the same topic within the ACL Anthology. Contradictions in scientific literature can mislead future research; this work aims to develop a model capable of detecting such inconsistencies automatically.

Contributions

• Manually annotated sentences extracted from ACL Anthology papers following a pre-defined annotation guideline
• Generated artificial sentence pairs that contradict sentences extracted from real papers using various methods, including ChatGPT, to construct a contradiction dataset for model training and evaluation
• Analyzed the impact of different prompting strategies on the quality of generated counter-claims
• Compared the performance of a model fine-tuned on the constructed dataset against a generic baseline model