Modelling Stance Detection as Textual Entailment Recognition and Leveraging Measurement Knowledge from Social Sciences

Stance detection (SD), an important task in natural language processing, concerns automatically determining the viewpoint of a text’s author towards a target. This viewpoint, also known as stance, can be “for”, “against” or “neutral”. SD can be considered a special case of textual entailment recognition (TER), a more generic natural language task that concerns determining whether a hypothesis follows from a premise, where three outcomes are possible (“entailment”, “contradiction” and “neutral”). By reformulating SD as determining whether a text (i.e., a premise) entails a stance towards a target (i.e., a hypothesis), SD becomes a TER problem. This offers two potential benefits. First, in typical SD research, a separate model is trained for each target to predict stances towards that specific target. The resulting models are therefore target-specific and, in general, do not perform well on new targets. In a TER setup, however, a hypothesis necessarily contains information about the target of interest. This allows a model to incorporate information about the targets when learning to predict stances. As such, TER-based models are not target-specific and may be applied to unseen targets. Second, labelled SD datasets are limited to a few predefined targets and languages. In contrast, TER datasets are not limited to specific targets and are available in more languages. Therefore, we can use additional TER datasets for model training when considering SD as TER. This benefit can be especially relevant when labelled SD datasets do not exist for a specific language.

To the best of our knowledge, no study has empirically investigated the idea of reformulating SD as TER. In this paper, we aim to provide an initial analysis into the effectiveness of this approach. Specifically, we use a dataset of Dutch political tweets, with the goal to predict stances towards traditional gender role division on both tweet and political party levels.

As far as we are aware, no labelled SD dataset exists for the Dutch language or for the target ``traditional gender role division''. Therefore, this test case is both relevant (i.e., this is a common problem for many non-English languages in SD research) and difficult (i.e., our model can only train on Dutch TER data and needs to predict stances towards an unseen target). Furthermore, to improve model performance, we leverage established survey measurement practices from social sciences where stance detection (or more accurately, stance measurement) is well-studied.

We focus on answering two questions: First, can a model trained on only Dutch TER data achieve good downstream stance prediction on both tweet and party levels? Second, does using survey instruments improve model performance?