Linguistic analysis of highly formal texts: Detecting stance in intelligence agency reports

Government agencies frequently document their work and inform the public via highly formalised reports, which potentially reveal their stance towards various societal groups and issues. We demonstrate that such stance can be detected using textual analysis. This in turn is relevant considering the impact of such agencies on policymaking and the public discourse. In this work, we examine yearly reports (2015-2021) of the German Federal Domestic Intelligence Agency (Bundesverfassungsschutz/BfV), which report on the developments of domestic extremism.

In this study, we investigate the stance, extent of reasoning as well as speculative nature of the different reports per chapter on the extreme left and the extreme right. To this end, the reports are examined using LIWC, a dictionary-based method which classifies words into various psychological categories. It can capture psychological states of the writer. We analyse the aggregated counts per category and chapter.

The results indicate discrepancies between the texts: we have found higher levels of cognitive processes indicating complex reasoning, greater tentativeness and increasing certainty simultaneously, as well as a higher focus on the future on the left. However, the emotional dimensions are almost even. Thus, neither of the political scenes is characterized in a particularly negative or positive fashion, yet differences remain.

To contextualise the LIWC analysis, we select a subset of sentences that allow us to make a direct qualitative comparison between chapters on right-wing and left-wing extremism. We focus on sentences which report on plausibly verifiable real-world developments and events such as changes in number of reported crimes. Ultimately, this selection enables an improved understanding of stance through the following qualitative analysis. The latest and the oldest available report were used to extract keywords indicating possibly relevant sentences. Based on the keywords a sample of sentences was analysed. Overall, we found more dramatizing descriptions for developments on the left and a higher degree of tentativeness with socially desirable developments whereas violent incidents of the right are often prefaced by indicating violent potential.

Taken together, the chapters on the left and on the right differ, which can be interpreted as follows. The complex writing for the left could be necessary to justify dramatizing wording. The latter naturally supports warnings directed towards the future, in turn explaining the future-focus but also tentativeness. In summary, we see more speculation and warning about the left although reality may not support such formulations compared to the political right. However, the explanations are rather speculative, and it remains unclear how the increased certainty can be explained.

In conclusion, through our novel approach we have shown that there are indications of bias in favour of the right in the BfV reports, which is concerning considering the agency’s role and function. This finding is surprising given the similarity in terms of topic, high formalism and supposedly large number of authors. Our results are interesting because of LIWC’s design-focus on personal writing. Based on our exploratory work, we recommend investigating the specific relation between the extracted LIWC analysis, and the trends highlighted through the qualitative analysis .