Modeling offensive content detection solutions for TikTok: pitfalls and insights

The rise of social media and the prevalence of technology have led to a shift in the way that we interact with each other and consume information. In this digital environment, users with different intentions engage in online conversations and further actions which imply the increase in the use of offensive language and harmful behavior. At the same time, social media platforms collect large amounts of data, both from the user's own information shared on the platform, as well as data derived from the user's actions and general behavior. Such data is crucial for social media platforms who develop machine learning and data-driven approaches, e.g., for gaining customer insights, brand monitoring, performing competitive analysis, or building solutions for dealing with social manipulation activities like disinformation or offensive content. The availability of such datasets to a general audience that includes researchers and field organizations and practitioners is for particular social media platforms and in relation to specific events, however scarce.

In particular for TikTok which offers users a unique mix of features enabling individuals to easily create personalized content using a combination of text, music, and video, and to share these with their peers, limited datasets and corresponding data analytics research is available in relation to offensive content. Despite existing efforts from the platform, taking both social and technological developments, such content is on the rise. Hence, making datasets publicly available and building AI-based solutions on this behalf is necessary. To tackle this knowledge gap, this research collects and analyzes TikTok data containing offensive content and builds a series of Machine Learning and Deep Learning models for offensive content detection. To achieve this research aim, the following research question is formulated: How to develop a series of computational models to detect offensive content on TikTok?

For the purpose of answering this research question, a Data Science methodological approach is pursued. The following three deliverables contributing to the ongoing research and practitioner linguistic and social media efforts and discourses are proposed:

- Dataset collected using a combination of web scraping techniques which allowed the collection of a total of 120.423 comments that were further manually labeled and used to compile a balanced dataset.

- Data insights gathered based on a combination of frequency analysis using unigrams, bigrams, trigrams, topic modeling, and TF-IDF to quantitatively investigate the linguistic patterns associated with offensive language and see the prevalence of certain specific words and word combinations in offensive comments on TikTok. Moreover, the association of specific emojis with offensive versus neutral language is quantitatively investigated revealing that there are both overlaps and distinct patterns in the use of emojis for offensive language on TikTok.

- Modeling offensive content detection solutions using a combination of baseline Machine Learning techniques and BERT-based models with and without the integration of emoji and slang. Accordingly, comparative results show that BERT-based models perform the best and especially when integrating emoji and slang tokenization going from a F1 score of 0.851 to 0.863.