Shared Task

Welcome to the CLIN33 shared task on the Automatic Detection of AI-Generated Texts!

With the surge of powerful automatic content generation comes the important task of distinguishing texts generated by language models from texts written by humans. Therefore, the CLIN33 shared task invites you to contribute to the development of automatic detection systems of AI-generated texts. The shared task consists of two tracks.

Visit the shared task website for more information

Detection Track

In this track you will develop a system that detects whether a document is generated by a language model or written by a human. The system will be evaluated on a test dataset that will remain hidden. For tuning your system you will be provided with a sample of the test data as a development set.

• Your system will be submitted as a notebook (colab or jupyter) and be self-contained
• The test data will not be shared
• You will be provided with a subset of the test data for validation and development
• No training data will be provided, you are expected to create your own when using Machine Learning
• There will be a scoreboard for testing your model on the validation data that you can use to monitor progress during development
• There will be three genres of text: newspaper text, tweets, and poems
• The text will be generated by different language models (we will provide information about which models later)
• The generated text will be produced using different prompt engineering strategies (standard, academic level, middle school level, human-like, ...)
• There will be an English and a Dutch subtrack. Participants are expected to participate in both.

There will be a prize for the team that develops the best performing system averaged over both languages and all conditions.

Explanation Track (optional)

In this track, you will provide a qualitative analysis, empirically supported by the output of your system on the validation set, of the knowledge used by your model to solve the task. This explanation can be in terms of statistical properties or in terms of linguistic patterns and phenomena and will the form of a short paper (4 pages max). This track will be evaluated by the organizers, extended with additional judges representing the users of detection systems (for example teachers or editors). The most insightful submission will be awarded a special prize.