Towards exploratory dialogue: Classifying User Transitional Intent to Enable Multi-turn conversational agents

A framework is introduced for therapy chatbots that enables further, deeper, non-scripted exploration using natural language generation (NLG). The study shows that user engagement is improved by using a set number of NLG exchanges, compared to human-authored and single-turn NLG dialogue. It proposes a transitioning strategy based on the transitional intent (TI) expressed in the user's response to reactively transition between dialogue states. A dataset is collected to train a state-of-the-art TI-classifier using a fine-tuned BERT, BERTTI. The paper highlights the potential of transformer-based chatbots in the medical field and the importance of balancing scripted and non-scripted dialogue for effective therapy.