A comparative study of using multilingual LLMs for AMR generation

We present a comparative study of using state-of-the-art multilingual LLMs such as OpenAI’s GPT-models, and sequence-to-sequence models such as umT5 and MBART for the task of text-to-AMR generation. We may add other open-source alternatives to ChatGPT, although early testing showed that while ChatGPT is able to construct valid AMR, some open alternatives fail drastically. More exploratory work will be done in this respect. Some of our models are already out (https://huggingface.co/spaces/BramVanroy/text-to-amr) but new results as well as structured prompts and training code will also be made publicly available by the time of the conference.

AMR, or abstract meaning representation, captures the semantic events, concepts and relationships in a given sentence (or document) and structures these in a graph. As such, generating AMR is in fact a sequence-to-graph problem although most times the complexity is reduced to a sequence-to-sequence task by linearizing the AMR graph into a sequence of meaningful tokens that can be delinearized into the required graph notation.