Evaluating Large-Scale Construction Grammars in Semantic Frame Extraction

This research explores the capabilities of computational construction grammars in the context of Semantic Role Labeling (SRL) and its extension, Semantic Frame Extraction (SFE), both essential tasks in Natural Language Processing (NLP). Playing a significant role in Natural Language Understanding (NLU), SRL and SFE facilitate the interpretation and extraction of meaning from text by identifying relationships among various constituents of a sentence. This study includes an evaluation of multiple computational construction grammars in the context of the SemBrowse project, an innovative corpus-linguistic tool developed by the Vrije Universiteit Brussel and the Université de Namur. SemBrowse utilizes Fluid Construction Grammar (FCG) to pinpoint instances of frame-semantic patterns in text corpora.

The study looks at the effectiveness and performance of these grammars, identifying both strengths and weaknesses, to inform future developments. The findings demonstrate considerable differences in efficiency and performance across the grammars, based on several configurations and heuristics. For instance, the speed of the grammar comprehension process, as well as the accuracy in labeling semantic roles, were both significantly impacted by the application of different heuristics.

Results also indicated the benefits of grouping PropBank role sets into semantically similar clusters using VerbAtlas mappings for an expansive understanding of grammar performance across semantic categories.

The findings contribute to the further development and operationalization of large-scale, computational construction grammars, providing insights into the strengths and areas for improvement of current grammars, and suggesting directions for future research.