RI: Small: New tools for studying structural and inductive bias in NLP models

Modern natural language processing systems, based on neural networks trained using large amounts of text, are a key part of the infrastructure of the nation and the world. These systems power practical tools like machine translation, web search, or automatic question answering, as well as research tools that help scientists and policy makers. These language processing models have made enormous progress in many ways, yet systems still fail unexpectedly, their successes cannot be explained, and their blind spots lead to biases.

This project develops new tools for studying language models: why they work as well as they do, what their limitations are, and what distortions they introduce into language understanding, with the goal of improved systems and helping mitigate negative impacts on society.

This project develops and investigates four kinds of new analytic tools for studying the inductive biases of language models - the structural tendencies that determine what they can learn. The structural transfer-learning paradigm involves training language models on artificial languages that can be manipulated, to see which structural aspects improve performance on natural language.

The challenge-task paradigm brings humans in the loop to develop new evaluations to study why and how language processing systems fail, such as on aspect of language that change over time. The new theoretical framework of sensitivity models the complexity of language processing tasks by measuring how responsive the classification is to minor changes in the input, demonstrating which tasks or examples are easy or hard.

And new tools are introduced to measure how embeddings of words introduce structural distortions - exaggerations or understatements in word relationships - that can cause models to fail. Understanding the limitations of technology and what makes one system better or one task or dataset harder than another is a crucial step toward building better language processing systems.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.