Nanyun Peng (UCLA) - Controllable and Creative Natural Language Generation

Abstract: Recent advances in large language models (LLMs) have achieved remarkable results across a wide range of natural language processing (NLP) applications, including text classification, summarization, machine translation, and dialogue systems. As LLMs grow increasingly capable, the need to control their generation process becomes more pressing, particularly for high-stakes applications that demand reliable outputs adhering to specific guidelines or creative outputs within defined boundaries. However, the dominant auto-regressive paradigm—training models to predict the next word based on prior context—poses significant challenges for enforcing structural or content-specific constraints.

In this talk, I will present our recent work on controllable natural language generation (NLG) that moves beyond the conventional auto-regressive framework to enhance both the reliability and creativity of generative models. I will introduce controllable decoding-time algorithms that guide auto-regressive models to better align with user-specified constraints. Additionally, I will discuss a novel insertion-based generation paradigm that breaks away from the limitations of auto-regressive methods. These approaches enable more reliable and creative outputs, with applications spanning creative writing, lexical-controlled generation, and commonsense-compliant text generation.