While gradients are fundamental to optimization in deep learning, their utility in prompt optimization for large language models (LLMs) remains limited. This gap stems from challenges unique to language, such as its discrete nature and the fact that LLM objectives are only defined over discrete inputs. As a result, gradient information is often underutilized, limiting the effectiveness of tasks like prompt engineering and red-teaming.

In this talk, I will present my work on gradient-based prompt optimization, focusing on two components: optimization methods and objectives. First, I will introduce a controllable generation method that generates fluent prompts while eliciting desired model behaviors, such as jailbreaks or false refusals. Next, I will show that existing optimization objectives offer limited control over model behavior. To address this, we propose a new objective that improves behavior control and simplifies the optimization process. Finally, I will explore the potential and challenges of grad-based optimization without search and share early results of a prototype method. This talk aims to provide an overview of recent progress and remaining challenges in gradient-based prompt optimization for LLMs.