Language models can now generate novels that are published by major publishing houses like Hachette, as well as professional reports for firms like KPMG. These models increasingly shape what people read and how they write, yet their ability to produce long-form texts has outpaced our ability to rigorously evaluate and improve them. Long-form texts contain so many details that human annotators struggle to evaluate them thoroughly, and because no single ground truth exists, their judgments are often subjective and inconsistent. As a result, reliable training signals are difficult to obtain, which hinders the improvement of models on long-form text generation tasks. This proposal studies how intermediate structures, such as topics, outlines, and user intents, make long-form text easier to verify. We use these structures to understand document collections, build training data for long-form tasks, and evaluate model outputs.

Part I presents methods for extracting intermediate structures from human writing and using them to build grounded training data. TopicGPT describes a document collection with topics in natural language and provides a supporting quotation for each document-topic assignment. CLIPPER and Suri then use intermediate structures to construct training data for long-form tasks. Specifically, CLIPPER uses book summaries and chapter outlines to construct training data for narrative claim verification, while Suri pairs human-written long-form texts with outlines and introduces a preference-tuning method based on corrupted outlines. Fine-tuning on the CLIPPER and Suri data improves book-length reasoning and long-form instruction following, respectively. These projects show how intermediate structures support corpus analysis and reduce the cost of constructing training data for long-form tasks.

Part II presents methods to evaluate long-form generation in realistic settings, where humans interact with models over multiple turns to improve the final output. Our proposed work builds a multi-turn writing benchmark with a simulated writer. We will validate the simulated writer against human writing sessions, evaluate whether the assistant preserves the writer’s original intent across turns, and study whether turn-level feedback can train better writing assistants.