Foundation models, large neural networks trained on vast collections of text and images, now power widely used AI systems, from chatbots and image generators to agents that search the web and use tools on a user's behalf. These models are mostly judged by average scores on benchmarks, yet a high average score, measured without an adversary, does not mean that a model is safe and reliable to use. We develop methods that audit foundation models for problems that such scores hide, and that steer the models, and the systems built around them, toward safer and more reliable behavior.

Our work spans four areas: (i) failure mode discovery, where we propose PRIME to find groups of inputs on which an image classifier consistently fails and to describe them with human-understandable tags; (ii)machine unlearning, where we introduce Model State Arithmetic (MSA), which uses checkpoints saved during training to remove the influence of copyrighted, private, or false training data; (iii) detection and provenance of AI-generated content, where we prove fundamental limits of image watermarks and detectors, show practical attacks on image and text detectors, and propose more robust watermarking and source identification methods, IConMark and DREW; (iv) and reliable and efficient interaction with the environment, where we introduce SpecHop, which lets retrieval agents keep working during tool calls and verifies each speculative step, reducing latency by up to 40% while leaving the agent's trajectory unchanged in all but a few cases. Together, these results show that safety and reliability must be examined directly, not inferred from average benchmark scores.