We present COGNAC, a novel strategy for compiling quantum circuits inspired by numerical optimization algorithms from scientific computing. Observing that shorter-duration “partially entangling” gates tend to be less noisy than the typical “maximally entangling” gates, we use a simple and versatile noise model to construct a differentiable cost function. Standard gradient-based optimization algorithms running on a GPU can then quickly converge to a local optimum that closely approximates the target unitary. By reducing rotation angles to zero, COGNAC removes gates from a circuit, producing smaller quantum circuits. We have implemented this technique as a general-purpose Qiskit compiler plugin and compared performance with state-of-the-art optimizers on a variety of standard benchmarks. Testing our compiled circuits on superconducting quantum hardware, we find that COGNAC’s optimizations produce circuits that usually outperform existing optimizers while remaining competitive in terms of its optimization time.