Modern scientific and engineering challenges rooted in physics often require searching immense solution spaces. Sampling these high-dimensional spaces can be computationally prohibitive. I observe that many underlying dynamical processes are nearly everywhere differentiable: physical states evolve smoothly in time, and even brief discontinuities preserve overall continuity. When a system’s next state and its gradient with respect to the current state are available, those gradients act as beacons that steer optimization far more efficiently than gradient‑free exploration. Consequently, researchers are investing heavily in differentiable models of real‑world phenomena, especially within physical sciences and engineering, where the laws of motion and interaction naturally provide the required smoothness.

Building on this pervasive continuity in natural and engineered systems, this thesis demonstrates how exploiting differentiability can systematically accelerate problem-solving in both decision making and geometric reconstruction. I first develop gradient-aware reinforcement learning methods that use system-level derivatives to guide exploration: one that augments Proximal Policy Optimization (PPO) with analytic gradients to constrain policy updates and speed convergence, and another that trains a continuous-time world model by randomly sampling time steps, greatly improving sample efficiency. These algorithmic advances motivate two differentiable systems at the core of the thesis: (1) a GPU-accelerated traffic simulator with exact vehicle-kinematic Jacobians, enabling rapid optimization of autonomous-vehicle policies and traffic-signal timing; and (2) a family of differentiable mesh formulations that make traditionally discrete triangle and tetrahedral meshes amenable to gradient-based optimization, including a probabilistic triangular-mesh representation that relaxes edge existence into continuous variables and a differentiable tetrahedral-mesh extraction pipeline for 3D object/mesh reconstruction. By weaving these case studies together, this thesis charts a unified path for integrating physics and geometry into a common, scalable, gradient-based optimization framework.