PhD Defense: Learning-based Physics Simulation with Collision Handling
Qingyang Tan
Tuesday, June 27, 2023, 1:00-3:00 pm 
Abstract
Numerous physics-based simulation approaches have been proposed to generate realistic and vivid deformations for 3D models. These include the mass-spring system, the finite element approach, the thin-shell model, and others. However, previous systems based on analytic and numerical methods tend to be computationally intensive. Achieving an ideal balance between simulation accuracy and efficiency still poses several challenges.
In this dissertation, we present novel learning-based physics simulations and collision-handling algorithms that leverage the benefits of neural networks and optimization techniques. We use neural networks to compress high-dimensional 3D deformable models and accelerate the processing time. We also employ algorithms such as reinforcement learning, active learning, and imitation learning to capture complex physical behaviors that lack closed-form analytic models.
We propose multiple novel approaches for novel learning-based physics simulation. First, we train a learning-based collision detector for 3D deformable models and utilize the detector as a surrogate constraint in an optimization-based collision handler. To perform stable collision prediction performance in large and unseen spaces, we employ active learning by progressively incorporating new collision data based on network inferences, reaching a collision detection accuracy of up to 98.1%. Second, we present an approach to accelerate collision response computations by incorporating an additional repulsive force unit in the learning-based pipeline. Our experiments demonstrate that backbone networks trained with the repulsive force unit can significantly decrease the number of collisions, boosting collision-free models from 49% to 77%, while maintaining real-time performance, adding only 2 milliseconds to the inference system. Third, we present a neural volumetric deformable object simulator with collision detection and handling based on an actor-critic neural architecture. Our critic network learns to estimate collision penetrations, while our actor network learns to minimize the penalty function through a series of gradient descent steps, resulting in nearly collision-free quasistatic deformable object poses. Finally, we introduce a novel framework for randomly reposing 3D humans to arbitrary poses based on a geometric optimization approach that incorporates control information into diffusion-based inpainting. Our geometric inpainting algorithm reduces errors by 93% when moving different body parts to random locations.
In practice, our learning-based physics simulation systems can generate realistic 3D models that satisfy various constraints. We have tested our method using several large-scale datasets, including AMASS for humans and Tailor-Net for garments. Our approach can generate plausible results and we observe 100-300x speedups over numerical or analytic methods.
In this dissertation, we present novel learning-based physics simulations and collision-handling algorithms that leverage the benefits of neural networks and optimization techniques. We use neural networks to compress high-dimensional 3D deformable models and accelerate the processing time. We also employ algorithms such as reinforcement learning, active learning, and imitation learning to capture complex physical behaviors that lack closed-form analytic models.
We propose multiple novel approaches for novel learning-based physics simulation. First, we train a learning-based collision detector for 3D deformable models and utilize the detector as a surrogate constraint in an optimization-based collision handler. To perform stable collision prediction performance in large and unseen spaces, we employ active learning by progressively incorporating new collision data based on network inferences, reaching a collision detection accuracy of up to 98.1%. Second, we present an approach to accelerate collision response computations by incorporating an additional repulsive force unit in the learning-based pipeline. Our experiments demonstrate that backbone networks trained with the repulsive force unit can significantly decrease the number of collisions, boosting collision-free models from 49% to 77%, while maintaining real-time performance, adding only 2 milliseconds to the inference system. Third, we present a neural volumetric deformable object simulator with collision detection and handling based on an actor-critic neural architecture. Our critic network learns to estimate collision penetrations, while our actor network learns to minimize the penalty function through a series of gradient descent steps, resulting in nearly collision-free quasistatic deformable object poses. Finally, we introduce a novel framework for randomly reposing 3D humans to arbitrary poses based on a geometric optimization approach that incorporates control information into diffusion-based inpainting. Our geometric inpainting algorithm reduces errors by 93% when moving different body parts to random locations.
In practice, our learning-based physics simulation systems can generate realistic 3D models that satisfy various constraints. We have tested our method using several large-scale datasets, including AMASS for humans and Tailor-Net for garments. Our approach can generate plausible results and we observe 100-300x speedups over numerical or analytic methods.
Examining Committee
Bio
Qingyang Tan is a PhD student at the University of Maryland, College Park, advised by Prof. Dinesh Manocha. His research interests lie in applying deep learning in physical simulation and geometry processing. He received the B.Eng. degree in Computer Science and Technology from the University of Chinese Academy of Sciences.
This talk is organized by Tom Hurst