THESIS DEFENSE: On Physics-Inspired Generative Models

Abstract: Physics-inspired generative models such as diffusion models constitute a powerful family of generative models. The advantages of models in this family come from relatively stable training process and high capacity. A number of possible improvements remain possible. In this talk, I will discuss the enhancement and design of physics-inspired generative models. I will first present a sampling algorithm that combines the best of previous samplers, greatly accelerating the generation speed of text-to-image Stable Diffusion models. Secondly, I will discuss a training framework that introduces learnable discrete latents into continuous diffusion models. These latents simplify complex noise-to-data mappings and reduce the curvature of generative trajectories. Finally, I will introduce Poisson Flow Generative Models (PFGM), a new generative model arising from electrostatic theory, rivaling leading diffusion models. The extended version, PFGM++, places diffusion models and PFGM under the same framework and introduces new, better models. Several algorithms discussed in the talk are the state-of-the-art methods across standard benchmarks.

Committee Members: Tommi Jaakkola (advisor, MIT), Phillip Isola (MIT), Karsten Kreis (NVIDIA)