Seminar Title: Kernel-Smoothed Score for Diffusion Probabilistic Models
Abstract: Diffusion probabilistic models have become the state-of-the-art tool in generative methods, used to generate high-resolution samples from very high-dimension distributions (e.g. images). Although very effective, they suffer some drawbacks: 1) as opposed to variational encoders, the dimension of the problem remains high during the generation process and 2) they can be prone to memorization of the training dataset.
In this talk, Veiga will introduce a kernel-smoothed empirical score and study the bias-variance of this estimator. We find improved bounds on the KL-divergence between a true measure and an approximate measure generated by using the smoothed empirical score. This score estimator leads to less memorization and better generalization. We demonstrate these findings on synthetic and real datasets, combining diffusion models with encoders to reduce the dimensionality of the problem.
