Super-Samples from Kernel Herding


We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting "kernel herding" algorithm is an infinite memory deterministic process that learns to approximate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert space at a rate O(1/T) which is much faster than the usual O(1/pT) for iid random samples. We illustrate kernel herding by approximating Bayesian predictive distributions.
Submitted 15 Mar 2012 to Learning [cs.LG]
Published 16 Mar 2012
Subjects: cs.LG stat.ML
Author comments: Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI2010)
Report no: UAI-P-2010-PG-109-116
Proxy: auai