The problem of minimizing convex functionals of probability distributions is solved under the assumption that the density of every distribution is bounded from above and below. First, a system of sufficient and necessary first order optimality conditions, which characterize global minima as solutions of a fixed-point equation, is derived. Based on these conditions, two algorithms are proposed that iteratively solve the fixed-point equation via a block coordinate descent strategy. While the first algorithm is conceptually simpler and more efficient, it is not guaranteed to converge for objective functions that are not strictly convex. This shortcoming is overcome in the second algorithm, which uses an additional outer proximal iteration, and, which is proven to converge under very mild assumptions. Two examples are given to demonstrate the theoretical usefulness of the optimality conditions as well as the high efficiency and accuracy of the proposed numerical algorithms.
Submitted 17 Mar 2017 to Information Theory
Published 20 Mar 2017
Author comments: 11 pages, 5 figures, 1 table, to be submitted to the IEEE Transactions on Signal Processing
Msc class: 62C20http://arxiv.org/abs/1703.06128http://arxiv.org/pdf/1703.06128.pdf