On the Minimization of Convex Functionals of Probability Distributions Under Band Constraints


The problem of minimizing convex functionals of probability distributions is solved under the assumption that the density of every distribution is bounded from above and below. A system of sufficient and necessary first-order optimality conditions as well as a bound on the optimality gap of feasible candidate solutions are derived. Based on these results, two numerical algorithms are proposed that iteratively solve the system of optimality conditions on a grid of discrete points. Both algorithms use a block coordinate descent strategy and terminate once the optimality gap falls below the desired tolerance. While the first algorithm is conceptually simpler and more efficient, it is not guaranteed to converge for objective functions that are not strictly convex. This shortcoming is overcome in the second algorithm, which uses an additional outer proximal iteration, and, which is proven to converge under mild assumptions. Two examples are given to demonstrate the theoretical usefulness of the optimality conditions as well as the high efficiency and accuracy of the proposed numerical algorithms.
Submitted 17 Mar 2017 to Information Theory [cs.IT]
Published 20 Mar 2017
Updated 2 Mar 2018
Subjects: cs.IT math.IT
Author comments: 13 pages, 5 figures, 2 tables, published in the IEEE Transactions on Signal Processing
Msc class: 62C20
Journal ref: M. Fau{\ss} and A. M. Zoubir, "On the Minimization of Convex Functionals of Probability Distributions Under Band Constraints," in IEEE Transactions on Signal Processing, vol. 66, no. 6, pp. 1425-1437, March15, 15 2018
Doi: 10.1109/TSP.2017.2788427