Optimization and Control (math.OC)

  • PDF
    We study the Wasserstein natural gradient in parametric statistical models with continuous sample space. Our approach is to pull back the $L^2$-Wasserstein metric tensor in probability density space to parameter space, under which the parameter space become a Riemannian manifold, named the Wasserstein statistical manifold. The gradient flow and natural gradient descent method in parameter space are then derived. When parameterized densities lie in $\bR$, we show the induced metric tensor establishes an explicit formula. Computationally, optimization problems can be accelerated by the proposed Wasserstein natural gradient descent, if the objective function is the Wasserstein distance. Examples are presented to demonstrate its effectiveness in several parametric statistical models.
  • PDF
    We show that it is possible to obtain an $O(\epsilon^{-4/3})$ runtime --- including computational cost --- for finding $\epsilon$-stationary points of nonconvex functions using cutting plane methods. This improves on the best known epsilon dependence achieved by cubic regularized Newton of $O(\epsilon^{-3/2})$ as proved by Nesterov and Polyak (2006)\nocitenesterov2006cubic. Our techniques utilize the convex until proven guilty principle proposed by Carmon, Duchi, Hinder, and Sidford (2017)\nocitecarmon2017convex.
  • PDF
    We prove that a simple Sequential Quadratic Programming (SQP) algorithm for equality constrained optimization has local linear convergence with rate $1 - 1/\kappa_R$, where $\kappa_R$ is the condition number of the Riemannian Hessian. Our analysis builds on insights from Riemannian optimization and indicates that first-order Riemannian algorithms and "simple" SQP algorithms have nearly identical local behavior. Unlike Riemannian algorithms, SQP avoids calculating the projection or retraction of the points onto the manifold for each iterates. All the SQP iterates will automatically be quadratically close the the manifold near the local minimizer.
  • PDF
    We consider a new stochastic gradient descent algorithm for efficiently solving general min-max optimization problems that arise naturally in distributionally robust learning. By focusing on the entire dataset, current approaches do not scale well. We address this issue by initially focusing on a subset of the data and progressively increasing this support to statistically cover the entire dataset.
  • PDF
    Gradient descent generalises naturally to Riemannian manifolds, and to hyperbolic $n$-space, in particular. Namely, having calculated the gradient at the point on the manifold representing the model parameters, the updated point is obtained by travelling along the geodesic passing in the direction of the gradient. Previous works employing optimisation in hyperbolic space have not attempted this procedure, however, employing instead various approximations to avoid a calculation that was considered to be too complicated. In this work, we demonstrate that in the hyperboloid model of hyperbolic space, the necessary calculations are in fact straight-forward. The advantages of the approach are then both illustrated and quantified for the optimisation problem of computing the Fr├ęchet mean (the analogue of the barycentre) of points in hyperbolic space.
  • PDF
    The minimum-gain eigenvalue assignment/pole placement problem (MGEAP) is a classical problem in LTI systems with static state feedback. In this paper, we study the MGEAP when the state feedback has arbitrary sparsity constraints. We formulate the sparse MGEAP problem as an equality-constrained optimization problem and present an analytical characterization of its solution in terms of eigenvector matrices of the closed loop system. This result is used to provide a geometric interpretation of the solution of the non-sparse MGEAP, thereby providing additional insights for this classical problem. Further, we develop an iterative projected gradient descent algorithm to solve the sparse MGEAP using a parametrization based on the Sylvester equation. We present a heuristic algorithm to compute the projections, which also provides a novel method to solve the sparse EAP. Also, a relaxed version of the sparse MGEAP is presented and an algorithm is developed to obtain approximately sparse solutions to the MGEAP. Finally, numerical studies are presented to compare the properties of the algorithms, which suggest that the proposed projection algorithm converges in almost all instances.
  • PDF
    In this paper, we utilize ADCSO (Adaptive Dynamic Cat Swarm Optimization) to estimate the parameters of Fractional Order Grey Model. The parameters of Fractional Order Grey Model affect the prediction accuracy of the model. In order to solve the problem that general swarm intelligence algorithms easily fall into the local optimum and optimize the accuracy of the model, ADCSO is utilized to reduce the error of the model. Experimental results for the data of container throughput of Wuhan Port and marine capture productions of Zhejiang Province show that the different parameter values affect the prediction results. The parameters estimated by ADCSO make the prediction error of the model smaller and the convergence speed higher, and it is not easy to fall into the local convergence compared with PSO (Particle Swarm Optimization) and LSM (Least Square Method). The feasibility and advantage of ADCSO for the parameter estimation of Fractional Order Grey Model are verified.
  • PDF
    For the sub-Riemannian problem on the group of motions of Euclidean space we present explicit formulas for extremal controls in a special case, when one of the initial momenta is fixed.
  • PDF
    For parabolic PDEs, we present a new certainty equivalence-based adaptive boundary control scheme with a least-squares identifier of an event-triggering type, where the triggering is based on the size of the regulation error (as opposed to the identifier updates being triggered by the estimation error, or the control changes being triggered by the regulation error). The scheme guarantees exponential convergence of the state to zero in the L2 norm and a finite-time convergence of the parameter estimates to the true values of the unknown parameters. The scheme is developed for a specific benchmark problem with Dirichlet actuation, where the only unknown parameters are the reaction coefficient and the high-frequency gain. For this specific problem, no existing adaptive scheme can handle the unknown high-frequency gain. An illustrative example allows the comparison with other adaptive control design methodologies.

Recent comments

Bin Shi Oct 05 2017 00:07 UTC

Welcome to give the comments for this paper!