Skip to main content
padlock icon - secure page this page is secure

Statistical inference for the population landscape via moment‐adjusted stochastic gradients

Buy Article:

$47.00 + tax (Refund Policy)

Modern statistical inference tasks often require iterative optimization methods to compute the solution. Convergence analysis from an optimization viewpoint informs us only how well the solution is approximated numerically but overlooks the sampling nature of the data. In contrast, recognizing the randomness in the data, statisticians are keen to provide uncertainty quantification, or confidence, for the solution obtained by using iterative optimization methods. The paper makes progress along this direction by introducing moment‐adjusted stochastic gradient descent: a new stochastic optimization method for statistical inference. We establish non‐asymptotic theory that characterizes the statistical distribution for certain iterative methods with optimization guarantees. On the statistical front, the theory allows for model misspecification, with very mild conditions on the data. For optimization, the theory is flexible for both convex and non‐convex cases. Remarkably, the moment adjusting idea motivated from ‘error standardization’ in statistics achieves a similar effect to acceleration in first‐order optimization methods that are used to fit generalized linear models. We also demonstrate this acceleration effect in the non‐convex setting through numerical experiments.
No References
No Citations
No Supplementary Data
No Article Media
No Metrics

Keywords: Acceleration; Diffusion process; Discretized Langevin algorithm; Model misspecification; Non‐asymptotic inference; Population landscape; Stochastic gradient methods

Document Type: Research Article

Publication date: April 1, 2019

  • Access Key
  • Free content
  • Partial Free content
  • New content
  • Open access content
  • Partial Open access content
  • Subscribed content
  • Partial Subscribed content
  • Free trial content
Cookie Policy
X
Cookie Policy
Ingenta Connect website makes use of cookies so as to keep track of data that you have filled in. I am Happy with this Find out more