ML p(r)ior | A Proximal Stochastic Quasi-Newton Algorithm

A Proximal Stochastic Quasi-Newton Algorithm

2016-01-31
1602.00223 | cs.LG
In this paper, we discuss the problem of minimizing the sum of two convex functions: a smooth function plus a non-smooth function. Further, the smooth part can be expressed by the average of a large number of smooth component functions, and the non-smooth part is equipped with a simple proximal mapping. We propose a proximal stochastic second-order method, which is efficient and scalable. It incorporates the Hessian in the smooth part of the function and exploits multistage scheme to reduce the variance of the stochastic gradient. We prove that our method can achieve linear rate of convergence.
PDF

Highlights - Most important sentences from the article

Login to like/save this paper, take notes and configure your recommendations

Related Articles

2015-08-09

We propose a new stochastic L-BFGS algorithm and prove a linear convergence rate for strongly convex… show more
PDF

Highlights - Most important sentences from the article

2017-05-16

We consider the minimization of non-convex functions that typically arise in machine learning. Speci… show more
PDF

Highlights - Most important sentences from the article

2017-07-20

Due to their simplicity and excellent performance, parallel asynchronous variants of stochastic grad… show more
PDF

Highlights - Most important sentences from the article

2019-05-29
1905.12412 | math.OC

We propose a novel randomized incremental gradient algorithm, namely, VAriance-Reduced Accelerated G… show more
PDF

Highlights - Most important sentences from the article

2018-02-05
1802.01504 | math.OC

We consider the convex-concave saddle point problem $\min_{x}\max_{y} f(x)+y^\top A x-g(y)$ where $f… show more
PDF

Highlights - Most important sentences from the article

2019-01-27
1901.09344 | cs.LG

Stochastic approximation (SA) is a classical approach for stochastic convex optimization. Previous s… show more
PDF

Highlights - Most important sentences from the article

2019-02-16

Proximal gradient method has been playing an important role to solve many machine learning tasks, es… show more
PDF

Highlights - Most important sentences from the article

2019-05-07
1905.02374 | stat.ML

In this paper, we propose a unified view of gradient-based algorithms for stochastic convex composit… show more
PDF

Highlights - Most important sentences from the article

2019-01-25
1901.08788 | stat.ML

In this paper, we propose a unified view of gradient-based algorithms for stochastic convex composit… show more
PDF

Highlights - Most important sentences from the article

2019-01-24
1901.08523 | math.OC

This paper introduces an efficient second-order method for solving the elastic net problem. Its key … show more
PDF

Highlights - Most important sentences from the article

2018-10-25

There has been extensive research on developing stochastic variance reduced methods to solve large-s… show more
PDF

Highlights - Most important sentences from the article

2018-09-17

In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex… show more
PDF

Highlights - Most important sentences from the article

2018-07-20
1807.08023 | math.OC

Stochastic gradient descent (SGD) is one of the most widely used optimization methods for parallel a… show more
PDF

Highlights - Most important sentences from the article

2018-05-30

Derivative-free optimization has become an important technique used in machine learning for optimizi… show more
PDF

Highlights - Most important sentences from the article

2018-02-24
1802.08880 | cs.LG

We study stochastic algorithms for solving nonconvex optimization problems with a convex yet possibl… show more
PDF

Highlights - Most important sentences from the article

2018-10-07

This paper proposes an accelerated proximal stochastic variance reduced gradient (ASVRG) method, in … show more
PDF