ML p(r)ior | Adaptive Subgradient Methods for Online AUC Maximization

Adaptive Subgradient Methods for Online AUC Maximization

2016-02-01
Learning for maximizing AUC performance is an important research problem in Machine Learning and Artificial Intelligence. Unlike traditional batch learning methods for maximizing AUC which often suffer from poor scalability, recent years have witnessed some emerging studies that attempt to maximize AUC by single-pass online learning approaches. Despite their encouraging results reported, the existing online AUC maximization algorithms often adopt simple online gradient descent approaches that fail to exploit the geometrical knowledge of the data observed during the online learning process, and thus could suffer from relatively larger regret. To address the above limitation, in this work, we explore a novel algorithm of Adaptive Online AUC Maximization (AdaOAM) which employs an adaptive gradient method that exploits the knowledge of historical gradients to perform more informative online learning. The new adaptive updating strategy of the AdaOAM is less sensitive to the parameter settings and maintains the same time complexity as previous non-adaptive counterparts. Additionally, we extend the algorithm to handle high-dimensional sparse data (SAdaOAM) and address sparsity in the solution by performing lazy gradient updating. We analyze the theoretical bounds and evaluate their empirical performance on various types of data sets. The encouraging empirical results obtained clearly highlighted the effectiveness and efficiency of the proposed algorithms.
PDF

Highlights - Most important sentences from the article

Login to like/save this paper, take notes and configure your recommendations

Related Articles

2014-12-22
1412.6980 | cs.LG

We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective … show more
PDF

Highlights - Most important sentences from the article

2015-09-30
1510.00012 | stat.CO

In a variety of research areas, the weighted bag of vectors and the histogram are widely used descri… show more
PDF

Highlights - Most important sentences from the article

2017-11-10

Deep Neural Networks (DNNs) are typically trained by backpropagation in a batch learning setting, wh… show more
PDF

Highlights - Most important sentences from the article

2019-02-27

We study the problem of meta-learning through the lens of online convex optimization, developing a m… show more
PDF

Highlights - Most important sentences from the article

2016-10-28

SOL is an open-source library for scalable online learning algorithms, and is particularly suitable … show more
PDF

Highlights - Most important sentences from the article

2018-09-12

We present a unified framework for Batch Online Learning (OL) for Click Prediction in Search Adverti… show more
PDF

Highlights - Most important sentences from the article

2019-01-25

Adaptive gradient-based optimization methods such as \textsc{Adagrad}, \textsc{Rmsprop}, and \textsc… show more
PDF

Highlights - Most important sentences from the article

2018-11-19

Learning a deep neural network requires solving a challenging optimization problem: it is a high-dim… show more
PDF

Highlights - Most important sentences from the article

2016-11-07
1611.02101 | stat.ML

Generalized linear model with $L_1$ and $L_2$ regularization is a widely used technique for solving … show more
PDF

Highlights - Most important sentences from the article

2017-10-27

Conjugate gradient (CG) methods are a class of important methods for solving linear equations and no… show more
PDF

Highlights - Most important sentences from the article

2019-04-17

This paper introduces a novel approach for dengue fever classification based on online learning para… show more
PDF

Highlights - Most important sentences from the article

2019-01-29
1901.10443 | cs.LG

Motivated by concerns that machine learning algorithms may introduce significant bias in classificat… show more
PDF

Highlights - Most important sentences from the article

2018-10-26
1811.00178 | cs.LG

Online learning makes sequence of decisions with partial data arrival where next movement of data is… show more
PDF

Highlights - Most important sentences from the article

2018-03-30
1803.11521 | stat.ML

Current online learning methods suffer issues such as lower convergence rates and limited capability… show more
PDF

Highlights - Most important sentences from the article

2019-01-30

Adaptive gradient-based optimizers such as AdaGrad and Adam are among the methods of choice in moder… show more
PDF

Highlights - Most important sentences from the article

2017-10-02

The area under the ROC curve (AUC) is a measure of interest in various machine learning and data min… show more
PDF