ML p(r)ior | Large-scale Kernel-based Feature Extraction via Budgeted Nonlinear Subspace Tracking

Large-scale Kernel-based Feature Extraction via Budgeted Nonlinear Subspace Tracking

2016-01-28
Kernel-based methods enjoy powerful generalization capabilities in handling a variety of learning tasks. When such methods are provided with sufficient training data, broadly-applicable classes of nonlinear functions can be approximated with desired accuracy. Nevertheless, inherent to the nonparametric nature of kernel-based estimators are computational and memory requirements that become prohibitive with large-scale datasets. In response to this formidable challenge, the present work puts forward a low-rank, kernel-based, feature extraction approach that is particularly tailored for online operation, where data streams need not be stored in memory. A novel generative model is introduced to approximate high-dimensional (possibly infinite) features via a low-rank nonlinear subspace, the learning of which leads to a direct kernel function approximation. Offline and online solvers are developed for the subspace learning task, along with affordable versions, in which the number of stored data vectors is confined to a predefined budget. Analytical results provide performance bounds on how well the kernel matrix as well as kernel-based classification and regression tasks can be approximated by leveraging budgeted online subspace learning and feature extraction schemes. Tests on synthetic and real datasets demonstrate and benchmark the efficiency of the proposed method when linear classification and regression is applied to the extracted features.
PDF

Highlights - Most important sentences from the article

Login to like/save this paper, take notes and configure your recommendations

Related Articles

2018-10-31

We investigate how to train kernel approximation methods that generalize well under a memory budget.… show more
PDF

Highlights - Most important sentences from the article

2018-01-31
1802.00043 | stat.ML

Incremental versions of batch algorithms are often desired, for increased time efficiency in the str… show more
PDF

Highlights - Most important sentences from the article

2016-01-17
1601.04366 | cs.LG

Efficient and accurate low-rank approximations of multiple data sources are essential in the era of … show more
PDF

Highlights - Most important sentences from the article

2019-02-11
1902.03999 | cs.LG

In this article, we introduce a novel boosting algorithm called `KTBoost', which combines kernel boo… show more
PDF

Highlights - Most important sentences from the article

2018-09-14
1809.05247 | cs.LG

Kernel method has been developed as one of the standard approaches for nonlinear learning, which how… show more
PDF

Highlights - Most important sentences from the article

2018-04-17

Sparse subspace clustering (SSC) is a popular method in machine learning and computer vision for clu… show more
PDF

Highlights - Most important sentences from the article

2016-11-21

Survival analysis is a fundamental tool in medical research to identify predictors of adverse events… show more
PDF

Highlights - Most important sentences from the article

2019-04-21

Gaussian processes (GP) for machine learning have been studied systematically over the past two deca… show more
PDF

Highlights - Most important sentences from the article

2015-05-03

Kernel methods are widespread in machine learning; however, they are limited by the quadratic comple… show more
PDF

Highlights - Most important sentences from the article

2018-05-25

Independent component analysis (ICA) is a widespread data exploration technique, where observed sign… show more
PDF

Highlights - Most important sentences from the article

2018-08-06

Big data problems frequently require processing datasets in a streaming fashion, either because all … show more
PDF

Highlights - Most important sentences from the article

2017-06-09

Kernel $k$-means clustering can correctly identify and extract a far more varied collection of clust… show more
PDF

Highlights - Most important sentences from the article

2018-08-01

Matrix completion and extrapolation (MCEX) are dealt with here over reproducing kernel Hilbert space… show more
PDF

Highlights - Most important sentences from the article

2018-08-31

Traditional kernels or their combinations are often not sufficiently flexible to fit the data in com… show more
PDF

Highlights - Most important sentences from the article

2019-04-08

In computer vision, image datasets used for classification are naturally associated with multiple la… show more
PDF

Highlights - Most important sentences from the article

2018-10-09

Kernel methods offer the flexibility to learn complex relationships in modern, large data sets while… show more
PDF

Highlights - Most important sentences from the article