ML p(r)ior | On Deep Multi-View Representation Learning: Objectives and Optimization

On Deep Multi-View Representation Learning: Objectives and Optimization

2016-02-02
We consider learning representations (features) in the setting in which we have access to multiple unlabeled views of the data for learning while only one view is available for downstream tasks. Previous work on this problem has proposed several techniques based on deep neural networks, typically involving either autoencoder-like networks with a reconstruction objective or paired feedforward networks with a batch-style correlation-based objective. We analyze several techniques based on prior work, as well as new variants, and compare them empirically on image, speech, and text tasks. We find an advantage for correlation-based representation learning, while the best results on most tasks are obtained with our new variant, deep canonically correlated autoencoders (DCCAE). We also explore a stochastic optimization procedure for minibatch correlation-based objectives and discuss the time/performance trade-offs for kernel-based and neural network-based implementations.
PDF

Highlights - Most important sentences from the article

Login to like/save this paper, take notes and configure your recommendations

Related Articles

2015-05-28

We introduce a new representation learning approach for domain adaptation, in which data at training… show more
PDF

Highlights - Most important sentences from the article

2015-11-19

Bilinear models has been shown to achieve impressive performance on a wide range of visual tasks, su… show more
PDF

Highlights - Most important sentences from the article

2016-12-01

We present a variational approximation to the information bottleneck of Tishby et al. (1999). This v… show more
PDF

Highlights - Most important sentences from the article

2015-04-27

Common Representation Learning (CRL), wherein different descriptions (or views) of the data are embe… show more
PDF

Highlights - Most important sentences from the article

2012-06-24

The success of machine learning algorithms generally depends on data representation, and we hypothes… show more
PDF

Highlights - Most important sentences from the article

2015-11-19

Supervised, semi-supervised, and unsupervised learning estimate a function given input/output sample… show more
PDF

Highlights - Most important sentences from the article

2017-08-22

In an era when big data are becoming the norm, there is less concern with the quantity but more with… show more
PDF

Highlights - Most important sentences from the article

2015-11-15

We introduce Deep Linear Discriminant Analysis (DeepLDA) which learns linearly separable latent repr… show more
PDF

Highlights - Most important sentences from the article

2015-02-09

Canonical correlation analysis (CCA) has proven an effective tool for two-view dimension reduction d… show more
PDF

Highlights - Most important sentences from the article

2018-04-13
1804.04888 | cs.LG

One-class support vector machine (OC-SVM) for a long time has been one of the most effective anomaly… show more
PDF

Highlights - Most important sentences from the article

2018-01-04

Many of the existing machine learning algorithms, both supervised and unsupervised, depend on the qu… show more
PDF

Highlights - Most important sentences from the article

2018-10-28

Variational autoencoders (VAE) are a powerful and widely-used class of models to learn complex data … show more
PDF

Highlights - Most important sentences from the article

2015-08-31

The problem of domain generalization is to take knowledge acquired from a number of related domains … show more
PDF

Highlights - Most important sentences from the article

2019-04-03

We propose Deep Multiset Canonical Correlation Analysis (dMCCA) as an extension to representation le… show more
PDF

Highlights - Most important sentences from the article

2019-01-29
1901.10177 | cs.CV

Person re-identification (Re-ID) aims to match identities across non-overlapping camera views. Resea… show more
PDF

Highlights - Most important sentences from the article

2016-04-21

In this paper we present a deep neural network topology that incorporates a simple to implement tran… show more
PDF