ML p(r)ior | Cluster-Seeking James-Stein Estimators

Cluster-Seeking James-Stein Estimators

2016-02-01
This paper considers the problem of estimating a high-dimensional vector of parameters $\boldsymbol{\theta} \in \mathbb{R}^n$ from a noisy observation. The noise vector is i.i.d. Gaussian with known variance. For a squared-error loss function, the James-Stein (JS) estimator is known to dominate the simple maximum-likelihood (ML) estimator when the dimension $n$ exceeds two. The JS-estimator shrinks the observed vector towards the origin, and the risk reduction over the ML-estimator is greatest for $\boldsymbol{\theta}$ that lie close to the origin. JS-estimators can be generalized to shrink the data towards any target subspace. Such estimators also dominate the ML-estimator, but the risk reduction is significant only when $\boldsymbol{\theta}$ lies close to the subspace. This leads to the question: in the absence of prior information about $\boldsymbol{\theta}$, how do we design estimators that give significant risk reduction over the ML-estimator for a wide range of $\boldsymbol{\theta}$? In this paper, we propose shrinkage estimators that attempt to infer the structure of $\boldsymbol{\theta}$ from the observed data in order to construct a good attracting subspace. In particular, the components of the observed vector are separated into clusters, and the elements in each cluster shrunk towards a common attractor. The number of clusters and the attractor for each cluster are determined from the observed vector. We provide concentration results for the squared-error loss and convergence results for the risk of the proposed estimators. The results show that the estimators give significant risk reduction over the ML-estimator for a wide range of $\boldsymbol{\theta}$, particularly for large $n$. Simulation results are provided to support the theoretical claims.
PDF

Highlights - Most important sentences from the article

Login to like/save this paper, take notes and configure your recommendations

Related Articles

2016-04-24

We consider the problem of estimating the mean and covariance of a distribution from iid samples in … show more
PDF

Highlights - Most important sentences from the article

2014-11-17

We propose and analyze estimators for statistical functionals of one or more distributions under non… show more
PDF

Highlights - Most important sentences from the article

2019-04-26
1904.13309 | cs.DS

Consider a community where initially, each individual is positive or negative regarding a reform pro… show more
PDF

Highlights - Most important sentences from the article

2018-12-26
1812.10551 | stat.ML

A common challenge in estimating parameters of probability density functions is the intractability o… show more
PDF

Highlights - Most important sentences from the article

2019-04-05

Monge matrices and their permuted versions known as pre-Monge matrices naturally appear in many doma… show more
PDF

Highlights - Most important sentences from the article

2019-05-02
1905.00586 | cs.LG

Recently, a method called the Mutual Information Neural Estimator (MINE) that uses neural networks h… show more
PDF

Highlights - Most important sentences from the article

2018-11-28
1811.11368 | stat.ML

This paper studies distributed estimation and inference for a general statistical problem with a con… show more
PDF

Highlights - Most important sentences from the article

2019-06-04

This paper proposes a class of resilient state estimators for LTV discrete-time systems. The dynamic… show more
PDF

Highlights - Most important sentences from the article

2019-03-11
1903.04416 | math.ST

We introduce the {\it diffusion $K$-means} clustering method on Riemannian submanifolds, which maxim… show more
PDF

Highlights - Most important sentences from the article

2017-07-28
1707.09161 | cs.IT

The problem of estimating a high-dimensional sparse vector $\boldsymbol{\theta} \in \mathbb{R}^n$ fr… show more
PDF

Highlights - Most important sentences from the article

2019-04-11
1904.05543 | cs.DS

In the subspace sketch problem one is given an $n\times d$ matrix $A$ with $O(\log(nd))$ bit entries… show more
PDF

Highlights - Most important sentences from the article

2018-10-26
1810.11344 | cs.LG

Expectation Maximization (EM) is among the most popular algorithms for maximum likelihood estimation… show more
PDF

Highlights - Most important sentences from the article

2018-07-12

In sensor networks, it is not always practical to set up a fusion center. Therefore, there is need f… show more
PDF

Highlights - Most important sentences from the article

2018-05-17
1805.06834 | cs.LG

We present a high-dimensional analysis of three popular algorithms, namely, Oja's method, GROUSE and… show more
PDF

Highlights - Most important sentences from the article

2019-03-19

Interpolators---estimators that achieve zero training error---have attracted growing attention in ma… show more
PDF

Highlights - Most important sentences from the article

2018-08-29

We clarify what fairness guarantees we can and cannot expect to follow from unconstrained machine le… show more
PDF

Highlights - Most important sentences from the article