### Cluster-Seeking James-Stein Estimators

**2016-02-01**

1602.00542 | cs.IT

This paper considers the problem of estimating a high-dimensional vector of
parameters $\boldsymbol{\theta} \in \mathbb{R}^n$ from a noisy observation. The
noise vector is i.i.d. Gaussian with known variance. For a squared-error loss
function, the James-Stein (JS) estimator is known to dominate the simple
maximum-likelihood (ML) estimator when the dimension $n$ exceeds two. The
JS-estimator shrinks the observed vector towards the origin, and the risk
reduction over the ML-estimator is greatest for $\boldsymbol{\theta}$ that lie
close to the origin. JS-estimators can be generalized to shrink the data
towards any target subspace. Such estimators also dominate the ML-estimator,
but the risk reduction is significant only when $\boldsymbol{\theta}$ lies
close to the subspace. This leads to the question: in the absence of prior
information about $\boldsymbol{\theta}$, how do we design estimators that give
significant risk reduction over the ML-estimator for a wide range of
$\boldsymbol{\theta}$?
In this paper, we propose shrinkage estimators that attempt to infer the
structure of $\boldsymbol{\theta}$ from the observed data in order to construct
a good attracting subspace. In particular, the components of the observed
vector are separated into clusters, and the elements in each cluster shrunk
towards a common attractor. The number of clusters and the attractor for each
cluster are determined from the observed vector. We provide concentration
results for the squared-error loss and convergence results for the risk of the
proposed estimators. The results show that the estimators give significant risk
reduction over the ML-estimator for a wide range of $\boldsymbol{\theta}$,
particularly for large $n$. Simulation results are provided to support the
theoretical claims.

**Login to like/save this paper, take notes and configure your recommendations**

# Related Articles

**2016-04-24**

1604.06968 | cs.DS

We consider the problem of estimating the mean and covariance of a
distribution from iid samples in … show more

**2014-11-17**

1411.4342 | stat.ML

We propose and analyze estimators for statistical functionals of one or more
distributions under non… show more

**2019-04-26**

1904.13309 | cs.DS

Consider a community where initially, each individual is positive or negative
regarding a reform pro… show more

**2018-12-26**

1812.10551 | stat.ML

A common challenge in estimating parameters of probability density functions
is the intractability o… show more

**2019-04-05**

1904.03136 | math.ST

Monge matrices and their permuted versions known as pre-Monge matrices
naturally appear in many doma… show more

**2019-05-02**

1905.00586 | cs.LG

Recently, a method called the Mutual Information Neural Estimator (MINE) that
uses neural networks h… show more

**2018-11-28**

1811.11368 | stat.ML

This paper studies distributed estimation and inference for a general
statistical problem with a con… show more

**2019-06-04**

1906.01714 | cs.SY

This paper proposes a class of resilient state estimators for LTV
discrete-time systems. The dynamic… show more

**2019-03-11**

1903.04416 | math.ST

We introduce the {\it diffusion $K$-means} clustering method on Riemannian
submanifolds, which maxim… show more

**2017-07-28**

1707.09161 | cs.IT

The problem of estimating a high-dimensional sparse vector
$\boldsymbol{\theta} \in \mathbb{R}^n$ fr… show more

**2019-04-11**

1904.05543 | cs.DS

In the subspace sketch problem one is given an $n\times d$ matrix $A$ with
$O(\log(nd))$ bit entries… show more

**2018-10-26**

1810.11344 | cs.LG

Expectation Maximization (EM) is among the most popular algorithms for
maximum likelihood estimation… show more

**2018-07-12**

1807.04566 | stat.ML

In sensor networks, it is not always practical to set up a fusion center.
Therefore, there is need f… show more

**2018-05-17**

1805.06834 | cs.LG

We present a high-dimensional analysis of three popular algorithms, namely,
Oja's method, GROUSE and… show more

**2019-03-19**

1903.08560 | math.ST

Interpolators---estimators that achieve zero training error---have attracted
growing attention in ma… show more

**2018-08-29**

1808.10013 | cs.LG

We clarify what fairness guarantees we can and cannot expect to follow from
unconstrained machine le… show more