ML p(r)ior | Extremal Relations Between Shannon Entropy and $\ell_α$-Norm

Extremal Relations Between Shannon Entropy and $\ell_α$-Norm

2016-01-28
1601.07678 | cs.IT
The paper examines relationships between the Shannon entropy and the $\ell_{\alpha}$-norm for $n$-ary probability vectors, $n \ge 2$. More precisely, we investigate the tight bounds of the $\ell_{\alpha}$-norm with a fixed Shannon entropy, and vice versa. As applications of the results, we derive the tight bounds between the Shannon entropy and several information measures which are determined by the $\ell_{\alpha}$-norm, e.g., R\'{e}nyi entropy, Tsallis entropy, the $R$-norm information, and some diversity indices. Moreover, we apply these results to uniformly focusing channels. Then, we show the tight bounds of Gallager's $E_{0}$ functions with a fixed mutual information under a uniform input distribution.
PDF

Highlights - Most important sentences from the article

Login to like/save this paper, take notes and configure your recommendations

Related Articles

2019-04-30

This paper presents the first proof of polarization for the deletion channel with a constant deletio… show more
PDF

Highlights - Most important sentences from the article

2019-03-25
1903.10101 | math.ST

Log-concave distributions include some important distributions such as normal distribution, exponent… show more
PDF

Highlights - Most important sentences from the article

2018-07-08

A distributed binary hypothesis testing problem involving two parties, a remote observer and a detec… show more
PDF

Highlights - Most important sentences from the article

2019-04-26

Polar transforms are central operations in the study of polar codes. This paper examines polar trans… show more
PDF

Highlights - Most important sentences from the article

2018-11-23
1811.09652 | cs.IT

We introduce a novel generalization of entropy and conditional entropy from which most definitions f… show more
PDF

Highlights - Most important sentences from the article

2018-09-04

In this paper, we investigate the convergence speed of the Arimoto algorithm. By analyzing the Taylo… show more
PDF

Highlights - Most important sentences from the article

2018-09-03
1809.00459 | cs.IT

We consider discrete linear time invariant (LTI) channels satisfying the phase independence (PI) ass… show more
PDF

Highlights - Most important sentences from the article

2017-02-20

We extend Fano's inequality, which controls the average probability of events in terms of the averag… show more
PDF

Highlights - Most important sentences from the article

2017-07-08

We investigate the problem of estimating a random variable $Y\in \mathcal{Y}$ under a privacy constr… show more
PDF

Highlights - Most important sentences from the article

2015-02-01

We consider estimating the Shannon entropy of a discrete distribution $P$ from $n$ i.i.d. samples. R… show more
PDF

Highlights - Most important sentences from the article

2018-05-03
1805.01401 | cs.IT

We study the convexity of mutual information as a function of time along the flow of the Ornstein-Uh… show more
PDF

Highlights - Most important sentences from the article

2019-01-10

This paper studies an $n$-dimensional additive Gaussian noise channel with a peak-power-constrained … show more
PDF

Highlights - Most important sentences from the article

2018-01-09
1801.02876 | cs.IT

Fano's inequality is one of the most elementary, ubiquitous and important tools in information theor… show more
PDF

Highlights - Most important sentences from the article

2018-11-29
1811.12257 | cs.LG

We consider a setup in which confidential i.i.d. samples $X_1,\dotsc,X_n$ from an unknown finite-sup… show more
PDF

Highlights - Most important sentences from the article

2018-09-25
1809.09696 | cs.IT

Let $T_{\epsilon}$ be the noise operator acting on functions on the boolean cube $\{0,1\}^n$. Let $f… show more
PDF

Highlights - Most important sentences from the article

2018-06-29
1806.11531 | cs.IT

The sphere packing bound is proved for codes on the discrete stationary product channels with feedba… show more
PDF

Highlights - Most important sentences from the article