Interplay between CLT and convergence in Total Variation

Given a random variable $X$ with bounded moments such that $E[X] = 0, E[X^2] = 1$, let $F_n$ denote the distribution $\sum\limits_{i=1}^d\frac{X_i}{\sqrt{n}}$ where each $X_i$ is an independent copy of $X$. It is known that as $n \to \infty$, $F_n$ converges in distribution to a standard Gaussian.

The Berry-Esseen inequality then gives us a quantitative estimate for the rate of convergence. If $\Phi$ denotes the standard Gaussian distribution we have::
$$\sup|F_n – \Phi| \leq C \frac{1}{\sqrt{n}} $$

Where (throught the discussion) $C$ is a positive constant depending on the moments of $X$.

We may also look at a stronger type of convergence. Namely, convergence in total variation (in the $L_1$ norm). Clearly, without further demands from $X$, nothing can be expected. However, some sufficient conditions can be found. For example, when $X$ satisfies a Poincare type inequality or when $X$ has a finite Kullback-Leibler divergence from $\Phi$. In both those cases one may use the Entropic Central Limit Theorem (as stated here,, for example) to give a similar bound on the total variation distance.
$$||F_n – \Phi||_{\mathrm{TV}} \leq C \frac{1}{\sqrt{n}}$$.

So, when all $X_i$ are identically distributed and satisfy some simple condition, convergence in Total Variation and convergence in distribution happen together at the same rate.

Now, let us consider the non-i.i.d case. Let $A_n = \{a_i\}_{i=1}^n$ and denote by $F_{A,n}$ the distribution of $\sum a_i X_i$, where again all $X_i$ are independent copies of $X$.

In this case, the Berry-Esseen inequality gives the following bound:
$$\sup|F_{A,n} – \Phi| \leq C \frac{\left(\sum a_i^3\right)}{\left(\sum a_i^2\right)^{(3/2)}}$$

Looking at the Total Variation though, bounding the Kullback-Liebler divergence of $X$ from standard normal no longer makes sense, since the divergence of $a_iX_i$ can grow as a function of $a_i$.

The known bound of the Entropic CLT, given that $X$ satisfies a Poincare inequality (see when joined with Pinsker’s inequality then gives:
$$||F_{A,n} – \Phi||_{\mathrm{TV}} \leq C\frac{\sqrt{\sum a_i^4}}{\sum a_i^2}$$
Unlike in the i.i.d case this gives a real difference between Total Variation distance and convergence in distribution.

I am not aware whether the Entropic CLT is tight. But, this suggests the existence of a nice enough random variable $X$ (satisfying Poincare inequality) and a set of coefficients $A$, such that $\sum a_iX_i$ convergence in distribution to a Gaussian in rate $O(\frac{1}{\sqrt{n}})$, while converging in Total Variation in a much slower rate (Consider for example the case where $a_i \sim \frac{1}{i^{(1/3)}}$)
In particular, there is some event on the real line, which cannot be a finite union of intervals, on which the two distributions differ by a substantial amount.

My question is then:

  • Is there a natural example of such a random variable?

  • If so, is there a natural description to the event on which the distributions differ?


Source : Link , Question Author : Cain , Answer Author : Community

Leave a Comment