# Show that $\mathbb{E}\left|\hat{f_n}-f \right| \leq \frac{2}{n^{1/3}}$ where $\hat{f_n}$ is a density estimator for $f$

## Question

Suppose we have a continuous probability density $$f : \mathbb{R} \to [0,\infty)$$ such that $$\text{sup}_{x \in \mathbb{R}}(\left|f(x)\right| + \left|f'(x)\right|) \leq 1. \;$$ Define the density estimator:

$$\hat{f_n} = \frac{1}{n^{2/3}}\sum^n_{i=1}\mathbf{1}\{-1/2 \leq n^{1/3}(x-X_i) \leq 1/2\}, \: x \in \mathbb{R}$$

Show that, for every $$x \in \mathbb{R} \:$$ and every $$n \in \mathbb{N},$$
$$\mathbb{E}\left|\hat{f_n}(x)-f(x)\right| \leq \frac{2}{n^{1/3}}$$

## Attempt

It’s easy to show that $$\mathbb{E}\hat{f_n} = \frac{F(x+\frac{1}{2}n^{-1/3}) – F(x-\frac{1}{2}n^{-1/3})}{n^{-1/3}}$$
where $$F$$ is the CDF of $$f$$. From here, using the condition $$\:\text{sup}_{x \in \mathbb{R}}(\left|f(x)\right| + \left|f'(x)\right|) \leq 1 \:$$ and in particular, $$\:\text{sup}_{x \in \mathbb{R}}\left|f'(x)\right|\leq 1\:$$ we can show (geometrically) that,
$$\left|\mathbb{E}\hat{f_n}-f\right| \leq \frac{1}{4n^{1/3}}$$

However, I’m not sure how to get a bound on $$\mathbb{E}\left|\hat{f_n}-f\right|$$.

### General

The density estimator is a special case of a kernel density estimator (KDE) with rectangular kernel $$K(x)=\mathbf{1}\{-1/2\leq x\leq 1/2\}$$ and bandwidth $$h=n^{-1/3}$$. It would therefore be possible to use the general theory of KDEs and deduce the result.

I will not follow that approach but instead derive properties of this specific estimator directly and link the results to the general KDE theory if possible.

### Distribution of $$\hat{f_n}$$

As (apart from a multiplicative constant) the estimator is effectively counting the number of times an observation of $$X_i$$ is falling into the interval $$\left[x-\frac{1}{2}n^{-1/3}, x+\frac{1}{2}n^{-1/3}\right]$$ when calculating $$\hat{f_n}(x)$$ we can immediately see that $$n^{2/3} \hat{f_n}\sim\operatorname{Binomial}(n,p)$$ with $$p$$ equal to $$F\left(x+\frac{1}{2}n^{-1/3}\right)-F\left(x-\frac{1}{2}n^{-1/3}\right).$$ This secretly assumes that all the $$X_i$$ are independent and identically distributed with CDF $$F$$. From this we can infer directly that $$\mathbb{E}\hat{f_n}(x)=n^{1/3}p$$ and $$\operatorname{Var}(\hat{f_n}(x))=n^{-1/3}p(1-p)$$.

### Asymptotics

To get the asymptotic behaviour of the expectation and variance we have to analyse $$p$$ from above. Using a Taylor approximation at $$x$$ we see that $$F\left(x\pm\frac{1}{2}n^{-1/3}\right)=F(x)\pm f(x)\frac{1}{2}n^{-1/3}+f'(\xi_\pm)\frac{1}{8}n^{-2/3}$$ for some $$\xi_\pm$$ between $$x$$ and $$x\pm\frac{1}{2}n^{-1/3}$$. Knowing that $$|f’|\leq 1$$ we get $$\left|\mathbb{E}\hat{f_n}(x)-f(x)\right|\leq\frac{1}{4}n^{-1/3}\tag1.$$

Remark
Assuming that also $$f”$$ exists and is continuous we could get a stronger result. In this case the bias of $$\hat{f_n}$$ would tend to zero with the order $$n^{-2/3}$$. This is in line with the more general result that under mild assumptions on the kernel $$h$$ we have that $$\left|\mathbb{E}\hat{f_n}(x)-f(x)\right|=\mathcal{O}(h^2)$$ for $$n$$ towards infinity.

For the variance we also use a shortened version of the Taylor approximation $$F\left(x\pm\frac{1}{2}n^{-1/3}\right)=F(x)\pm f(\zeta_\pm)\frac{1}{2}n^{-1/3}$$ for some $$\zeta_\pm$$ between $$x$$ and $$x\pm\frac{1}{2}n^{-1/3}$$. The variance can now be written as $$n^{-1/3}p(1-p)=n^{-1/3}(p-p^2)$$ with $$p-p^2=f(x)n^{-1/3}+(f'(\xi_+)-f'(\xi_-))\frac{1}{8}n^{-2/3} – (f(\zeta_+)+f(\zeta_-))^2\frac{1}{4}n^{-2/3}.$$ Again, using the given upper bounds for $$f$$ and $$f’$$ we can bound the variance from above by $$\operatorname{Var}(\hat{f_n}(x))\leq n^{-1/3}\left(n^{-1/3}+\frac{1}{4}n^{-2/3}\right)\leq\frac{5}{4}n^{-2/3}.\tag2$$

Remark
This result is similar to a result from the general theory of KDEs, which gives $$\operatorname{Var}(\hat{f_n}(x)) =\mathcal{O}((nh)^{-1})$$ for $$n$$ towards infinity.

### Combining the results

To get the desired result we use a decomposition of the mean absolute error (MAE) similar to that of the mean squared error. We start with \begin{align}\mathbb{E}\left|\hat{f_n}(x)-f(x)\right|&\leq\mathbb{E}\left|\hat{f_n}(x)-\mathbb{E}\hat{f_n}(x)\right|+\mathbb{E}\left|\mathbb{E}\hat{f_n}(x)-f(x)\right|\\&\leq\left(\mathbb{E}\left|\hat{f_n}(x)-\mathbb{E}\hat{f_n}(x)\right|^2\right)^{1/2}+\left|\mathbb{E}\hat{f_n}(x)-f(x)\right|\\&=\operatorname{Var}\left(\hat{f_n}(x)\right)^{1/2}+\left|\mathbb{E}\hat{f_n}(x)-f(x)\right|\end{align} using the triangle and Hölder inequalities. Using bounds $$(1)$$ and $$(2)$$ we finally get $$\mathbb{E}\left|\hat{f_n}(x)-f(x)\right|\leq\left(\frac{\sqrt{5}}{2}+\frac{1}{4}\right)n^{-1/3}\leq 2n^{-1/3}$$ and have the result.