What does it mean to integrate with respect to the distribution function?

If $f(x)$ is a density function and $F(x)$ is a distribution function of a random variable $X$ then I understand that the expectation of x is often written as:

$$E(X) = \int x f(x) dx$$

where the bounds of integration are implicitly $-\infty$ and $\infty$. The idea of multiplying x by the probability of x and summing makes sense in the discrete case, and it’s easy to see how it generalises to the continuous case. However, in Larry Wasserman’s book All of Statistics he writes the expectation as follows:

$$E(X) = \int x dF(x)$$

I guess my calculus is a bit rusty, in that I’m not that familiar with the idea of integrating over functions of $x$ rather than just $x$.

  • What does it mean to integrate over the distribution function?
  • Is there an analogous process to repeated summing in the discrete case?
  • Is there a visual analogy?

UPDATE:
I just found the following extract from Wasserman’s book (p.47):

The notation $\int x d F(x)$ deserves some comment. We use it merely
as a convenient unifying notation so that we don’t have to write
$\sum_x x f(x)$ for discrete random variables and $\int x f(x) dx$ for
continuous random variables, but you should be aware that $\int x d F(x)$ has a precise meaning that is discussed in a real analysis
course.

Thus, I would be interested in any insights that could be shared about what is the precise meaning that would be discussed in a real analysis course?

Answer

There are many definitions of the integral, including the Riemann integral, the Riemann-Stieltjes integral (which generalizes and expands upon the Riemann integral), and the Lebesgue integral (which is even more general.) If you’re using the Riemann integral, then you can only integrate with respect to a variable (e.g. $x$), and the notation $dF(x)$ isn’t defined.

The Riemann-Stieltjes integral generalizes the concept of the Riemann integral and allows for integration with respect to a cumulative distribution function that isn’t continuous.

The notation $\int_{a}^{b} g(x)dF(x)$ is roughly equivalent of $\int_{a}^{b} g(x) f(x) dx$ when $f(x)=F'(x)$. However, if $F(x)$ is a function that isn’t differentiable at all points, then you simply can’t evaluate $\int_{a}^{b} g(x) f(x) dx$, since $f(x)=F'(x)$ isn’t defined.

In probability theory, this situation occurs whenever you have a random variable with a discontinuous cumulative distribution function. For example, suppose $X$ is $0$ with probability $\frac{1}{2}$ and $1$ with probability $\frac{1}{2}$. Then

$$
\begin{align}
F(x) &= 0 & x &< 0 \\
F(x) &= 1/2 & 0 &\leq x < 1 \\
F(x) &= 1 & x &\geq 1 \\
\end{align}
$$

Clearly, $F(x)$ doesn’t have a derivative at $x=0$ or $x=1$, so there isn’t a probability density function $f(x)$ at those points.

Now, suppose that we want to evaluate $E[X^3]$. This can be written, using the Riemann-Stieltjes integral, as

$$E[X^3]=\int_{-\infty}^{\infty} x^3 dF(x).$$

Note that because there isn’t a probability density function $f(x)$, we can’t write this as

$$E[X^{3}]=\int_{-\infty}^{\infty} x^3 f(x) dx.$$

However, we can use the fact that this random variable is discrete to evaluate the expected value as:

$$E[X^{3}]=(0)^{3}(1/2)+(1)^{3}(1/2)=1/2$$

So, the short answer to your question is that you need to study alternative definitions of the integral, including the Riemann and Riemann-Stieltjes integrals.

Attribution
Source : Link , Question Author : Jeromy Anglim , Answer Author : Brian Borchers

Leave a Comment