Strong Law of Large Number

About Posts which Tagged by 'Probability'

Let $\{X_n\}$ be a sequence of independent and identically distributed random variables.  Define $S_n=\sum_j X_j$.  Then we have $$\mathscr{E}|X_1|<\infty\implies\frac{S_n}{n}\rightarrow \mathscr{E}(X_1)\;\mbox{ a.s.}$$ $$\mathscr{E}|X_1|=\infty\implies\underset{n\rightarrow\infty}{\limsup}\frac{|S_n|}{n}=+\infty\;\mbox{ a.s.}$$



$\bullet$ Proof. (The Converge Part)
First, we introduce an important result and its corollary obtained by the Three Series Theorem.

[Theorem] Let $\phi$ be a positive, even and continuous function on $(-\infty,\infty)$ such that as $|x|$ increases, $$\frac{\phi(x)}{|x|}\uparrow,\;\frac{\phi(x)}{x^2}\downarrow.$$ Suppose $\mathscr{E}(X_n)=0$ for every $n$ and $0<a_n\uparrow\infty$.  If, additionally, $\phi$ satisfies $$\sum_n\frac{\mathscr{E}\left(\phi(X_n)\right)}{\phi(a_n)}<\infty,$$ then $$\sum_n\frac{X_n}{a_n}\mbox{ converges  a.e.}$$See Application of Three Series Theorem on Strong Convergence

[Corollary] Under the some hypothesis, we have $\frac{1}{a_n}\sum_{j=1}^nX_j\rightarrow0\;\mbox{ a.e.}$


Back to the proof.  Define $$Y_n=\begin{cases}X_n, &\mbox{if }X_n\leq n\\ 0,&\mbox{o.w.}\end{cases}$$ and $T_n=\sum_j Y_j$. Since $\mathscr{E}|X_n|<\infty$, $$\sum_n\mathscr{P}\{X_n\neq Y_n\}=\sum_n\mathscr{P}\{|X_n|>n\}<\infty.$$  Thus, $\{Y_n\}$ and $\{X_n\}$ are equivalent.

According to the Theorem and Corollary above.  Let $\phi(x)=x^2$ and $a_n=n$.  We know $\mathscr{E}\left(Y_n-\mathscr{E}(Y_n)\right)=0$. $$\begin{array}{ccl}\sum_{n=1}^\infty\frac{\mathscr{E}\left(Y_n-\mathscr{E}(Y_n)\right)^2}{n^2}&=&\sum_{n=1}^\infty\sigma^2(\frac{Y_n}{n}) \\
&\leq&\sum_{n=1}^\infty\frac{1}{n^2}\mathscr{E}\left(Y_n^2\right)=\sum_{n=1}^\infty\frac{1}{n^2}\int_{|x|\leq n}x^2\,d\mathcal{F}\\
&=&\sum_{n=1}^\infty\frac{1}{n^2}\left[\sum_{j=1}^n\int_{j-1< |x|\leq  j}x^2\,d\mathcal{F}\right] \\
&\leq&\sum_{j=1}^\infty\left\{\int_{j-1< |x|\leq  j}x^2\,d\mathcal{F}\left(\sum_{n=j}^\infty\frac{1}{n^2}\right)\right\} \quad\mbox{by (A)}\\
&\leq&\sum_{j=1}^\infty j\int_{j-1< |x|\leq  j}|x|\,d\mathcal{F}\left(\frac{2}{j}\right)\qquad\quad\;\mbox{by (B)} \\
&\leq&2\sum_{j=1}^\infty\int_{j-1\leq |x|\leq  j}|x|\,d\mathcal{F} =2\mathscr{E}|X_1|<\infty. \end{array}$$ We have $\frac{1}{n}\sum_{j=1}^n\{Y_j-\mathscr{E}(Y_j)\}\rightarrow0\;\mbox{ a.e.}$

Clearly, $\mathscr{E}(Y_n)=\int_{|X_n|\leq n}x\,d\mathcal{F}\rightarrow\mathscr{E}(X_1)$ as $n\rightarrow\infty$.  Which implies $\frac{1}{n}\sum_{j=1}^n \mathscr{E}(Y_j)\rightarrow\mathscr{E}(X_1)\;\mbox{ a.e.}$, and thus, $\frac{1}{n}\sum_{j=1}^n Y_j\rightarrow\mathscr{E}(X_1)\;\mbox{ a.e.}$  By the equivalence between $\{Y_n\}$ and $\{X_n\}$, we have $$\frac{1}{n}\sum_{j=1}^n X_j=\frac{S_n}{n}\rightarrow \mathscr{E}(X_1).$$

Supplementary:
For (A), consider the fraction and the integral area in $$\sum_{n=1}^\infty\frac{1}{n^2}\left[\sum_{j=1}^n\int_{j-1< |x|\leq  j}x^2\,d\mathcal{F}\right].$$ $$\begin{array}{c|cccc}
n\setminus j & 1 & 2 & 3 & \cdots \\ \hline
1                  &\frac{1}{1^2}\int_{[0,1)} & & & \\
2                  &\frac{1}{2^2}\int_{[0,1)} & \frac{1}{2^2}\int_{[1,2)} & & \\
3                  &\frac{1}{3^2}\int_{[0,1)} & \frac{1}{3^2}\int_{[1,2)} & \frac{1}{3^2}\int_{[2,3)} & \\
\vdots          & \vdots  & \vdots  & \vdots  & \cdots \\ \hline
                    & \sum_{n=1}^\infty\frac{1}{n^2}\int_{[0,1)} & \sum_{n=2}^\infty\frac{1}{n^2}\int_{[1,2)} & \sum_{n=3}^\infty\frac{1}{n^2}\int_{[2,3)} & \cdots \\
\end{array}$$  Then we can exchange the summation signs w.r.t. $n$ and $j$ as $$\sum_{j=1}^\infty\left\{\int_{j-1< |x|\leq  j}x^2\,d\mathcal{F}\left(\sum_{n=j}^\infty\frac{1}{n^2}\right)\right\}.$$

For (B), $$\sum_{n=j}^\infty\frac{1}{n^2}\leq\frac{1}{j}+\sum_{n=j+1}^\infty\frac{1}{n(n-1)}=\frac{1}{j}+\sum_{n=j+1}^\infty\left(\frac{1}{n-1}-\frac{1}{n}\right)\leq\frac{2}{j}.$$

$\Box$

$\bullet$ Proof. (The Diverge Part)
Let $\mathscr{E}|X_1|=\infty$, then this implies that for any $A>0$, $$\mathscr{E}\left(\frac{|X_1|}{A}\right)=\infty.$$  Thus, by Borel-Cantelli Lemma, we have $$\sum_{n=1}^\infty\mathscr{P}\left\{\frac{|X_1|}{A}\geq n\right\}=\infty\implies\mathscr{P}\left\{\frac{|X_n|}{A}\geq n\mbox{ i.o}\right\}=1.$$
Consider that $$n\leq\frac{|X_n|}{A}=\frac{|S_n-S_{n-1}|}{A}\leq\frac{|S_n|}{A}+\frac{|S_{n-1}|}{A}.$$  This implies either $\frac{|S_n|}{A}\geq \frac{n}{2}\mbox{ i.o.}$ or $\frac{|S_n|}{A}\geq\frac{n}{2}\mbox{ i.o.}$  Thus, $$\mathscr{P}\left\{\frac{|S_n|}{A}\geq \frac{n}{2}\mbox{ i.o}\right\}=\mathscr{P}\left\{\frac{|S_n|}{n}\geq \frac{A}{2}\mbox{ i.o}\right\}=1.$$ Since $A$ is arbitrary, we have, as $A\rightarrow\infty$, $$\underset{n\rightarrow\infty}{\limsup}\frac{|S_n|}{n}=+\infty\;\mbox{ a.s.}$$

$\Box$