Simple Limit Theorems

About Posts which Tagged by 'Probability'

Let $\{X_n\}_{n\geq1}$ be a sequence of random variables, and $S_n=\sum_{j=1}^n X_j$.  To verify $$\frac{S_n-\mathscr{E}(S_n)}{n}\overset{p}{\rightarrow}0,$$ we need to show $\mathscr{E}(S_n^2)=o(n^2)$ inspired by the $L^2$ convergence.  However, it might not be easy.

[Theorem] If $X_j$'s are uncorrelated and their second moment have a common bound, then $$\frac{S_n-\mathscr{E}(S_n)}{n}\rightarrow0$$ is true
(1) in $L^2$;
(2) in probability;
(3) almost surely.


$\bullet$ Proof.

Let $\mathscr{E}(X_j^2)\leq M$ for all $j$.

For (1) and (2), $$\mathscr{E}\left|\frac{S_n-\mathscr{E}(S_n)}{n}\right|^2\leq\frac{1}{n^2}\mathscr{E}(S^2_n)=\frac{1}{n^2}\sum_{j=1}^n \mathscr{E}(X^2_j)\leq\frac{nM}{n^2}\rightarrow0\mbox{ as }n\rightarrow\infty.$$
Thus, $$\frac{S_n-\mathscr{E}(S_n)}{n}\overset{L^2}{\rightarrow}0\mbox{ and hence }\frac{S_n-\mathscr{E}(S_n)}{n}\overset{p}{\rightarrow}0.$$

For (3), if directly applying Chebyshev's inequality and Borel-Cantelli Lemma for $\sum_n\mathscr{P}\{|S_n|>n\varepsilon\}$, this series would diverge to infinity (you can try).  Thus we start with the subsequence $\{n^2\}_{n\geq1}$.

WLOG, let $\mathscr{E}(X_j)=0$ for all $j$.  Since $\mathscr{E}(X_j^2)\leq M$, $\mathscr{E}(S_n^2)=\sum_j\mathscr{E}(X_j^2)\leq nM$
By Chebyshev's inequality,$$\mathscr{P}\{|S_{n^2}|>n^2\varepsilon\}\leq\frac{n^2M}{n^4\varepsilon^2}=\frac{M}{n^2\varepsilon^2}.$$  Then $$\sum_n\mathscr{P}\left\{\frac{|S_{n^2}|}{n^2}>\varepsilon\right\}\leq\sum_n\frac{M}{n^2\varepsilon^2}<\infty.$$  By Borel-Cantelli Lemma, we have $$\mathscr{P}\left\{\frac{|S_{n^2}|}{n^2}>\varepsilon\mbox{ i.o.}\right\}=0\implies\frac{S_{n^2}}{n^2}\overset{a.s.}{\rightarrow}0.$$

Next, let $D_n=\underset{n^2\leq k<(n+1)^2}{\max}|S_k-S_{n^2}|$.  Then $$D_n^2=\underset{n^2\leq k<(n+1)^2}{\max}|S_k-S_{n^2}|^2\leq\sum_{k=n^2}^{(n+1)^2-1}(S_k-S_{n^2})^2\implies$$  $$\begin{array}{ccl}\mathscr{E}(D_n^2)
&\leq&\sum_{k=n^2}^{(n+1)^2-1}\mathscr{E}(S_k-S_{n^2})^2 \\
&=&\sum_{k=n^2}^{(n+1)^2-1}\mathscr{E}\left(\sum_{j=n^2}^{k}X_j-X_{n^2}\right)^2 \\
&\leq&\sum_{k=n^2}^{(n+1)^2-1}\mathscr{E}\left(\sum_{j=n^2}^{(n+1)^2-1}X_j\right)^2 \\
&=&\sum_{k=n^2}^{(n+1)^2-1}\sum_{j=n^2}^{(n+1)^2-1}\mathscr{E}(X_j^2)\;(\because\,\mbox{uncorrelated})\\
&\leq&4n^2M. \end{array}$$
By Chebyshev's inequality and Borel-Cantelli Lemma, $$\mathscr{P}\{|D_n|>n^2\varepsilon\}\leq\frac{4n^2M}{n^4\varepsilon^2}=\frac{4M}{n^2\varepsilon^2}\implies$$ $$\sum_n\mathscr{P}\left\{\frac{|D_n|}{n^2}>\varepsilon\right\}\leq\sum_n\frac{4M}{n^2\varepsilon^2}<\infty\implies\mathscr{P}\left\{\frac{|D_n|}{n^2}>\varepsilon\mbox{ i.o.}\right\}=0\implies\frac{D_n}{n^2}\overset{a.s.}{\rightarrow}0.$$

Finally, for $n^2\leq k<(n+1)^2$, $D_n\geq|S_k-S_{n^2}|\geq|S_k|-|S_{n^2}|$.  Thus, $$\frac{|S_k|}{k}\leq\frac{|S_k|}{n^2}\leq\frac{|S_{n^2}|+D_n}{n^2}\rightarrow0\mbox{ a.s.}$$ We have, for any $n$ and any $k$, $$\frac{|S_k|}{k}\overset{a.s.}{\rightarrow}0.$$

$\Box$