Converge Almost Surely v.s. Converge in Probability

About Posts which Tagged by 'Probability'

Let $X$ and $\{X_n\}_{n\geq1}$ be random variables.  $X_n$ converge to $X$ almost surely implies $X_n$ converge to $X$ in probability.  The converse is NOT true except for convergence along a subsequence.



(1) $X_n\overset{a.s.}{\longrightarrow}X\,\Rightarrow\, X_n\overset{p}{\longrightarrow}X$.

$\bullet$ Proof.
$X_n\overset{a.s.}{\longrightarrow}X\implies$
$$\forall\,\varepsilon>0\mbox{ and }\delta>0,\exists\,m(\varepsilon,\delta)\mbox{ s.t. }\mathscr{P}\left\{\underset{k\geq m}{\bigcap}|X_k-X|\leq\varepsilon\right\}\geq1-\delta.$$ Thus for $n>>m$, $$\mathscr{P}\left\{|X_n-X|\leq\varepsilon\right\}\geq\mathscr{P}\left\{\underset{n\geq m}{\bigcap}|X_n-X|\leq\varepsilon\right\}\geq1-\delta$$ by $\{|X_n-X|\leq\varepsilon\}\supset\left\{\underset{n\geq m}{\bigcap}|X_n-X|\leq\varepsilon\right\}.$ We have $$\mathscr{P}\left\{|X_n-X|>\varepsilon\right\}<\delta\;\forall\,n\geq m.$$  Since $\delta$ is arbitrary, $$\mathscr{P}\left\{|X_n-X|>\varepsilon\right\}\rightarrow0\mbox{ as }n\rightarrow\infty\;\forall\,\varepsilon>0.$$  Thus, $X_n\overset{p}{\longrightarrow}X.$

$\Box$

(2) Counterexample for $X_n\overset{p}{\longrightarrow}X\,\not\Rightarrow\, X_n\overset{a.s.}{\longrightarrow}X$.

$\bullet$ Counterexample.
Define for $n\geq1$, $\mathscr{P}\{X_n=0\}=1-\frac{1}{n}$, $\mathscr{P}\{X_n=1\}=\frac{1}{n}$ and $X_i$'s are independent.  For $\varepsilon>0$,  $$\mathscr{P}\{|X_n|>\varepsilon\}=\begin{cases}\mathscr{P}\{X_n=1\}=\frac{1}{n},&\mbox{if } 0<\varepsilon<1, \\ 0,&\mbox{if }\varepsilon\geq1.\end{cases}$$  Then $\underset{n\rightarrow\infty}{\lim}\mathscr{P}\{|X_n|>\varepsilon\}=0$ for all $\varepsilon>0$.  That is, $X_n\overset{p}{\longrightarrow}0$.

But, consider for $m>>0$, $$\mathscr{P}\left\{\bigcap_{n=m}^{n_0}X_n=0\right\}=\prod_{n=m}^{n_0}\left(1-\frac{1}{n}\right)=\frac{m-1}{m}\frac{m}{m+1}\cdots\frac{n_0-1}{n_0}=\frac{m-1}{n_0}\rightarrow0$$ as $n_0\rightarrow\infty.$  Thus, $\mathscr{P}\left\{\bigcap_{n\geq m}|X_n|\leq\varepsilon\right\}\rightarrow0$ for all $\varepsilon>0$ and $m>0$.  That is, $X_n\not\overset{a.s.}{\longrightarrow}0$.

$\Box$

(3) If $X_n\overset{p}{\longrightarrow}X$, then there exists a sequence $\{n_k\}$ such that $X_{n_k}\overset{a.s.}{\longrightarrow}X$.

$\bullet$ Proof.
$X_n\overset{p}{\longrightarrow}X\implies\forall\,k>0,\,\underset{n\rightarrow\infty}{\lim}\mathscr{P}\{|X_n-X|>\frac{1}{2^k}\}=0.$  Thus for each $k$, there exists $n_k$ such that $\mathscr{P}\{|X_{n_k}-X|>\frac{1}{2^k}\}\leq\frac{1}{2^k}$ and we have $$\sum_{k=1}^\infty\mathscr{P}\left\{|X_{n_k}-X|>\frac{1}{2^k}\right\}\leq\sum_{k=1}^\infty\frac{1}{2^k}<\infty.$$  By Borel-Cantelli Lemma, for the subsequence $\{n_k\}_{k\geq1}$, $$\mathscr{P}\left\{|X_{n_k}-X|>\frac{1}{2^k}\mbox{ i.o.}\right\}=0.$$  That is, $\mathscr{P}\left\{|X_{n_j}-X|\leq\frac{1}{2^j}\,\forall\,j\geq k\right\}\rightarrow1$ as $k\rightarrow\infty$ $\implies X_{n_k}\overset{a.s.}{\longrightarrow}X$.

$\Box$