Converge in Probability v.s. Converge in Distribution

About Posts which Tagged by 'Probability'

Let $X$ and $\{X_n\}_{n\geq1}$ be random variables with distribution functions $F$ and $\{F_n\}_{n\geq1}$.  $X_n$ converge to $X$ in probability implies $X_n$ converge to $X$ in distribution.  The converse is NOT true except for $F$ degenerating to a constant.




(1) $X_n\overset{p}{\longrightarrow}X\,\Rightarrow\, X_n\overset{d}{\longrightarrow}X$.

$\bullet$ Proof.
Let $x_0$ be the continuous point of $F$.  For $\varepsilon>0$,
$$\begin{array}{ccl}F_n(x_0)&=&\mathscr{P}(X_n\leq x_0) \\
& = &\mathscr{P}(\{X_n\leq x_0\}\cap[\{|X_n-X|\leq\varepsilon\}\cup\{|X_n-X|>\varepsilon\}]) \\
&\leq&\mathscr{P}(\{X_n\leq x_0\}\cap\{|X_n-X|\leq\varepsilon\})+\mathscr{P}(\{X_n\leq x_0\}\cap\{|X_n-X|>\varepsilon\}) \\
&\leq& \mathscr{P}(\{X_n\leq x_0\}\cap\{X_n-X\geq -\varepsilon\})+\mathscr{P}(|X_n-X|>\varepsilon) \\
&\leq & \mathscr{P}(X\leq x_0+\varepsilon)+\mathscr{P}(|X_n-X|>\varepsilon) \\
&\leq & F(x_0+\varepsilon)+\mathscr{P}(|X_n-X|>\varepsilon) \\ \end{array}$$
Similarly,
$$\begin{array}{ccl}1-F_n(x_0) &=& \mathscr{P}(X_n> x_0) \\
&=&\mathscr{P}(\{X_n> x_0\}\cap[\{|X_n-X|\leq\varepsilon\}\cup\{|X_n-X|>\varepsilon\}]) \\
&\leq&\mathscr{P}(\{X_n> x_0\}\cap\{|X_n-X|\leq\varepsilon\})+\mathscr{P}(\{X_n\leq x_0\}\cap\{|X_n-X|>\varepsilon\}) \\
&\leq&\mathscr{P}(\{X_n> x_0\}\cap\{X_n-X\leq\varepsilon\})+\mathscr{P}(|X_n-X|>\varepsilon) \\
&\leq&\mathscr{P}(X> x_0-\varepsilon)+\mathscr{P}(|X_n-X|>\varepsilon) \\
&\leq&1-F(x_0-\varepsilon)+\mathscr{P}(|X_n-X|>\varepsilon) \\ \end{array}$$
We have $$F_n(x_0)\leq F(x_0+\varepsilon)+\mathscr{P}(|X_n-X|>\varepsilon)$$ $$F(x_0-\varepsilon)\leq F_n(x_0)+\mathscr{P}(|X_n-X|>\varepsilon).$$ Since $X_n\overset{p}{\longrightarrow}X$, $\mathscr{P}(|X_n-X|>\varepsilon)\rightarrow0\mbox{ as }n\rightarrow\infty$.  Thus, $$F(x_0-\varepsilon)\leq\liminf_n F_n(x_0)\leq\limsup_n F_n(x_0)\leq F(x_0+\varepsilon).$$  Note that $x_0$ is the continuous point of $F$. Let $\varepsilon\rightarrow0$, we have $$F_n(x_0)\rightarrow F(x_0)\mbox{ as }n\rightarrow\infty.$$ That is, $X_n\overset{d}{\longrightarrow}X$.

$\Box$

(2) Counterexample for $X_n\overset{d}{\longrightarrow}X\,\not\Rightarrow\, X_n\overset{p}{\longrightarrow}X$.

$\bullet$ Counterexample.
Let $X$ and $\{X_n\}_{n\geq1}$ are i.i.d. random variables with standard normal distribution, $N(0,1)$.  Thus, we immediately conclude that $X_n\overset{d}{\longrightarrow}X \sim N(0,1)$.

But, for $\varepsilon>0$, $$\begin{array}{ccl}P\{|X_n-X|>\epsilon\}
&=&\mathscr{P}\{X_n-X<-\epsilon\}+\mathscr{P}\{X_n-X>\epsilon\}\\
&=&\mathscr{P}\left\{\frac{X_n-X}{\sqrt{2}}<-\frac{\epsilon}{\sqrt{2}}\right\}+\mathscr{P}\left\{\frac{X_n-X}{\sqrt{2}}>\frac{\epsilon}{\sqrt{2}}\right\} \\
&=&\Phi\left(-\frac{\epsilon}{\sqrt{2}}\right)+\left(1-\Phi\left(\frac{\epsilon}{\sqrt{2}}\right)\right) \\
&=&2\Phi\left(-\frac{\epsilon}{\sqrt{2}}\right)\neq0 \\ \end{array}$$ for all $n$.  Thus, $X_n\not\overset{p}{\longrightarrow}X.$

$\Box$

(3) If $X_n\overset{d}{\longrightarrow}c$, $c$ is a constant, then $X_n\overset{p}{\longrightarrow}c$.

$\bullet$ Proof.

For $\varepsilon>0$, $$\begin{array}{ccl}\mathscr{P}\{|X_n-c|\leq\varepsilon\}
&=&\mathscr{P}\{c-\varepsilon\leq X_n\leq c+\varepsilon\} \\
&=&\mathscr{P}\{X_n\leq c+\varepsilon\}-\mathscr{P}\{X_n< c-\varepsilon\} \\
&\rightarrow&\mathscr{P}\{X\leq c+\varepsilon\}-\mathscr{P}\{X< c-\varepsilon\}\mbox{ as }n\rightarrow\infty\mbox{ since } X_n\overset{d}{\longrightarrow}c \\
&=&1-0=1. \end{array}$$ Thus, we have $\underset{n\rightarrow\infty}{\lim}\mathscr{P}\{|X_n-c|>\varepsilon\}=0.$  That is, $X_n\overset{p}{\longrightarrow}c$.

$\Box$