Convergence Modes and Their Relationship

About Posts which Tagged by 'Probability'

Let $\Omega$ be the sample space, and, $X$ and $\{X_n\}_{n\geq1}$ be random variables with distribution functions $F$ and $F_n$.  There are four common modes of convergence.

1. Converge almost surely.  [See More]$$\begin{array}{ccl}X_n\overset{a.s.}{\longrightarrow}X &\Leftrightarrow&\exists\mbox{ null set }N\mbox{ such that }\forall\,\omega\in\Omega\setminus N,\,\underset{n\rightarrow\infty}{\lim}X_n(\omega)=X(\omega)\mbox{ finite} \\
& & \\
&\Leftrightarrow&\forall\,\varepsilon>0,\,\underset{m\rightarrow\infty}{\lim}\mathscr{P}\left\{|X_n-X|\leq\varepsilon,\,\forall\,n\geq m\right\}=1 \\
& &\qquad\mbox{or, }\underset{m\rightarrow\infty}{\lim}\mathscr{P}\left\{|X_n-X|>\varepsilon,\,\mbox{for some}\,n\geq m\right\}=0 \\
& & \\
&\Leftrightarrow&\forall\,\varepsilon>0,\,\mathscr{P}\left\{|X_n-X|>\varepsilon,\,\mbox{i.o.}\right\}=\mathscr{P}\left\{\bigcap_{m=1}^\infty\bigcup_{n\geq m}\{|X_n-X|>\varepsilon\} \right\}=0 \\ \end{array}$$
2. Converge in $r$-th mean.  $$X_n\overset{L^r}{\longrightarrow}X\iff X_n\in L^r,\,X\in L^r\mbox{ and }\underset{n\rightarrow\infty}{\lim}\mathscr{E}\left(|X_n-X|^r\right)=0.$$
3. Converge in probability.  $$X_n\overset{p}{\longrightarrow}X\iff\forall\,\epsilon>0,\,\underset{n\rightarrow\infty}{\lim}\mathscr{P}\left\{|X_n-X|>\varepsilon\right\}=0.$$
4. Converge vaguely.  Let $\{\mu_n\}_{n\geq1}$ and $\mu$ be subprobability measures (s.p.m.'s, $\mu(\mathbb{R})\leq1$) on $(\mathbb{R}, \mathscr{B})$, where $\mathscr{B}$ is a Borel field. $$\begin{array}{rcl}\mu_n\overset{v}{\longrightarrow}\mu&\iff&\exists\,\mbox{a dense set }D\in\mathbb{R}\mbox{ such that }\\
& &\forall\,a,b\in D,\,a<b,\;\mu_n((a,b])\rightarrow\mu((a,b])\mbox{ as }n\rightarrow\infty.\end{array}$$
5. Converge in distribution.  $$\begin{array}{rl}X_n\overset{d}{\longrightarrow}X&\iff&\forall\,x\in C(F)=\{\mbox{points that }F\mbox{ is continuous}\},\\ & &F_n(x)\rightarrow F(x)\mbox{ as }n\rightarrow\infty.\end{array}$$

The relationship between those modes are as follows
$$\begin{array}{ccccccc}
X_n\overset{a.s.}{\longrightarrow}X & \Rightarrow
& X_n\overset{p}{\longrightarrow}X & \Rightarrow
& X_n\overset{d}{\longrightarrow}X & \equiv
& \mu_n\overset{v}{\longrightarrow}\mu \\
& & \Uparrow& & & &\\
& & X_n\overset{L^r}{\longrightarrow}X & & & &\\ \end{array}$$The converse are false except for some special cases.

$\star$ Converge Almost Surely v.s. Converge in r-th Mean
$\star$ Converge Almost Surely v.s. Converge in Probability
$\star$ Converge in r-th Mean v.s. Converge in Probability
$\star$ Converge in Probability v.s. Converge in Distribution
$\star$ Converge in Distribution and Vague Convergence (1): Equivalence for s.p.m.'s
$\star$ Converge in Distribution and Vague Convergence (2): Equivalence for p.m.'s