About Posts which Tagged by 'Probability'

$\star$ Contents

Probability Theories
  Definitions
    Probability Measure

  Important Lemma and Theorems
    Convergence Theorems
    Proof of Fatou's Lemma
    Borel-Cantelli Lemma
    Counterexample for Converse of Borel-Cantelli Lemma I
    Varied Type of Borel-Cantelli Lemma I
    Varied Type of Borel-Cantelli Lemma II
    Extension of Borel-Cantelli Lemma II
 
  About The Expectation
    Expectation and Tail Probability (1)
    Expectation and Tail Probability (2)
    Expectation and Tail Probability (3)
    Independence and Fubini's Theorem

  Inequalities
    Inequalities for Random Variable

  Convergence Theories
    Convergence Modes and Their Relationship
    Almost Surely Convergence
    Converge Almost Surely v.s. Converge in r-th Mean
    Converge Almost Surely v.s. Converge in Probability
    Converge in r-th Mean v.s. Converge in Probability
    Converge in Distribution and Vague Convergence (1): Equivalence for s.p.m.'s
    Converge in Distribution and Vague Convergence (2): Equivalence for p.m.'s
    Converge in Probability v.s. Converge in Distribution
    Slutsky's Theorem
    Varied Type of Slutsky's Theorem (1): Converge in Probability
    Varied Type of Slutsky's Theorem (2): Converge in r-th Mean
    Uniformly Integrability
    Convergence of Moments (1)
    Convergence of Moments (2)
    Convergence of Moments (3)

  The Law of Large Numbers
    Simple Limit Theorems
    Weak Law of Large Number
    Extension of Weak Law of Large Number (1)
    Extension of Weak Law of Large Number (2)
    Kolmogorov's Three Series Theorem
    Equivalence of Convergence of Sum of Random Variables
    Application of Three Series Theorem on Strong Convergence
    Strong Law of Large Number
    Extension of Strong Law of Large Number
    Strong LLN v.s. Weak LLN

  Characteristic Function
    Characteristic Functions
    Convergence of the Characteristic Functions
    Representation of the Characteristic Function

  The Central Limit Theorems
    The Classical Central Limit Theorem
    Uniformly Asymptotically Negligible
    Uniformly Asymptotically Negligible (2): Connect to the Characteristic Function
    Lyapunov's Central Limit Theorem
    Linderberg-Feller's Central Limit Theorem (short version)
    Linderberg-Feller's Central Limit Theorem (completed)
    Lindeberg's Condition Implies Each Variance to Be Similarly Small
    Counterexample for Omitting UAN Condition in Feller's Proof
    Lindeberg's CLT v.s. Lyapunov's CLT

Applications
    Application of Fubini's Theorem (1)
    Application of Fubini's Theorem (2)
    Application of Fatou's Lemma
    Application of Dominate Convergence Theorem
    Application of Borel-Cantelli Lemma
    Related Topic with Uniformly Integrable

    Cantelli's Law of Large Numbers
    Application of Three Series Theorem on Strong Convergence

    Application of the Characteristic Function (1)
    Application of the Characteristic Function (2)

    Application of The Classical Central Limit Theorem (1)
    Application of The Classical Central Limit Theorem (2)

    Application of Lyapunov's Central Limit Theorem (1)
    Application of Lyapunov's Central Limit Theorem (2): Coupon Collector's Problem
    Application of Lyapunov's Central Limit Theorem (3)
    Application of Lyapunov's Central Limit Theorem (4)

    Application of Lindeberg's Central Limit Theorem (1)
    Application of Lindeberg's Central Limit Theorem (2)
    Application of Lindeberg's Central Limit Theorem (3): NOT converge to Normal

$\star$ All the content of the posts tagged by 'Probability' are not my original publication.  They are my notes of a class in 2014 Spring, named "Advanced Probability Theory", in Dept. of Stat., NCKU, Taiwan.  The readers can also find the similar contents in the following textbooks, or any articles which introduce the probability theory.

$\bullet$ Reference
Billingsley, P. (1995) Probability and Measure. John Wiley & Sons.
Chung, K. L. (2001). A course in probability theory. Academic press.
Ferguson, T. S. (1996). A course in large sample theory. London: Chapman & Hall.

Varied Type of Borel-Cantelli Lemma II

About Posts which Tagged by 'Probability'

Let $\{E_n\}$ be arbitrary events in $\mathscr{F}$.  If  for each $m$, $\sum_{n>m}\mathscr{P}\{E_n\mid E_m^c\cap\cdots\cap E_{n-1}^c\}=\infty$, then $\mathscr{P}\{E_n\mbox{ i.o.}\}=1.$

$\bullet$ Proof.

Convergence of Moments (3)

About Posts which Tagged by 'Probability'

Let $\{X_n\}$ and $X$ be random variables.  Let $0<r<\infty$, $X_n\in L^r$, and $X_n\rightarrow X$ in probability.  Then the following three propositions are equivalent.

(1) $\{|X_n|^r\}$ is uniformly integrable;
(2) $X_n\rightarrow X$ in $L^r$;
(3) $\mathscr{E}|X_n|^r\rightarrow\mathscr{E}|X|^r<\infty$.

$\bullet$ Proof.

Convergence of Moments (2)

About Posts which Tagged by 'Probability'

Let $\{X_n\}$ and $X$ be random variables.  If $X_n$ converges in distribution to $X$, and for some $p>0$, $\sup_n\mathscr{E}|X_n|^p=M<\infty$, then for each $r<p$, $$\underset{n\rightarrow\infty}{\lim}\mathscr{E}|X_n|^r=\mathscr{E}|X|^r<\infty.$$

$\bullet$ Proof.

Convergence of Moments (1)

About Posts which Tagged by 'Probability'

Let $\{X_n\}$ and $X$ be random variables.  If $X_n\rightarrow X$ a.e., then for every $r>0$, $$\mathscr{E}|X|^r\leq\underset{n\rightarrow\infty}{\underline{\lim}}\mathscr{E}|X_n|^r.$$If $X_n\rightarrow X$ in $L^r$, and $X\in L^r$, then $\mathscr{E}|X_n|^r\rightarrow\mathscr{E}|X|^r$.

$\bullet$ Proof.

Characteristic Functions

About Posts which Tagged by 'Probability'

For any random variable $X$ with probability measure $\mu$ and distribution function $F$, the characteristic function (ch.f.) is a function $f$ on $\mathbb{R}$ defined as $$f(t)=\mathscr{E}\left(e^{itX}\right)=\int_{-\infty}^\infty e^{itx}\,dF(x)\mbox{  for all }t\in\mathbb{R}.$$There are some simple properties of ch.f.:

Cantelli's Law of Large Numbers

About Posts which Tagged by 'Probability'

If $\{X_n\}$ are independent random variables such that the fourth moments $\mathscr{E}(X_n^4)$ have a common bound and define $S_n=\sum_{j=1}^nX_j$, then $$\frac{S_n-\mathscr{E}(S_n)}{n}\rightarrow0\mbox{  a.e.}$$

$\bullet$ Proof.
WLOG, suppose $\mathscr{E}(X_n)=0$ for all $n$ and denote the common bound of $\mathscr{E}(X_n^4)$ to be $$\mathscr{E}(X_n^4)\leq M_4<\infty\mbox{  for all }n.$$Then by Lyapunov's inequality, we have the second moments $$\mathscr{E}|X_n|^2\leq\left[\mathscr{E}|X_n|^4\right]^\frac{2}{4}\leq \sqrt{M_4}<\infty.$$Consider the fourth moment of $S_n$, $$\begin{array}{rl}\mathscr{E}(S_n^4)
&=\mathscr{E}\left[\left(\sum_{j=1}^nX_j\right)^4\right]\\ &= \mathscr{E}\left[\sum_{j=1}^nX_j^4+{4\choose1}\sum_{i\neq j}X_iX_j^3+{4\choose2}\sum_{i\neq j}X_i^2X_j^2\right.\\ &\quad\left.+{4\choose1}{3\choose1}\sum_{i\neq j\neq k}X_iX_jX_k^2+{4\choose1}{3\choose1}{2\choose1}\sum_{i\neq j\neq k\neq l}X_iX_jX_kX_l\right]\\
&=\sum_{j=1}^n\mathscr{E}(X_j^4)+4\sum_{i\neq j}\mathscr{E}(X_i)\mathscr{E}(X_j^3)+6\sum_{i\neq j}\mathscr{E}(X_i^2)\mathscr{E}(X_j^2)\quad(\because\mbox{ indep.})\\ &\quad+12\sum_{i\neq j\neq k}\mathscr{E}(X_i)\mathscr{E}(X_j)\mathscr{E}(X_k^2)+24\sum_{i\neq j\neq k\neq l}\mathscr{E}(X_i)\mathscr{E}(X_j)\mathscr{E}(X_k)\mathscr{E}(X_l)\\ &=\sum_{j=1}^n\mathscr{E}(X_j^4)+6\sum_{i\neq j}\mathscr{E}(X_i^2)\mathscr{E}(X_j^2)\qquad\qquad(\because\mbox{ assuming }\mathscr{E}(X_n)=0.) \\ &\leq nM_4+3n(n-1)\sqrt{M_4}\sqrt{M_4}=n(3n-2)M_4.\end{array}$$By Markov's inequality, for $\varepsilon>0$, $$\mathscr{P}\{|S_n|>n\varepsilon\}\leq\frac{\mathscr{E}(S_n^4)}{n^4\varepsilon^4}\leq\frac{n(3n-2)M_4}{n^4\varepsilon^4}=\frac{3M_4}{n^2\varepsilon^4}+\frac{2M_4}{n^3\varepsilon^4}.$$Thus, $$\sum_n\mathscr{P}\{|S_n|>n\varepsilon\}\leq\sum_n\frac{3M_4}{n^2\varepsilon^4}+\frac{2M_4}{n^3\varepsilon^4}<\infty.$$By Borel-Cantelli Lemma I, we have $$\mathscr{P}\{|S_n|>n\varepsilon\mbox{ i.o.}\}=0\implies\frac{S_n}{n}\rightarrow0\mbox{  a.e.}$$

$\Box$