Extension of Strong Law of Large Number

About Posts which Tagged by 'Probability'

Let $\{X_n\}$ be a sequence of independent and identically distributed random variables with $\mathscr{E}|X_1|=\infty$.  Let $\{a_n\}$ be a sequence of positive numbers satisfying the condition $a_n/n \uparrow$.  Define $S_n=\sum_j X_j$.  Then we have $$\underset{n\rightarrow\infty}{\limsup}\frac{|S_n|}{a_n}=0\;\mbox{ a.s.,}\mbox{  or  }=+\infty\;\mbox{ a.s.}$$ according as $$\sum_n\mathscr{P}\{|X_n|\geq a_n\}=\sum_n\int_{|x|\geq a_n}\,dF(x)<\infty,\mbox{  or  }=+\infty.$$

$\bullet$ Proof. (The Converge Part)

Here we truncate $X_n$ in a different way because they have infinite moments.  Let $\mu_n=\int_{|x|<a_n}x\,dF(x)$ and define $$Y_n=\begin{cases}X_n-\mu_n, &\mbox{if }X_n<a_n\\ -\mu_n,&\mbox{if }X_n\geq a_n.\end{cases}$$  Before starting to prove this theory, we first give the outline of this proof.
(A1) We show the equivalence between $\{Y_n\}$ and $\{X_n-\mu_n\}$.  Then, we conclude the convergence of $\frac{1}{a_n}\sum_{j=1}^n X_j$ by finding that $\frac{1}{a_n}\sum_{j=1}^n Y_j$ and $\frac{1}{a_n}\sum_{j=1}^n\mu_j$ converge to zero almost everywhere in the following two steps.
(A2) We show that $\frac{1}{a_n}\sum_{j=1}^n Y_j\rightarrow0\;\mbox{a.s.}$ by following the proof of SLLN.
(A3) We show that $\frac{1}{a_n}\sum_{j=1}^n\mu_j\rightarrow0\;\mbox{a.s.}$
=======================================================================

First, consider the assumption $\sum_n\mathscr{P}\{|X_n|\geq a_n\}<\infty$.  $$\begin{array}{ccl} \sum_n\mathscr{P}\{|X_n|\geq a_n\}
& =  & \sum_n\int_{|X_n|\geq a_n}\,dF(x) \\
& =  & \sum_{n=0}^\infty\sum_{k=n}^\infty\int_{a_k\leq|X_n|<a_{k+1}}\,dF(x) \\
& =  & \sum_{k=1}^\infty k\int_{a_{k-1}\leq|X_n|<a_k}\,dF(x) \\
\end{array}$$ by exchanging the summations w.r.t. $n$ and $k$ according to the following arrangement for integral area. $$\begin{array}{c|ccccc}
n\setminus k & 0 & 1 & 2 & 3 & \cdots \\ \hline
0            & [a_0,a_1) & [a_1,a_2) & [a_2,a_3) & [a_3,a_4) & \cdots \\
1            &           & [a_1,a_2) & [a_2,a_3) & [a_3,a_4) & \cdots \\
2            &           &           & [a_2,a_3) & [a_3,a_4) & \cdots \\
3            &           &           &           & [a_3,a_4) & \cdots \\
\vdots       &           &           &           &           &        \\
\end{array}$$
Thus, $$\sum_n\mathscr{P}\{|X_n|\geq a_n\}<\infty\iff\sum_{k=1}^\infty k\int_{a_{k-1}\leq|X_n|< a_k}\,dF(x)<\infty.\qquad\mbox{(IMPT)}$$

(A1)
$\sum_n\mathscr{P}\{Y_n\neq X_n-\mu_n\}=\sum_n\mathscr{P}\{|X_n|\geq a_n\}<\infty$ by assumption.  We have, $\{Y_n\}$ is equivalent to $\{X_n-\mu_n\}\iff\{X_n\}$ is equivalent to $\{Y_n+\mu_n\}$.  Thus, since $a_n\uparrow$, $$\frac{1}{a_n}\sum_{j=1}^nX_j=\frac{S_n}{a_n}\rightarrow0\;\mbox{ a.s.}\iff\frac{1}{a_n}\sum_{j=1}^n(Y_j+\mu_j)\rightarrow0\;\mbox{ a.s.}$$  The next two parts discuss the convergence of $\frac{1}{a_n}\sum_{j=1}^nY_j$ and $\frac{1}{a_n}\sum_{j=1}^n\mu_j$.


(A2)
First, recall the following Theorem and its Corollary:

[Theorem] Let $\phi$ be a positive, even and continuous function on $(-\infty,\infty)$ such that as $|x|$ increases, $$\frac{\phi(x)}{|x|}\uparrow,\;\frac{\phi(x)}{x^2}\downarrow.$$ Suppose $\mathscr{E}(X_n)=0$ for every $n$ and $0<a_n\uparrow\infty$.  If, additionally, $\phi$ satisfies $$\sum_n\frac{\mathscr{E}\left(\phi(X_n)\right)}{\phi(a_n)}<\infty,$$ then $$\sum_n\frac{X_n}{a_n}\mbox{ converges  a.e.}$$See Application of Three Series Theorem on Strong Convergence

[Corollary] Under the some hypothesis, we have $\frac{1}{a_n}\sum_{j=1}^nX_j\rightarrow0\;\mbox{ a.e.}$

Back to (A2).  $$\begin{array}{ccl} \mathscr{E}(Y_n)
&=& \int_{|x|< a_n}(x-\mu_n)\,dF(x)+\int_{|x|\geq a_n}(-\mu_n)\,dF(x) \\
&=& \int_{|x|< a_n}x\,dF(x)-\mu_n\int\,dF(x) = \mu_n-\mu_n = 0. \\
\end{array}$$ Define $\phi(x)=x^2$, $$\begin{array}{ccl} \sum_n\frac{\mathscr{E}\left(\phi(Y_n)\right)}{\phi(a_n)}
& =  & \sum_n\frac{\mathscr{E}\left(Y_n^2\right)}{a_n^2} \\
&\leq& \sum_n\frac{1}{a_n^2}\int_{|x|<a_n}x^2\,dF(x)\quad\qquad\qquad\qquad\qquad\mbox{(Supp.1)} \\
& =  & \sum_{n=1}^\infty\frac{1}{a_n^2}\sum_{k=1}^n\int_{a_{k-1}\leq|x|<a_k}x^2\,dF(x) \\
& =  & \sum_{k=1}^\infty\left[\int_{a_{k-1}\leq|x|<a_k}x^2\,dF(x)\left(\sum_{n=k}^\infty\frac{1}{a_n^2}\right)\right]\qquad\mbox{(Supp.2)} \\
&\leq& \sum_{k=1}^\infty\left[a_k^2\left(\sum_{n=k}^\infty\frac{1}{a_n^2}\right)\int_{a_{k-1}\leq|x|<a_k}\,dF(x)\right] \\
&\leq& \sum_{k=1}^\infty 2k\int_{a_{k-1}\leq|x|<a_k}\,dF(x)\qquad\qquad\qquad\qquad\mbox{(Supp.3)} \\
& =  & 2\sum_n\mathscr{P}\{|X_n|\geq a_n^2\}<\infty\qquad\mbox{(IMPT)} \\
\end{array}$$ By the Theorem and Corollary above, we have $$\frac{1}{a_n}\sum_{j=1}^nY_j\rightarrow0\;\mbox{a.s.}$$

(A3)
$$\begin{array}{ccl} \left|\frac{1}{a_n}\sum_{k=1}^n\mu_k\right|
&\leq& \frac{1}{a_n}\sum_{k=1}^n\int_{|x|<a_k}|x|\,dF(x) \\
&\leq& \frac{1}{a_n}\sum_{k=1}^n\int_{|x|<a_n}|x|\,dF(x)\qquad(\because a_k\uparrow)\\
& =  & \frac{n}{a_n}\int_{|x|<a_n}|x|\,dF(x) \\
\mbox{for fixed }N<n
& =  & \frac{n}{a_n}\left[\int_{|x|<a_N}|x|\,dF(x)+\int_{a_N\leq|x|<a_n}|x|\,dF(x)\right] \\
&\leq& \frac{n}{a_n}\left[a_N+\int_{a_N\leq|x|<a_n}|x|\,dF(x)\right]. \\
\end{array}$$  We have that $\frac{1}{a_n}\sum_{k=1}^n\mu_k$ is bounded in absolute value by $$\frac{n}{a_n}a_N+\frac{n}{a_n}\int_{a_N\leq|x|<a_n}|x|\,dF(x),\mbox{ for any }N<n.$$  Now we consider the convergence of each part of this bound.  Via proving with contradiction, the first part converges to zero.  Note that, in the assumption, $a_n/n\uparrow$. Suppose $a_n/n$ is bounded by $M<\infty$ for all $n$.  Recall that $$\mathscr{E}|X|<\infty\iff\sum_n\mathscr{P}\{|X|\geq cn\}<\infty,\mbox{ where }c\mbox{ is a constant.}$$  Here, since $\mathscr{E}|X_1|=\infty$, we must have $$\mathscr{P}\{|X_1|\geq Mn\}\leq\mathscr{P}\{|X_1|\geq (a_n/n)n\}=\mathscr{P}\{|X_1|\geq a_n\}=\infty,$$ which contradict to the assumption, $\mathscr{P}\{|X_1|\geq a_n\}<\infty$.  Thus, we have that $a_n/n$ is not bounded.  That is, $$\frac{a_n}{n}\rightarrow\infty\implies\frac{n}{a_n}a_N\rightarrow0\mbox{ as }n\rightarrow\infty.$$  For the second part, $$\begin{array}{ccl} \frac{n}{a_n}\int_{a_N\leq|x|<a_n}|x|\,dF(x)
& =  & \frac{n}{a_n}\sum_{j=N+1}^n\int_{a_{j-1}\leq|x|<a_j}|x|\,dF(x) \\
&\leq& \frac{n}{a_n}\sum_{j=N+1}^na_j\int_{a_{j-1}\leq|x|<a_j}\,dF(x) \\
&\leq& \sum_{j=N+1}^n j\int_{a_{j-1}\leq|x|<a_j}\,dF(x)\;(\because j\leq n\Rightarrow\frac{a_j}{j}\leq\frac{a_n}{n}\Rightarrow\frac{n}{a_n}a_j\leq j) \\
&\leq& \sum_{j=N+1}^\infty j\int_{a_{j-1}\leq|x|<a_j}\,dF(x)
\end{array}$$  By (IMPT), since $\sum_{k=1}^\infty j\int_{a_{k-1}\leq|x|<a_k}\,dF(x)<\infty$, we have
$$\frac{n}{a_n}\int_{a_N\leq|x|<a_n}|x|\,dF(x)\leq\sum_{j=N+1}^\infty j\int_{a_{j-1}\leq|x|<a_j}\,dF(x)\rightarrow0\mbox{ as }N\rightarrow\infty.$$
Therefore, $$\left|\frac{1}{a_n}\sum_{k=1}^n\mu_k\right|\leq\frac{n}{a_n}a_N+\frac{n}{a_n}\int_{a_N\leq|x|<a_n}|x|\,dF(x)\rightarrow0\implies\frac{1}{a_n}\sum_{k=1}^n\mu_k\rightarrow0.$$

Finally, we summarize this proof.  Combining (A2) and (A3), we have $\frac{1}{a_n}\sum_{j=1}^n(Y_j+\mu_j)\rightarrow0\;\mbox{ a.s.}$  In (A1), by the equivalence property between $\{Y_n\}$ and $\{X_n-\mu_n\}$, we have

$$\frac{1}{a_n}\sum_{j=1}^nX_j=\frac{S_n}{a_n}\rightarrow0\;\mbox{ a.s.}\iff\frac{1}{a_n}\sum_{j=1}^n(Y_j+\mu_j)\rightarrow0\;\mbox{ a.s.}$$  This proof is completed.

Supplementary:
(Supp.1) $$\begin{array}{ccl} \frac{\mathscr{E}\left(Y_n^2\right)}{a_n^2}
& =  & \frac{1}{a_n^2}\left[\int_{|x|<a_n}(x-\mu_n)^2\,dF(x)+\int_{|x|\geq a_n}(-\mu_n)^2\,dF(x)\right] \\
& =  & \frac{1}{a_n^2}\left[\int_{|x|<a_n}x^2\,dF(x)-2\mu_n\int_{|x|<a_n}x\,dF(x)+\mu_n^2\int\,dF(x)\right] \\
& =  & \frac{1}{a_n^2}\left[\int_{|x|<a_n}x^2\,dF(x)-\mu_n^2\right] \\
\end{array}$$


(Supp.2)
Consider the fraction and the integral area in $$\sum_{n=1}^\infty\frac{1}{a_n^2}\left[\sum_{k=1}^n\int_{a_{k-1}\leq |x|<a_k}x^2\,dF(x)\right].$$ $$\begin{array}{c|cccc}
n\setminus k & 1 & 2 & 3 & \cdots \\ \hline
1                  &\frac{1}{a_1^2}\int_{[a_0,a_1)} & & & \\
2                  &\frac{1}{a_2^2}\int_{[a_0,a_1)} & \frac{1}{a_2^2}\int_{[a_1,a_2)} & & \\
3                  &\frac{1}{a_3^2}\int_{[a_0,a_1)} & \frac{1}{a_3^2}\int_{[a_1,a_2)} & \frac{1}{a_3^2}\int_{[a_2,a_3)} & \\
\vdots          & \vdots  & \vdots  & \vdots  & \cdots \\ \hline
                    & \sum_{n=1}^\infty\frac{1}{a_n^2}\int_{[a_0,a_1)} & \sum_{n=2}^\infty\frac{1}{a_n^2}\int_{[a_1,a_2)} & \sum_{n=3}^\infty\frac{1}{a_n^2}\int_{[a_2,a_3)} & \cdots \\
\end{array}$$  Then we can exchange the summation signs w.r.t. $n$ and $k$ as $$\sum_{k=1}^\infty\left\{\int_{a_{k-1}\leq|x|<a_k}x^2\,dF(x)\left(\sum_{n=k}^\infty\frac{1}{a_n^2}\right)\right\}.$$

(Supp.3)

$n\geq k\implies\frac{a_n}{n}\geq\frac{a_k}{k}$. Then $$\sum_{n=k}^\infty\frac{1}{a_n^2}=\sum_{n=k}^\infty\frac{1}{n^2}\frac{n^2}{a_n^2}\leq\frac{k^2}{a_k^2}\sum_{n=k}^\infty\frac{1}{n^2}.$$  Consider for fixed $N>k$, $$\begin{array}{rl}\sum_{n=k}^N\frac{1}{n^2} & =\frac{1}{k^2}+\sum_{n=k+1}^N\frac{1}{n^2}<\frac{1}{k}+\sum_{n=k+1}^N\frac{1}{n(n-1)} \\ & =\frac{1}{k}+\sum_{n=k+1}^N\left(\frac{1}{n-1}-\frac{1}{n}\right)=\frac{1}{k}+\frac{1}{k}-\frac{1}{N}\overset{N}{\rightarrow}\frac{2}{k}.\end{array}$$  Thus, $$\sum_{n=k}^\infty\frac{1}{a_n^2}\leq\frac{k^2}{a_k^2}\sum_{n=k}^\infty\frac{1}{n^2}\leq\frac{k^2}{a_k^2}\frac{2}{k}=\frac{2k}{a_k^2}.$$

$\Box$

$\bullet$ Proof. (The Diverge Part)

The proof of the divergence part is identical to the same part in the proof of SLLN.  Please see Strong Law of Large Number.

$\Box$