\begin{center}
{\textbf { 1. INTRODUCTION}}
\end{center}

\textbf{ 1.1  What is Statistics?}

{\sl {STATISTICS}}: a subject dealing with the \textbf{collection} and
the \textbf{use} of data in order to (help to) answer the questions
like:

\begin{quote}
{\small
$\bullet$ Do stock markets rise and fall randomly?

$\bullet$ Is global warming really happening?

$\bullet$ Does a certain new drug prolong life for AIDS sufferers?

$\bullet$ Are GCSE and A level examinations standards declining?

$\bullet$ Is the national lottery making us a nation of compulsive
gamblers?

$\bullet$ Is the gap between rich and poor widening in Britain?

$\bullet$ Do Persil adverts really make us want to buy Persil?

$\bullet$ $\cdots \cdots$
}
\end{quote}

Those questions are difficult to study in laboratory, and admit no
self-evident axioms

Typically {\sl Data} are subject to \textbf{uncertainty} --- {\sl
 random variables}

Statistics: \textbf{Let the Data Speak}!

\begin{itemize}
\item making guesses about the process generating the data -- {\sl
Estimation} \item testing the guesses -- {\sl  Testing hypotheses}
\item forecasting future -- {\sl   Prediction}
\end{itemize}

\vspace{1cm} {\small \textbf{ Note}. Statistics also deals with {\sl
experiment design} which will be untouched in this course. }


{\sl Statistics} offers quantitative approaches to solve practical
problems. Therefore, \textbf{ a good knowledge} on the practical world
behind data is essential for decent statistics practice.

\textbf{No unique} statistical solution for most practical problems!
Therefore
\begin{center}
{\sl Statistics is also an art}
\end{center}



Some guidelines for learning/applying statistics:
\begin{quote}
$\bullet$ Understand what data say in each specific context. All the methods
are just tools to help understand data

$\bullet$ Concentrate on what to do and why, rather than concrete calculation
and graphing

$\bullet$ It may take a while to catch the essences of statistics -- {
Keep thinking}!
\end{quote}


\textbf{ What is the difference between {\sl probability} and {\sl
statistical inference}? }

Probability --- a { mathematical} subject

Statistics --- an { applied oriented} subject: inference based on {
data} plus { `assumptions'}

{\small \textbf{ Note.} The assumptions imposed in statistical
inference are typically based on probability theory.}
\bigskip

Consider a simple experiment:  a coin is tossed $n$ times.

Define a random variable $X\; =$ the number of `heads'.


\textbf{ Probability Questions}:

{\small
\begin{itemize}
\item What is $P(X=1)$, $P(X\le 1)$, $P(X=x)$ or $P(X\le x)$?
\item What is $E(X)$, Var$(X)$, $E(X^m)$ or $E(e^{tX})$?
\end{itemize}
}

\textbf{ Probability Answers}

If we \textbf{assume} that the tosses are independent with a constant
probability of success $\pi$, then
\begin{itemize}
\item   $P(X=1)=n\pi(1-\pi)^{n-1}$, \\ $P(X\le 1)=(1-\pi)^{n-1}(1-\pi+n\pi)$,\\
$P(X=x)=\begin{pmatrix}n\\x\end{pmatrix}\pi^x(1-\pi)^{n-x}$, etc
\item  $E(X)=n\pi$, Var$(X)=n\pi(1-\pi)$, etc
\end{itemize}


\textbf{ Statistical Questions}

{\small
\begin{itemize}
\item What is the value of $\pi$?
\item Is there any evidence that the coin is `biased' i.e. $\pi \not =
0.5$?
\item Are the tosses independent?
\end{itemize}
}

\textbf{ Statistical Answers}
\begin{itemize}
\item Estimate $\pi$ by the estimator $x/n$. Give an interval in which
$\pi$ may be expected to fall.
\item Reject the Null Hypothesis that $\pi =0.5$ if $x$ is too far away
from $n/2$
\item Reject the model of independent trials if there is too much
regularity in the observations - e.g. HTHTHTHTHTHTHTHT....
\end{itemize}


\textbf{  1.2 Terminology and Notation}

The classic statistical inference is based on a \textbf{ sample} (i.e.
a set of \textbf{data})
\[
\bX =(X_1, \cdots, X_n)^T
\]
and the \textbf{assumption} that $\{ X_i \}$ i.i.d. with
\[
X_i \sim F_{X}(x; \theta), \quad \theta \in \Theta,
\]
where $F_{X}(\cdot; \theta) $ is a probability distribution and is
known up to an {\sl unknown} \textbf{ parameter} $\theta$, $\Theta$ is
the \textbf{ parameter space}.

\textbf{ Population}: $\{ F_{X}(\cdot; \theta), \; \theta \in
\Theta\}$, which is assumed to contain the \textbf{true} distribution
of $X_i$

\textbf{ Sample space}: consisting of all possible values of $\bX$

\textbf{ Statistic}: any (known) function of $\bX$ only


\textbf{  Dual identity} of a sample:
\begin{itemize}
\item
 a set of observed real
numbers in practice, and \item
 a set of i.i.d. r.v.s in theoretical exploration.
\end{itemize}

\textbf{Goal of statistical inference} in this course: estimating the
true value of $\theta$ or testing the hypotheses about $\theta$



\textbf{ Note}.  { A sample} or  { a random sample} refers to a set of
i.i.d. observations.


\textbf{ Example 1}. Let  $X_1, \cdots , X_n$ be i.i.d. from $N(\mu,
\sigma^2)$.  Then $(X_1 , \cdots, X_n)$ is a sample, $R^n$ is the
sample space, and
\[
\theta = \begin{pmatrix}\mu\\ \sigma^2\end{pmatrix},
\quad \quad
\Theta = R \times R^+.
\]
The population is
\[
\big\{ N(\mu, \sigma^2), \; (\mu, \sigma^2) \in \Theta \big\}.
\]
Both sample mean $\bar X = n^{-1} \sum_{1\le i\le n} X_i $ and sample
variance $S={\frac{1}{n-1}} \sum_{1\le i\le n}(X_i -\bar X)^2$ are
statistics. But $\bar X - \mu$ is not!


\bigskip

{\small \textbf{ Note}. The above setting is referred as a \textbf{
parametric model} since the population is known up to unknown
parameters.

In contrast, a \textbf{ nonparametric model} specifies that the true
distribution belongs to a class which cannot be specified by a { finite
number} of parameters only. For example, a population may be
\[
\{ \mbox{all continuous density functions in one variable} \}.
\]
}

We deal with parametric models first.


\begin{center}
{ Unknown Real World}
\end{center}

{

\psset{framearc=.4,  arrows=->}

\begin{psmatrix}[rowsep=100pt,colsep=118pt]
\psframebox{   Population }  & \psframebox{ $
\begin{array}{c}
{\rm
Distribution}\\
F_X(\cdot; \theta)
\end{array} $} \\
\psframebox{
$
\begin{array}{c}
\mbox{A sample} \\
X_1, \cdots, X_n
\end{array} $} &
\psframebox{
$
\begin{array}{c}
i.i.d.\mbox{ observations}\\
X_1, \cdots, X_n
\end{array}
$} \psset{linestyle=solid} \psset{offset=.2}
\ncline{1,1}{2,1}\naput{\tiny{Sampling}} \psset{offset=.2}
\ncline{2,1}{1,1}\naput{\tiny{Inference}} \psset{offset=.2}
\ncline{1,2}{2,2}\naput{\tiny{Sampling}} \psset{offset=.2}
\ncline{2,2}{1,2}\naput{\tiny{Inference}} \psset{linestyle=dotted}
\ncline{1,1}{1,2}\naput{\tiny{Math. Model}}
\ncline{2,1}{2,2}\naput{\tiny{Math. Model}}
\end{psmatrix}
\begin{center}
Known Information/Data
\end{center}
}



\textbf{  1.3 Statistics from Normal samples}

In this section we always assume that
$\bX= (X_1, \cdots, X_n)^\tau$ be a random sample from $N(0, 1)$.

Then the two summary statistics are sample mean and sample variance:
\[
\bar X = {\frac{1}{n}} \sum_{j=1}^n X_i, \quad S^2 = {1\over n-1}
\sum_{j=1}^n(X_i-\bar X )^2.
\]

\textbf{ Theorem 1}. The following statements hold.
\begin{quote}
(i) $\bar X \sim N(0, 1/n)$,\\
(ii) $(n-1)S^2 \sim \chi^2_{n-1}$,\\
(iii) $\bar X$ and $S^2 $ are independent.
\end{quote}

\textbf{ Note}. The distribution of $\sum_{1\le j\le n} X_i^2 $ is
called \textbf{  the $\chi^2$-square distribution with $n$ degrees of
freedom}, denoted as $\chi^2_n$ or $\chi^2(n)$.

\textbf{ Proof}.  We make a linear transformation from $X_1, \dots
,X_n$ to $ \bZ= (Z_1, \dots ,Z_n)^\tau$ in such a way that the $Z_i$'s
are still i.i.d. $N(0,1)$. This can be achieved by any transformation
$\bZ = \bH \bX$ with $\bH$ being a $n\times n$ orthogonal matrix (i.e.
$\bH^\tau \bH = \bI_n$). Let the first row of  $\bH$ be
\[
n^{-1/2}(1, 1,\dots  ,1).
\]
Then $Z_1 = \sqrt{n} \bar X$. Hence (i) holds.
Since
\[
\sum_{j=1}^n Z_j^2 = \bX^\tau \bH^\tau \bH \bX = \bX^\tau \bX
= \sum_{j=1}^n X_j^2,
\]
it holds that
\[
\sum_{j=2}^n Z_j^2 = \sum_{j=1}^n X_j^2 - Z_1^2 = \sum_{j=1}^n X_j^2 - n \bar X^2
=(n-1) S^2.
\]
Therefore (ii) and (iii) also hold.

\bigskip

{\small \textbf{ Remark}. The above proof indicates the following
decomposition {
\[
\begin{array}{ccccc}
\sum_{i=1}^n X_i^2& =& \sum_{i=1}^n(X_i-\bar{X})^2 &+& n\bar{X}^2\\
\chi^2_n&&\chi^2_{(n-1)}&&\chi^2_{(1)},
\end{array}
\]}
where the two $\chi^2$ random variables on the RHS are independent.
}



\textbf{ Example 1}. Let $Y_1, \cdots, Y_n$ iid from $N(\mu,
\sigma^2)$. Define
\[
X_i = (Y_i -\mu )/\sigma.
\]
Then $\{ X_i \}$ i.i.d. $N(0,1)$, and
\[
\bar Y = \sigma \bar X + \mu, \quad
S_y^2 = \sigma^2 S_x^2.
\]
Hence
\begin{quote}
(i) $ \bar Y \sim N(\mu, \sigma^2/n)$,

(ii) $(n-1)S^2_y/\sigma^2 \sim \chi^2_{n-1}$,

(iii) $\bar Y $ and $ S_y^2$ are independent.
\end{quote}
Further,
\[
\sum_{i=1}^n \frac{(Y_i-\mu)^2}{\sigma^2}
=\sum_{i=1}^n\frac{(Y_i-\bar{Y})^2}{\sigma^2}+n\frac{(\bar{Y}-\mu)^2}{\sigma^2}.
\]




\textbf{  F distribution}

\textbf{ Definition}. Let $Y\sim\chi^2_{n}$ and $X\sim\chi^2_{m}$, and
$X$ and $Y$ are independent. Then the random variable
\[
U \equiv { X/m \over Y/n} = { n\over m} { X \over Y}
\]
has an $F$-distribution with degrees of freedom $(m,n)$.
We write $U \sim F(m,n)$ or $U\sim F_{m,n}$.

Obviously, $1/U \sim F_{n,m}$.

\textbf{A formal notation}: $ F_{m,n} = { \chi^2_m/m \over \chi^2_n
/n}. $



\textbf{Typical use of F distribution}

Let
$X_1, X_2,\dots ,X_m$ and $Y_1, Y_2,\dots ,Y_n$ be two indep
samples from $N(\mu_x, \sigma^2_x)$ and $N(\mu_y, \sigma^2_y)$ respectively. Then
\[
\frac{\frac{1}{(m-1)\sigma_x^2}\sum_{i=1}^m (X_i-\bar{X})^2}{\frac{1}{(n-1)\sigma_y^2}\sum_{i=1}^n (Y_i-\bar{Y})^2}\sim F_{(m-1),(n-1)}.
\]


\textbf{  Student's $t$-distribution}

\textbf{ Definition}. Let $Z\sim N(0,1)$ and $U\sim \chi^2_{(k)}$, and
$Z$ and $U$ be independent. Then the random variable
\[
T=\frac{Z}{[\frac{1}{k}U]^{\frac{1}{2}}},
\]
has a $t$-distribution with $k$ degrees of freedom.
We write $T\sim t_k$ or $T\sim t(k)$.

Obviously, $T^2 \sim F_{1,k}$.

Also $T$ and $-T$ have the same distribution, i.e. the pdf is symmetric.
In fact $ET=0$ for $k>1$.

\textbf{A formal notation}: $ t_k = {N(0,1) \over \{ \chi^2_k/k\}^{1/2}
}.
 $


\textbf{Typical use of t distribution}

If $Y_1, Y_2,\dots  ,Y_n$ are iid $N(\mu, \sigma^2)$, then
\[
\frac{\bar{Y}-\mu}{\sigma/\sqrt{n}}\sim N(0,1),
\]
and
\[
(n-1) S^2_y /\sigma^2 =
\frac{\sum_{i=1}^n(Y_i-\bar{Y})^2}{\sigma^2}\sim \chi^2_{(n-1)}
\]
and these two are independent. It follows that
\[
\frac{\bar{Y}-\mu}{\sqrt{S^2_y/n}} \sim t_{n-1}.
\]
