### Limit Theorems

#### Weak law of large numbers

The sample average converges in probability to its expected value.

#### Strong law of large numbers

The sample average converges almost surely to its expected value.

#### Central limit theorem

The average of a large number of IID random variables, when shifted by the population average and scaled by the square root of the number of variables, approaches a normal distribution. More precisely, let {x1, x2, …, xn} be n IID random variables such that E(xi) = μ, Var(xi) = σ2. Let Sn = (x1 + x2 + … + xn)/n, then √n(Sn – μ) → N(0, σ2)

#### Extreme value theorem

The maximum (minimum) of a large number of IID random variables, when centered and scaled properly, converges to a generalized extreme value distribution. The Weibull, Gumbel, and Frechet distributions are members of this family. The centering and scaling here are not as universal as the central limit theorem.

#### Bootstrap theorem (for sample mean)

Let {x1, x2, …, xn} be an i.i.d with sample mean X̅, sample variance S, and a defined population variance. Let X̅B denote the mean of a bootstrap sample drawn from {x1, x2, …, xn}. Then, √n(X̅B – X̅)/S approaches the standard normal distribution in the limit of large n.