Category: probability and statistics

Sufficient Statistics

Statistical Inference Statistical inference, or “learning” as it is called in CS, is the process of using data to infer the distribution that generated the…

Limit Theorems Weak law of large numbers: casinos work for small mean but large variance–a lot of nerve is required.. Central Limit Theorem In this…

Stochastic Convergence In order to understand what ”stochastic converge“ is, we have to remember first what ”deterministic convergence“ means. Deterministic Convergence Definition: A deterministic sequence…

Probabilistic Inequalities Markov’s Inequality Theorem: If $X$ is a random variable, $X>0$ and $a>0$ is a positive real number, then \begin{eqnarray} \mathrm{P}(X>a) \leqslant \frac{\mathrm{E}(X)}{a} \end{eqnarray}…

Multivariate Distributions Random Vectors Definition: A vector $X$ is called a random vector if it has an associated pdf $f(.)$ \begin{eqnarray} X =\left[ \begin{array}{c} x_1\\…

Beta and Dirichlet Distributions Beta Distribution Beta distribution is frequently used as a prior. Dirichlet Distribution and Dirichlet Process Dirichlet distribution is a multinomial generalization…

Sample mean, sample variance and Bessel’s correction Sample mean is defined as \begin{eqnarray} \overline{X} = \frac{1}{n}\sum_{i=1}^n X_i \end{eqnarray} and sample variance is defined as \begin{eqnarray}…

Snedecor’s F-distribution These two distributions are extremely important in statistics, as they underly the t-test and F-test. Here we simply define these distributions without touching…