A brief review of probability theory

Fundamental rules

product rule:

$p(A,B)=p(A \cap B)=p(A|B)p(B)$
yield chain rule:
$p(X_{1:D}) = p(X_1)p(X_2\mid X_1)p(X_3\mid X_1,X_2)\dots p(X_D \mid X_{1:D-1})$

sum rule:

$p(A) = \sum_{i}{p(A\mid B_{i})p(B_i)}$

Bayes rule:

$p(X \mid Y)= \frac{p(X,Y)}{p(Y)}=\frac{p(Y\mid X)p(X)}{\sum_{i}p(Y,X_i)p(X_i)}$

Quantiles(分位数)

cdf是 $F$ ，逆函数是 $F^{-1}$ ， $\alpha$ 分位数的作用是，有 $x_{\alpha} = F^{-1}(\alpha)$ , 表示的意思是 $p(X\le x_{\alpha})=\alpha$ 。

也就是说， $\alpha$ 是一个概率值，代入累积分布的逆函数中，返回的是对应概率面积的截断点：

根据公式
$p(a \lt X \le b) = F(b) − F(a)$
测试：

import numpy as np
import scipy.stats as st
st.norm.ppf(0.975) # 1.959963984540054
st.norm.ppf(0.025) #  -1.9599639845400545
st.norm.cdf(1.959963984540054) - st.norm.cdf(-1.9599639845400545) #  0.95

The empirical distribution

image.png

Degenerate pdf

Covariance matrix:

image.png
Pearson correlation coefcient :
$Corr(X,Y) = \frac{\ cov(X,Y)}{\sqrt{var(X)\sqrt{\ var(Y)}}}$

$Corr(X,Y) =1$ ,则 $X$ 与 $Y$ 线性相关，不一定相互独立；但是相互独立则不线性相关， $Corr(X,Y) =0$ 。

Transformations of random variables

多元分布的变量转换公式：
$p_y(\mathbf y) = p_x(\mathbf x)\mid \det \mathbf J_{y \rightarrow x} \mid$
Jacobian matrix $\mathbf J$ :
image.png
例子：
image.png

Central limit theorem

Monte Carlo approximation

作用：Approximate the distribution of $f(X)$ by using the empirical distribution of ${f(x_s)}^S_{s=1}$ .
例子：

image.png

求 $\pi$ 值的例子：

def generation_point(nums,r):
    x = np.random.uniform(-r, r,nums) 
    y = np.random.uniform(-r, r, nums) 
    r_square = np.sqrt(np.square(x) + np.square(y))
    count = r_square<r 
    pi = sum(count) / nums * 4 
    return x, y,count, pi
x, y, count, pi = generation_point(10000,4)
print(pi) # 3.138

# plot

data = ps.DataFrame({'x':x,'y':y,"color":count})
sns.set(style='darkgrid')
plt.figure(dpi=300)
sns.scatterplot(x="x",y="y",data=data,hue='color')

面积图