【ML】EM Algorithm

作者: 盐果儿 | 来源:发表于2022-08-28 02:02 被阅读0次

【ML】EM Algorithm
EM Algorithm
EM算法
（EM算法）The EM Algorithm
EM 算法
机器学习(十):EM算法与GMM算法原理及案例分析
无监督学习聚类分析④
（十）EM算法
基于R的高级统计（EM算法）
Expectation Maximum Algorithm(EM

EM Algorithm is short for Expectation-Maximization Algorithm. It's an iterative method to find maximum likelihood in statistical models where the model depends on unobserved latend variables.

Properties:

It used when the dataset is incomplete.

It's an unsupervised model.

Example:

We have a transcript, but we don't know which class the students belong to.

1. Initial guess: $P(c_{1}) = P(c_{2}) = 0.5$

2. Expectation Step: Using the initial guess, we got the value of the marigianal likelihood (prior proabaility).

The probability density function:

$P(x | c_{1}) = \frac {1}{\sqrt {2 \pi \sigma _{1}}} exp(- \frac {(x - \mu_{1})^2}{2 \sigma _{1} ^2})$

$P(x | c_{2}) = \frac {1}{\sqrt {2 \pi \sigma _{2}}} exp(- \frac {(x - \mu_{2})^2}{2 \sigma _{2} ^2})$

$P(c_{1} | x_{i}) = \frac {P(x_{i} | c_{1})P(c_{1})}{P(x_{i} | c_{1})P(c_{1}) + P(x_{i} | c_{2})P(c_{2})}$

$P(c_{2} | x_{i}) = 1 - P(c_{1} | x_{i})$

3. Maximization Step: Using the probability to update the Gaussian distribution.

$\mu = \frac {c_{11}x_{1} + c_{12}x_{2} + ... + c_{1n}x_{n}}{c_{11} + c_{12} + ...+ c_{1n}}$

$\sigma ^2 = \frac {c_{11}(x_{1} - \mu_{1})^2 + c_{11}(x_{1} - \mu_{1})^2 +...+c_{11}(x_{1} - \mu_{1})^2}{c_{11} + c_{12} + ... + c_{1n}}$

4. Iterate step2 and step3 until find the maximum likelihood.

Gaussian Distribution (Normal Distribution): If the random variable X obeys a normal distribution with mathematical expectation μ and variance σ2, denoted as N(μ,σ2). Its probability density function determines its position for the expected value μ of a normal distribution, and its standard deviation σ determines the magnitude of the distribution. The normal distribution when μ = 0 and σ = 1 is the standard normal distribution.

Probability Density Function:

【ML】EM Algorithm
EM Algorithm is short for Expectation-Maximization Algori...
EM Algorithm
Expectation Maximization Algorithm EM算法和之前学的都不太一样，EM算法更多的...
EM算法
1. EM介绍 EM(Expectation Maximization Algorithm, EM)是Dempst...
（EM算法）The EM Algorithm
（EM算法）The EM Algorithm EM是我一直想深入学习的算法之一，第一次听说是在NLP课中的HMM...
EM 算法
参考: 从最大似然到EM算法浅解（EM算法）The EM Algorithm EM算法及其推广学习笔记 EM算法...
机器学习(十):EM算法与GMM算法原理及案例分析
一、简介 EM算法最大期望算法(Expectation-maximization algorithm，简称EM，...
无监督学习聚类分析④
无监督学习聚类分析④ EM（Expectation Maximization Algorithm） 1.EM算法...
（十）EM算法
1.EM算法是什么 EM算法的英文全称是Expectation Maximization Algorithm——...
基于R的高级统计（EM算法）
The EM Algorithm EM算法全称叫做Expectation-Maximization，也就是说它分成...
Expectation Maximum Algorithm(EM
1. 预备知识 1.1 凸函数的性质假设定义在实数域上的函数，对于任意的实数，都有则函数称为凸函数，反之，为凹...