Notes for "Variational Bayesian

作者: jjx323 | 来源:发表于2019-01-29 10:43 被阅读0次

Info: Michael A. Chappel et al. IEEE Transactions on Signal Processing, 2009
Remark: All the reference numbers are indicated the corresponding numbers in this article.

Formula (23)

Proof: For all the $q(\theta), q(\phi)$ appeared in the following, we all mean the probability distribution with parameters shown in (19)-(22).
$\begin{align} F & = \int\int q(\theta)q(\phi)\log(P(y|\{\theta, \phi\})P(\theta)P(\phi))d\theta d\phi \\ & \qquad \qquad \qquad - \int \int q(\theta)q(\phi)\left[ \log(q(\theta)) + \log(q(\phi)) \right] d\theta d\phi \\ & = I + II \tag{1} \label{for1} \end{align}$
Now, we consider $I$ and $II$ separately. For $I$ , we have
$\begin{align} I = & \int\int q(\theta)q(\phi)\Big\{ -\frac{1}{2}\phi (y-g(\theta))^T (y-g(\theta)) + (\frac{N}{2}+c_0 -1)\log(\phi) \\ & \qquad\qquad -\frac{1}{2}(\theta - m_0)^T \Lambda_0 (\theta -m_0) - \frac{1}{s_0}\phi + \text{Const} \Big\} d\theta d\phi. \end{align}$
Since $\int_0^{\infty} q(\phi)\log\phi d\phi = \log(s) + \psi(c)$ and the mean value of $q(\phi)$ is $sc$ , we find that
$\begin{align} I = & \left( \frac{N}{2} + c_0 - 1 \right)\left( \log(s) + \psi(c) \right) - \frac{sc}{s_0} -\frac{sc}{2}\int (y-g(\theta))^T (y-g(\theta))q(\theta)d\theta \\ & \qquad\qquad -\frac{1}{2}\int (\theta-m_0)^T \Lambda_0 (\theta - m_0)q(\theta)d\theta + \text{Const}. \tag{2} \label{for2} \end{align}$
For the last term of the first line in the above equality, we have
$\begin{align} & -\frac{sc}{2}\int (k-J(\theta -m))^T (k-J(\theta-m)) q(\theta)d\theta \\ & \qquad\qquad = -\frac{sc}{2}\int (k^T k + (\theta-m)^T J^T J (\theta-m))q(\theta) d\theta \\ & \qquad\qquad = -\frac{sc}{2}\big( k^T k + Tr(\Lambda^{-1}J^T J) \big). \end{align}$
For the first term of the second line in equality (1), we have
$\begin{align} & \quad -\frac{1}{2}\int (\theta-m_0)^T \Lambda_0 (\theta-m_0) q(\theta)d\theta \\ & = -\frac{1}{2} \int (\theta-m)^T \Lambda_0 (\theta-m) q(\theta)d\theta -\frac{1}{2}\int (m-m_0)^T \Lambda_0 (m-m_0) q(\theta) d\theta \\ & = -\frac{1}{2} Tr(\Lambda^{-1}\Lambda_0) - \frac{1}{2}(m-m_0)^T \Lambda_0 (m-m_0). \end{align}$
Now, we focus on term $II$ . Specifically, we find that
$\begin{align} II = & \int q(\theta) \log q(\theta) d\theta + \int q(\phi) \log q(\phi) d\phi \\ = & \int q(\theta) \big[ -\frac{1}{2}\log\det(\Lambda) - \frac{1}{2}(\theta-m)^T \Lambda (\theta-m) \big] d\theta \\ & \qquad + \int q(\phi) \big[ -\log \Gamma(c) + (c-1)\log\phi - c\log s - \frac{1}{s}\phi \big]d\phi \\ = & -\frac{1}{2}\log\det(\Lambda) - c + (c-1)\big( \log s + \psi(c) \big) - \log\Gamma(c) - c\log s. \end{align}$
Inserting estimations of $I$ and $II$ into formula (1), we finally arrive at
$\begin{align} F = & -\frac{sc}{s_0} + \bigg( \frac{N}{2}+c_0-1 \bigg)\big[ \log s + \psi(c) \big] \\ & - \frac{1}{2}\bigg\{ (m-m_0)^T \Lambda_0 (m-m_0) + Tr(\Lambda^{-1}\Lambda_0) \bigg\} \\ & -\frac{sc}{2}\bigg\{ k^T k + Tr(\Lambda^{-1}J^T J) \bigg\} -c\log s - \log\Gamma(c) \\ & -c + (c-1)\big[ \log s + \psi(c) \big] + \frac{1}{2}\log\det(\Lambda) + Const. \end{align}$
$\color{red}{\text{ This result is a little bit different from the result shown in the paper [formula (23)]. }}$

Notes for "Variational Bayesian

Formula (23)

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读