Notes for "A variational Bayesia

作者: jjx323 | 来源:发表于2019-01-14 12:58 被阅读0次

Notes for "A variational Bayesia
Notes for "A variational Bayesia
Notes for "Variational Bayesian
对抗网络2019-2020速览
net-compress_ICML(18')
CVPR 2019 | VID_最大化互信息知识蒸馏
变分推断 Variational Inference
VaDE代码问题记录
DEEP VARIATIONAL INFORMATION BOT
变分贝叶斯初探

Proof of Theorem 3.3

Theorem 3.3. Suppose that $D_{KL}$ has a multivariate Taylor expansion with a positive definite Hessian at $\Theta_{0}$ . Then we have $\Theta_{k} \rightarrow \Theta_{0}$ .

Proof: Given $\Theta_{0}$ there exists $\epsilon > 0$ such that

$D_{KL}(\Theta)=D_{KL}(\Theta_{0})+\frac{1}{2}(\Theta - \Theta_0)^{T}H(\Theta_0)(\Theta-\Theta_0) + o(\|\Theta - \Theta_0\|^2)$

with $H$ being the Hessian with respect to $\Theta$ and $\|\Theta - \Theta_0\|^2 < \epsilon$ . Considering the positive definiteness of the Hessian at the stationary point $\Theta_0$ and the smoothness of $D_{KL}$ , we know that if $\|\Theta - \Theta_0\|^2<\epsilon$ , then $D_{KL}(\Theta) - D_{KL}(\Theta_0) < \epsilon_1$ implies that $\|\Theta - \Theta_0\|^2<c\epsilon_1$ for some $c>0$ .

Since $\{\Theta_k\}_{k=1}^{\infty}$ is a bounded sequence, there is a subsequence $\{\Theta_{n_k}\}_{k=1}^{\infty}$ such that $\Theta_{n_k}\rightarrow \Theta_0$ as $k\rightarrow \infty$ . For every $\epsilon>0$ , choosing $\epsilon_1>0$ such that $c\epsilon_1 <\epsilon$ , we can take $K>0$ large enough such that $D_{KL}(\Theta_{n_K}) - D_{KL}(\Theta_0) \leq \epsilon_1$ . Recalling the above paragraph, we have $\|\Theta_{n_K} - \Theta_0\|^2<c\epsilon_1 < \epsilon$ . By taking $\ell > n_{K}$ , we have $D_{KL}(\Theta_{\ell}) - D_{KL}(\Theta_0) \leq \epsilon_1$ which implies $\|\Theta_{\ell} - \Theta_0\|^2 < \epsilon$ . At this stage, the conclusion obviously holds true.

Notes for "A variational Bayesia
Proof of Theorem 3.3 Theorem 3.3.Suppose thathas a multiv...
Notes for "A variational Bayesia
Info: B. Jin, Journal of Computational Physics, 2012Remar...
Notes for "Variational Bayesian
Info: Michael A. Chappel et al. IEEE Transactions on Sign...
对抗网络2019-2020速览
Variational Discriminator Bottleneck: Improving Imitation...
net-compress_ICML(18')
Pruning Compressing Neural Networks using the Variational...
CVPR 2019 | VID_最大化互信息知识蒸馏
CVPR 2019 | Variational Information Distillation for Know...
变分推断 Variational Inference
原文：Variational Inference with Normalizing Flowshttps://bl...
VaDE代码问题记录
论文名《Variational Deep Embedding:An Unsupervised and Genera...
DEEP VARIATIONAL INFORMATION BOT
DEEP VARIATIONAL INFORMATION BOTTLENECK 论文“Deep Variation...
变分贝叶斯初探
原题：A Beginner's Guide to Variational Methods: Mean-Field ...