Notes for "L1-norm low-rank matr

作者: jjx323 | 来源:发表于2019-02-11 17:33 被阅读0次

Notes for "L1-norm low-rank matr
Robust Subspace Segmentation by
卷积的空间不变性是过度的归纳偏置
语音识别TDNN-F声学模型
611. Knight Shortest Path
A pure L1-norm principal compone
DFS+Cache
Spark Summit2017上MLLIB的最新进展
【leetcode】59. Spiral Matrix II(螺
2021.2.25每日一题

Article Information: Q. Zhao, D. Meng, Z. Xu, W. Zuo, and Y. Yan, IEEE Transactions on Neural Networks and Learning Systems, 26(4), 2015, 825-839.

Details for deriving (22)-(26)

$\begin{align} \ln p(U,V,\tau_{u},\tau_{v},Z,X) = & -\frac{1}{2}\sum_{i,j}z_{ij}^{-1}w_{ij}(x_{ij}-u_{i}^{T}v_{j})^{2} - \frac{1}{2}\tau_{u}\sum_{i=1}^{m}u_{i}^{T}u_{i} \\ & -\frac{1}{2}\tau_{v}\sum_{j=1}^{n}v_{j}^{T}v_{j} + \frac{1}{2}mr\ln\tau_{u} + \frac{1}{2}nr\ln\tau_{v} \\ & - \sum_{ij}w_{ij}\Big( \frac{1}{\lambda}z_{ij} + \frac{1}{2}\ln z_{ij} \Big) + \ln p(\tau_{u}|a_{0}, b_{0}) + \ln p(\tau_{v}|c_{0},d_{0}) + C. \end{align}$
Derivation of (22)-(25):
Picking terms related to $U$ in $\ln p(U, V, \tau_u, \tau_v, Z, X)$ shown in (17)[there may be some small typos in (17)], we have
$\begin{align} \frac{1}{2}\Big\{\sum_{i,j}z_{ij}^{-1}w_{ij}\big[ x_{ij}^2 + u_{i}^{T}v_{j}v_{j}^{T}u_{i} - 2x_{ij}u_{i}^{T}v_{j} \big] + \tau_{u}\sum_{i=1}^{m}u_{i}^{T}u_{i}\Big\}, \end{align} \tag{formula 1}$
which is a quadratic form of $u_{i}$ with $i=1,2,\cdots, m$ . Hence, the posterior distribution of $u_{i}$ is Gaussian. By tedious calculations, we can transform (formula 1) into the following form
$\begin{align} \sum_{i=1}^{m}\Big\{ u_{i}^{T}\big[ \tau_{u} I + \sum_{j=1}^{n}w_{ij}z_{ij}^{-1}v_{j}v_{j}^{T} \big]u_{i} -2 z_{ij}^{-1}w_{ij}x_{ij}u_{i}^{T}v_{j} \Big\} + C. \tag{formula 2} \end{align}$
Formula (22) in the article can now be easily derived from (formula 2). Firstly, we should know that the Gamma distribution has PDF as follow:
$\Gamma(t|a,b) = \frac{b^a}{\text{Gamma}(a)}t^{a-1}\exp(-bt),$
where $\text{Gamma}(\cdot)$ is the usual Gamma function. Similar as above, we extract the terms related to $\tau_u$ as follow
$\begin{align} -\frac{1}{2}\tau_{u}\sum_{i=1}^{m}u_{i}^{T}u_{i} & + \frac{1}{2}mr\ln \tau_{u} + (a_{0}-1)\ln\tau_{u} - b_{0}\tau_{u} \\ & = \Big( a_{0}+\frac{1}{2}mr - 1 \Big)\ln\tau_{u} - \Big( b_{0}+\frac{1}{2}\sum_{i=1}^{m}u_{i}^{T}u_{i} \Big)\tau_{u}. \tag{formula 3} \end{align}$
The above (formula 3) indicates formula (23) holds true. Similar deductions will provide formula (24) and (25). Next, we focus on (26).
Derivation of (26):
Firstly, we should know that the inverse Gaussian distribution (Wald distribution) has PDF as follow:
$IG(x|\mu, \lambda) = \left( \frac{\lambda}{2\pi x^3} \right)^{1/2}\exp\left\{ \frac{-\lambda (x-\mu)^2}{2\mu^{2}x} \right\}$
for $x>0$ , where $\mu > 0$ is the mean and $\lambda>0$ is the shape parameter. (https://en.wikipedia.org/wiki/Inverse_Gaussian_distribution)
The logrithm is
$\ln IG(x|\mu,\lambda) = \frac{1}{2}\ln\lambda-\frac{1}{2}\ln 2\pi - \frac{3}{2}\ln x - \frac{\lambda}{2\mu^2}x - \frac{\lambda}{2x} + \frac{\lambda}{\mu}. \tag{formula 4}$
For $(i,j)\in \Omega$ , we can extract terms related to $y_{ij}=\frac{1}{z_{ij}}$ as follow:
$\begin{align} -\frac{1}{2}y_{ij}\mathbb{E}[(x_{ij}-u_{i}^{T}v_{j})^{2}] - \frac{1}{\lambda y_{ij}} - \frac{1}{2}\ln y_{ij}. \tag{formula 5} \end{align}$
Comparing (formula 4) and (formula 5), we arrive at (26). Here, we should notice $dz_{ij} = \frac{-1}{y_{ij}^{2}}dy_{ij}$ which makes $-\frac{3}{2}\ln x$ and $-\frac{1}{2}\ln y_{ij}$ coincide with each other.

Notes for "L1-norm low-rank matr
Article Information: Q. Zhao, D. Meng, Z. Xu, W. Zuo, and...
Robust Subspace Segmentation by
Robust Subspace Segmentation by Low-Rank Representation 使...
卷积的空间不变性是过度的归纳偏置
Revisiting Spatial Invariance with Low-Rank Local Connect...
语音识别TDNN-F声学模型
参考论文：Semi-Orthogonal Low-Rank Matrix Factorization for De...
611. Knight Shortest Path
Description Given a knight in a chessboard (a binary matr...
A pure L1-norm principal compone
A pure L1-norm principal component analysis 虽然没有完全弄清楚其中的数...
DFS+Cache
Longest Increasing Path in a Matrix Given an integer matr...
Spark Summit2017上MLLIB的最新进展
ADMM-BASED SCALABLE MACHINE LEARNING ON APACHE SPARK MATR...
【leetcode】59. Spiral Matrix II(螺
1、题目描述 Given a positive integer n, generate a square matr...
2021.2.25每日一题
867. 转置矩阵[https://leetcode-cn.com/problems/transpose-matr...