美文网首页
Speeding-up CNN using CP-Decompo

Speeding-up CNN using CP-Decompo

作者: 信步闲庭v | 来源:发表于2017-10-29 17:37 被阅读136次

Approach

We propose a simple two-step approach for speeding up convolution layers within large convolutional neural networks based on tensor decomposition and discriminative fine-tuning. Given a layer, we use non-linear least squares to compute a low-rank CP-decomposition of the 4D convolution kernel tensor into a sum of a small number of rank-one tensors. At the second step, this decomposition is used to replace the original convolutional layer with a sequence of four convolutional layers with small kernels. After such replacement, the entire network is fine tuned on the training data using standard backpropagation process.

The rank-R CP-decomposition (2) of the 4D kernel tensor has the form:


where Kx, Ky, Ks, Kt are the four components of the composition representing 2D tensors (matrices) of sizes d×R, d×R, S×R, and T×R respectively.

Experiment

References:
SPEEDING-UP CONVOLUTIONAL NEURAL NETWORKS USING FINE-TUNED CP-DECOMPOSITION, Vadim Lebedev, 2015, ICLR

相关文章

网友评论

      本文标题:Speeding-up CNN using CP-Decompo

      本文链接:https://www.haomeiwen.com/subject/eviypxtx.html