LSTM
1.peephole connection
:一种流行的LSTM变种,由Gers&Schmidhuber(2000)提出,加入“窥视孔连接”(peephole connection
),也就是让我们各种门可以观察到元细胞状态。
![](https://img.haomeiwen.com/i1686691/62e5c2205c40cd83.png)
Math
1.element-wise product/Hadamard product元素积:![](https://img.haomeiwen.com/i1686691/d24f70cd47487c93.png)
theano
1.theano.dimshuffle
:改变输入维度的顺序,返回原始变量的一个view。输入是一个包含[0, 1, ..., ndim - 1]
和任意数目的'x'
的组合:
则:
-
.dimshuffle('x')
:将标量变成1维数组 -
.dimshuffle(0, 1)
:与原始的2维数组相同 -
.dimshuffle(1, 0)
:交换2维数组的两个维度,形状从N * M
变成M * N
-
.dimshuffle('x', 0)
:形状从N
变成1 * N
-
.dimshuffle(0, 'x')
:形状从N
变成N * 1
-
.dimshuffle(2, 0, 1)
:形状从A * B * C
变成C * A * B
-
.dimshuffle(0, 'x', 1)
:形状从A * B
变成A * 1 * B
-
.dimshuffle(1, 'x', 0)
:形状从A * B
变成B * 1 * A
-
.dimshuffle(1,)
:将第0维去掉,除去的维度大小必须为1。形状从1 * A
变成A
2.theano.tnesor.concatenate
:拼接
import theano
import numpy as np
import theano.tensor as T
ones = theano.shared(np.float32([[1, 2, 3], [4, 5, 6],[7, 8, 9]]))
print(ones.get_value())
--->>[[1, 2, 3]
[4, 5, 6]
[7, 8, 9]]
result = T.concatenate([ones,ones], axis=0)
print(result.eval())
--->>
[[1, 2, 3]
[4, 5, 6]
[7, 8, 9]
[1, 2, 3]
[4, 5, 6]
[7, 8, 9]]
result = T.concatenate([ones, ones], axis=1)
print(result.eval())
--->>
[[ 1. 2. 3. 1. 2. 3.]
[ 4. 5. 6. 4. 5. 6.]
[ 7. 8. 9. 7. 8. 9.]]
当操作数为二维数组时,axis=0为第一维的方向,axis=1为第二维的方向。
3.theano.tensor.dot(a, b, axes)
:矩阵乘法
import theano
import numpy as np
import theano.tensor as T
ones = theano.shared(np.float32([[1, 2, 3],[4, 5, 6], [7, 8, 9]]))
print(ones.get_value())
--->>
[[ 1. 2. 3.]
[ 4. 5. 6.]
[ 7. 8. 9.]]
result = T.dot(ones, ones)
print(result.eval())
--->>
[[ 30. 36. 42.]
[ 66. 81. 96.]
[ 102. 126. 150.]]
网友评论