多元分类|第三天的Keras

今天学习了一个多元分类的机器学习方法，在这里分享给大家。
说是多元分类其实就是把四个象限的坐标分成四类。

废话不多说，先贴代码。

from keras.layers import Dense, Activation
from keras.models import Sequential
from keras import optimizers
import numpy as np

# collect the data
x_train1 = 100*np.random.random((100, 2))
x_train2 = [-100, 100]*np.random.random((100, 2))
x_train3 = -100*np.random.random((100, 2))
x_train4 = [100, -100]*np.random.random((100, 2))
x_train = np.concatenate((x_train1, x_train2, x_train3, x_train4))
y_train = np.array([[1, 0, 0, 0]*100 + [0, 1, 0, 0]*100 + [0, 0, 1, 0]*100 + [0, 0, 0, 1]*100])
y_train = y_train.reshape((400, 4))

x_test1 = 100*np.random.random((100, 2))
x_test2 = [-100, 100]*np.random.random((100, 2))
x_test3 = -100*np.random.random((100, 2))
x_test4 = [100, -100]*np.random.random((100, 2))
x_test = np.concatenate((x_test1, x_test2, x_test3, x_test4))
y_test = y_train

# set the model
model = Sequential()
model.add(Dense(4, input_dim=2, activation=None, use_bias=False))
model.add(Activation('softmax'))

# compile the model and pick the loss function and optimizer
ada = optimizers.Adagrad(lr=0.1, epsilon=1e-8)
model.compile(optimizer=ada, loss='categorical_crossentropy', metrics=['accuracy'])

# training the model
ly = []
for i in range(10):
    model.fit(x_train, y_train, batch_size=400, epochs=100, shuffle=False)
    ly.append(model.layers[0].get_weights())

# test the model

score = model.evaluate(x_test, y_test, batch_size=400)
print 'loss:', score[0], '\t\taccuracy:', score[1]
for i in range(10):
    print 'first weight:\t', ly[i][0][0], '\t\tsecond weight:\t', ly[i][0][1]

下面咱么开始分析

导入相关模块

from keras.layers import Dense, Activation
from keras.models import Sequential
from keras import optimizers
import numpy as np

因为之前介绍过这里就不多说了详情请了解线性回归|第一天的Keras、分类训练|第二天的Keras

收集训练数据

# collect the data
x_train1 = 100*np.random.random((100, 2))
x_train2 = [-100, 100]*np.random.random((100, 2))
x_train3 = -100*np.random.random((100, 2))
x_train4 = [100, -100]*np.random.random((100, 2))
x_train = np.concatenate((x_train1, x_train2, x_train3, x_train4))
y_train = np.array([[1, 0, 0, 0]*100 + [0, 1, 0, 0]*100 + [0, 0, 1, 0]*100 + [0, 0, 0, 1]*100])
y_train = y_train.reshape((400, 4))

x_test1 = 100*np.random.random((100, 2))
x_test2 = [-100, 100]*np.random.random((100, 2))
x_test3 = -100*np.random.random((100, 2))
x_test4 = [100, -100]*np.random.random((100, 2))
x_test = np.concatenate((x_test1, x_test2, x_test3, x_test4))
y_test = y_train

• 在这里用np.random.random((100,2))生成一百个大小在0～1之间的二维向量。为了增加其一般性，并扩大规模，我们乘上100后，就变成了在第一象限的100个坐标，范围为0～100。同理后面也是作相似处理。然后四个象限的坐标数据就收集好了，我们通过np.concatenate方法把这四个象限的数据合并到一起，x_train就完成了。
• y_train的处理比较简单，最后要把y_train进行重塑。变成4维的向量就是【0 ，1，0，0】1代表什么意思呢，就是第二类了。
• 好了，后面x_test和y_test就如同上面一样了。

设计训练模型

# set the model
model = Sequential()
model.add(Dense(4, input_dim=2, activation=None, use_bias=False))
model.add(Activation('softmax'))

在这里有一个新东西就是softmax下面解释一下：
 softmax 是对上一层的所有输入例如y1，y2......yn.进行如下操作。
 yi <- yi/(y1+y2+y3....+yn)
了解了吧。

编译模型

# compile the model and pick the loss function and optimizer
ada = optimizers.Adagrad(lr=0.1, epsilon=1e-8)
model.compile(optimizer=ada, loss='categorical_crossentropy', metrics=['accuracy'])

• 加速器选用AdaGrad，learing_rate=0.1,epsilon(小量)=1e-8(0.00000001)
• 值得一提的是loss = 'categorical_crossentropy'在进行多分类时一定要用到这一个loss_function.并且在使用此损失函数时，我们的y_train一定要写成【0，1，0，0】类似的向量。即向量中只能有0、1.维度为要分成的类数。

训练模型

ly = []
for i in range(10):
    model.fit(x_train, y_train, batch_size=400, epochs=100, shuffle=False)
    ly.append(model.layers[0].get_weights())

• 这里我们分成10此训练每次训练100轮然后每次训练我们可以获得它的weight，追踪weight的变化。
• ly = [] 是用来存储每此训练后的weight数组。
• 因为只有一层，所以layers[0]就行了。

测试模型

score = model.evaluate(x_test, y_test, batch_size=400)
print 'loss:', score[0], '\t\taccuracy:', score[1]
for i in range(10):
    print 'first weight:\t', ly[i][0][0], '\t\tsecond weight:\t', ly[i][0][1]

• 我们这里用evalute方法评估一下该训练好的模型。该模型返回loss，和accuracy。
• 然后我们再把weight打印出来。

得到数据

/home/kroossun/miniconda2/bin/python /home/kroossun/PycharmProjects/ML/multi_classfication.py
Using TensorFlow backend.
2017-09-02 19:01:10.912652: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 19:01:10.912672: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 19:01:10.912677: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 19:01:10.912681: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 19:01:10.912684: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Epoch 1/100
400/400 [==============================] - 0s - loss: 14.7780 - acc: 0.0500
Epoch 2/100
400/400 [==============================] - 0s - loss: 13.3719 - acc: 0.1025
.........
Epoch 99/100
400/400 [==============================] - 0s - loss: 0.0181 - acc: 0.9975
Epoch 100/100
400/400 [==============================] - 0s - loss: 0.0181 - acc: 0.9975
400/400 [==============================] - 0s
loss: 0.0185921788216       accuracy: 0.997500002384
first weight:   [ 0.15896426 -0.41137934 -0.40870011  0.18085584]       second weight:  [ 0.10717846  0.10917206 -0.71580416 -0.70930481]
first weight:   [ 0.17234965 -0.42518327 -0.42248955  0.19431336]       second weight:  [ 0.11188302  0.11416188 -0.72089857 -0.71434933]
first weight:   [ 0.18458323 -0.43780571 -0.43509614  0.20663606]       second weight:  [ 0.11647248  0.11900952 -0.72586221 -0.71925527]
first weight:   [ 0.19587189 -0.44945827 -0.44673175  0.21802464]       second weight:  [ 0.12095292  0.12372655 -0.730703   -0.72403347]
first weight:   [ 0.20636941 -0.46029857 -0.45755425  0.22862951]       second weight:  [ 0.12533006  0.12832268 -0.73542821 -0.72869295]
first weight:   [ 0.21619371 -0.47044694 -0.46768427  0.23856631]       second weight:  [ 0.12960933  0.13280627 -0.74004406 -0.73324138]
first weight:   [ 0.22543719 -0.47999874 -0.4772172   0.24792583]       second weight:  [ 0.13379583  0.13718463 -0.74455637 -0.73768568]
first weight:   [ 0.23417395 -0.48902947 -0.4862287   0.25678098]       second weight:  [ 0.13789426  0.14146428 -0.7489711  -0.74203229]
first weight:   [ 0.24246417 -0.49760085 -0.49478057  0.2651912 ]       second weight:  [ 0.14190902  0.14565097 -0.75329334 -0.74628681]
first weight:   [ 0.25035757 -0.50576413 -0.50292408  0.27320552]       second weight:  [ 0.14584419  0.1497499  -0.75752771 -0.75045419]

Process finished with exit code 0

• 这里我们训练了1000轮其实100轮已经够了，我们这里训练的有点多。
• 可以看到，在训练集上accuracy为0.9975，测试集上，也是0.9975。说明我们的训练效果还是不错的。