Keras 练习1 - MLP

《Tensorflow + Keras深度学习人工智能实践应用》 一书第7章的完整例子，一行行代码敲出来，更有利于理解整个例子的工作流程和结果。

机器学习的几个分类：

• 多层感知器（Multi-layer Perceptron，MLP）
• 深度神经网络（Deep Neural Network，DNN）
• 卷积神经网络（Convolutional Neural Network，CNN）
• 递归神经网络（Recurrent Neural Network，RNN）

import numpy as np
import pandas as pd
from keras.utils import np_utils
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout

import matplotlib.pyplot as plt
import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
np.random.seed(10)

(x_train_image, y_train_label), (x_test_image, y_test_label) = mnist.load_data()

x_train = x_train_image.reshape(60000, 784).astype("float32")
x_test = x_test_image.reshape(10000, 784).astype("float32")

x_train_normal = x_train / 255
x_test_normal = x_test / 255

y_train_onehot = np_utils.to_categorical(y_train_label)
y_test_onehot = np_utils.to_categorical(y_test_label)

model = Sequential()
model.add(Dense(units = 1000,
                input_dim = 784,
                kernel_initializer = 'normal',
                activation = 'relu'))
model.add(Dropout(0.5))

model.add(Dense(units = 1000,
                kernel_initializer = 'normal',
                activation = 'relu'))
model.add(Dropout(0.5))

model.add(Dense(units = 10,
                kernel_initializer = 'normal',
                activation = 'softmax'))

print(model.summary())


model.compile(loss = "categorical_crossentropy",
            optimizer = "adam", metrics = ["accuracy"])

history = model.fit(x = x_train_normal,
                y = y_train_onehot,
                validation_split = 0.2,
                epochs = 10,
                batch_size = 200,
                verbose = 2)

def show_train_history(train_history, train, val):
    plt.plot(train_history.history[train])
    plt.plot(train_history.history[val])
    plt.title("Train History")
    plt.ylabel(train)
    plt.xlabel("Epochs")
    plt.legend(["train", "validation"], loc="upper left")
    plt.show()

def plot_image_label_prediction(images, labels, prediction, idx = 0, num = 10):
    fig = plt.gcf()
    fig.set_size_inches(12, 14)
    if num > 25:
        num = 25
    for i in range(0, num):
        ax = plt.subplot(5, 5, 1 + i)
        ax.imshow(images[idx], cmap="binary")
        title = "label = " + str(labels[idx])
        if len(prediction) > 0:
            title += ", prediction = " + str(prediction[idx])
        ax.set_title(title, fontsize = 12)
        ax.set_xticks([])
        ax.set_yticks([])
        idx += 1
    plt.show()


show_train_history(history, "acc", "val_acc")
show_train_history(history, "loss", "val_loss")

scores = model.evaluate(x_test_normal, y_test_onehot)
print("accuracy = ", scores[1])

prediction = model.predict_classes(x_test_normal)
#plot_image_label_prediction(x_test_image, y_test_label, prediction, idx=340, num=25)

print(pd.crosstab(y_test_label, prediction, rownames = ["label"], colnames = ["predict"]))

df = pd.DataFrame({"label": y_test_label, "predict": prediction})
print(df[(df.label == 5) & (df.predict == 3)])

训练及精度：

Train on 48000 samples, validate on 12000 samples
Epoch 1/10
 - 10s - loss: 0.3634 - acc: 0.8870 - val_loss: 0.1342 - val_acc: 0.9608
Epoch 2/10
 - 10s - loss: 0.1585 - acc: 0.9520 - val_loss: 0.1003 - val_acc: 0.9702
Epoch 3/10
 - 10s - loss: 0.1182 - acc: 0.9626 - val_loss: 0.0889 - val_acc: 0.9725
Epoch 4/10
 - 10s - loss: 0.0965 - acc: 0.9704 - val_loss: 0.0857 - val_acc: 0.9745
Epoch 5/10
 - 10s - loss: 0.0838 - acc: 0.9733 - val_loss: 0.0798 - val_acc: 0.9781
Epoch 6/10
 - 10s - loss: 0.0762 - acc: 0.9756 - val_loss: 0.0803 - val_acc: 0.9770
Epoch 7/10
 - 10s - loss: 0.0640 - acc: 0.9800 - val_loss: 0.0756 - val_acc: 0.9770
Epoch 8/10
 - 10s - loss: 0.0625 - acc: 0.9798 - val_loss: 0.0787 - val_acc: 0.9765
Epoch 9/10
 - 10s - loss: 0.0550 - acc: 0.9816 - val_loss: 0.0751 - val_acc: 0.9802
Epoch 10/10
 - 10s - loss: 0.0520 - acc: 0.9836 - val_loss: 0.0780 - val_acc: 0.9772

10000/10000 [==============================] - 1s 100us/step
accuracy =  0.9797

如何判断过拟合呢？

如果loss和acc的在训练比validation的好过多，则说明过拟合。在例子中，通过增大网络中的单元数、增加中间隐藏层，增加dropout层这几种方法来减少过拟合。