美文网首页
输入 NumPy 数据(04)

输入 NumPy 数据(04)

作者: YX_Andrew | 来源:发表于2019-02-09 12:19 被阅读0次

对于小型数据集,请使用内存中的 NumPy 数组训练和评估模型。使用 fit 方法使模型与训练数据“拟合”:

import numpy as np

data = np.random.random((1000, 32))
labels = np.random.random((1000, 10))

model.fit(data, labels, epochs=10, batch_size=32)
Epoch 1/10
1000/1000 [==============================] - 0s 253us/step - loss: 11.5766 - categorical_accuracy: 0.1110
Epoch 2/10
1000/1000 [==============================] - 0s 64us/step - loss: 11.5205 - categorical_accuracy: 0.1070
Epoch 3/10
1000/1000 [==============================] - 0s 70us/step - loss: 11.5146 - categorical_accuracy: 0.1100
Epoch 4/10
1000/1000 [==============================] - 0s 69us/step - loss: 11.5070 - categorical_accuracy: 0.0940
Epoch 5/10
1000/1000 [==============================] - 0s 71us/step - loss: 11.5020 - categorical_accuracy: 0.1150
Epoch 6/10
1000/1000 [==============================] - 0s 72us/step - loss: 11.5019 - categorical_accuracy: 0.1350
Epoch 7/10
1000/1000 [==============================] - 0s 72us/step - loss: 11.5012 - categorical_accuracy: 0.0970
Epoch 8/10
1000/1000 [==============================] - 0s 72us/step - loss: 11.4993 - categorical_accuracy: 0.1180
Epoch 9/10
1000/1000 [==============================] - 0s 69us/step - loss: 11.4905 - categorical_accuracy: 0.1320
Epoch 10/10
1000/1000 [==============================] - 0s 66us/step - loss: 11.4909 - categorical_accuracy: 0.1410

tf.keras.Model.fit 采用三个重要参数:

  • epochs:以周期为单位进行训练。一个周期是对整个输入数据的一次迭代(以较小的批次完成迭代)。
  • batch_size:当传递 NumPy 数据时,模型将数据分成较小的批次,并在训练期间迭代这些批次。此整数指定每个批次的大小。请注意,如果样本总数不能被批次大小整除,则最后一个批次可能更小。
  • validation_data:在对模型进行原型设计时,您需要轻松监控该模型在某些验证数据上达到的效果。传递此参数(输入和标签元组)可以让该模型在每个周期结束时以推理模式显示所传递数据的损失和指标。
下面是使用 validation_data 的示例:
import numpy as np

data = np.random.random((1000, 32))
labels = np.random.random((1000, 10))

val_data = np.random.random((100, 32))
val_labels = np.random.random((100, 10))

model.fit(data, labels, epochs=10, batch_size=32,
          validation_data=(val_data, val_labels))
Train on 1000 samples, validate on 100 samples
Epoch 1/10
1000/1000 [==============================] - 0s 124us/step - loss: 11.5267 - categorical_accuracy: 0.1070 - val_loss: 11.0015 - val_categorical_accuracy: 0.0500
Epoch 2/10
1000/1000 [==============================] - 0s 72us/step - loss: 11.5243 - categorical_accuracy: 0.0840 - val_loss: 10.9809 - val_categorical_accuracy: 0.1200
Epoch 3/10
1000/1000 [==============================] - 0s 73us/step - loss: 11.5213 - categorical_accuracy: 0.1000 - val_loss: 10.9945 - val_categorical_accuracy: 0.0800
Epoch 4/10
1000/1000 [==============================] - 0s 73us/step - loss: 11.5213 - categorical_accuracy: 0.1080 - val_loss: 10.9967 - val_categorical_accuracy: 0.0700
Epoch 5/10
1000/1000 [==============================] - 0s 73us/step - loss: 11.5181 - categorical_accuracy: 0.1150 - val_loss: 11.0184 - val_categorical_accuracy: 0.0500
Epoch 6/10
1000/1000 [==============================] - 0s 72us/step - loss: 11.5177 - categorical_accuracy: 0.1150 - val_loss: 10.9892 - val_categorical_accuracy: 0.0200
Epoch 7/10
1000/1000 [==============================] - 0s 72us/step - loss: 11.5130 - categorical_accuracy: 0.1320 - val_loss: 11.0038 - val_categorical_accuracy: 0.0500
Epoch 8/10
1000/1000 [==============================] - 0s 74us/step - loss: 11.5123 - categorical_accuracy: 0.1130 - val_loss: 11.0065 - val_categorical_accuracy: 0.0100
Epoch 9/10
1000/1000 [==============================] - 0s 72us/step - loss: 11.5076 - categorical_accuracy: 0.1150 - val_loss: 11.0062 - val_categorical_accuracy: 0.0800
Epoch 10/10
1000/1000 [==============================] - 0s 67us/step - loss: 11.5035 - categorical_accuracy: 0.1390 - val_loss: 11.0241 - val_categorical_accuracy: 0.1100

输入 tf.data 数据集

使用 Datasets API 可扩展为大型数据集或多设备训练。将 tf.data.Dataset 实例传递到 fit 方法:

# Instantiates a toy dataset instance:
dataset = tf.data.Dataset.from_tensor_slices((data, labels))
dataset = dataset.batch(32)
dataset = dataset.repeat()

# Don't forget to specify `steps_per_epoch` when calling `fit` on a dataset.
model.fit(dataset, epochs=10, steps_per_epoch=30)
Epoch 1/10
30/30 [==============================] - 0s 6ms/step - loss: 11.4973 - categorical_accuracy: 0.1406
Epoch 2/10
30/30 [==============================] - 0s 2ms/step - loss: 11.5182 - categorical_accuracy: 0.1344
Epoch 3/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4953 - categorical_accuracy: 0.1344
Epoch 4/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4842 - categorical_accuracy: 0.1542
Epoch 5/10
30/30 [==============================] - 0s 2ms/step - loss: 11.5081 - categorical_accuracy: 0.1510
Epoch 6/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4939 - categorical_accuracy: 0.1615
Epoch 7/10
30/30 [==============================] - 0s 2ms/step - loss: 11.5049 - categorical_accuracy: 0.1823
Epoch 8/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4617 - categorical_accuracy: 0.1760
Epoch 9/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4863 - categorical_accuracy: 0.1688
Epoch 10/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4946 - categorical_accuracy: 0.1885

在上方代码中,fit 方法使用了 steps_per_epoch 参数(表示模型在进入下一个周期之前运行的训练步数)。由于 Dataset 会生成批次数据,因此该代码段不需要 batch_size。

数据集也可用于验证:

dataset = tf.data.Dataset.from_tensor_slices((data, labels))
dataset = dataset.batch(32).repeat()

val_dataset = tf.data.Dataset.from_tensor_slices((val_data, val_labels))
val_dataset = val_dataset.batch(32).repeat()

model.fit(dataset, epochs=10, steps_per_epoch=30,
          validation_data=val_dataset,
          validation_steps=3)

Epoch 1/10
30/30 [==============================] - 0s 8ms/step - loss: 11.4649 - categorical_accuracy: 0.1740 - val_loss: 11.0269 - val_categorical_accuracy: 0.0521
Epoch 2/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4794 - categorical_accuracy: 0.1865 - val_loss: 11.4233 - val_categorical_accuracy: 0.0521
Epoch 3/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4604 - categorical_accuracy: 0.1760 - val_loss: 11.4040 - val_categorical_accuracy: 0.0208
Epoch 4/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4475 - categorical_accuracy: 0.1771 - val_loss: 11.3095 - val_categorical_accuracy: 0.2396
Epoch 5/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4727 - categorical_accuracy: 0.1750 - val_loss: 11.0481 - val_categorical_accuracy: 0.0938
Epoch 6/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4569 - categorical_accuracy: 0.1833 - val_loss: 11.3550 - val_categorical_accuracy: 0.1562
Epoch 7/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4653 - categorical_accuracy: 0.1958 - val_loss: 11.4325 - val_categorical_accuracy: 0.0417
Epoch 8/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4246 - categorical_accuracy: 0.1823 - val_loss: 11.3625 - val_categorical_accuracy: 0.0417
Epoch 9/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4542 - categorical_accuracy: 0.1729 - val_loss: 11.0326 - val_categorical_accuracy: 0.0521
Epoch 10/10
30/30 [==============================] - 0s 2ms/step - loss: 11.4600 - categorical_accuracy: 0.1979 - val_loss: 11.3494 - val_categorical_accuracy: 0.1042

评估和预测

tf.keras.Model.evaluatetf.keras.Model.predict 方法可以使用 NumPy 数据和 tf.data.Dataset

要评估所提供数据的推理模式损失和指标,请运行以下代码:

data = np.random.random((1000, 32))
labels = np.random.random((1000, 10))

model.evaluate(data, labels, batch_size=32)

model.evaluate(dataset, steps=30)

1000/1000 [==============================] - 0s 83us/step
30/30 [==============================] - 0s 3ms/step


[11.43181880315145, 0.18333333333333332]

要在所提供数据(采用 NumPy 数组形式)的推理中预测最后一层的输出,请运行以下代码:

result = model.predict(data, batch_size=32)
print(result.shape)

(1000, 10)

相关文章

网友评论

      本文标题:输入 NumPy 数据(04)

      本文链接:https://www.haomeiwen.com/subject/ytwasqtx.html