2020 机器是如何认识世界的(3)

computer_vision.png

今天我们介绍另一种可视化技术帮助你理解卷积神经网是如何做到读取图片来进行图像识别分类。准备用一张热力图将机器通过对图片那部分区域感兴趣来实现对图片识别的。

我们为 vgg16 输入一张图片，vgg16 输出一个 特征图，然后根据这张特征图就可以获取足够信息对图片进行分类。

我们通过变动特征图，然后最终 VGG16 对图片识别结果影响有多大。然后在对同一个位置 215 通道的值进行平均就可以得到哪些位置对机器作出决定影响的程度。

最后我们用热力图来表示我们模型关注哪些部分得到最后结果。

cat_heatmap.jpg

一起看看代码。

from tensorflow.keras.applications.vgg16 import preprocess_input
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.models import load_model

from tensorflow.keras import preprocessing
from tensorflow.keras import backend as K
from tensorflow.keras import models

import tensorflow as tf
import numpy as np

import matplotlib.pyplot as plt
import cv2
image_size = 224

# Load pre-trained Keras model and the image to classify
model = tf.keras.applications.vgg16.VGG16()
# image = np.random.random((image_size, image_size, 3))
img_path = 'cat.jpeg'
image = load_img(img_path,target_size=(224,224))

img_tensor = preprocessing.image.img_to_array(image)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor = preprocess_input(img_tensor)

conv_layer = model.get_layer("block5_conv3")
heatmap_model = models.Model([model.inputs], [conv_layer.output, model.output])

# Get gradient of the winner class w.r.t. the output of the (last) conv. layer
with tf.GradientTape() as gtape:
    conv_output, predictions = heatmap_model(img_tensor)
    loss = predictions[:, np.argmax(predictions[0])]
    grads = gtape.gradient(loss, conv_output)
    pooled_grads = K.mean(grads, axis=(0, 1, 2))

heatmap = tf.reduce_mean(tf.multiply(pooled_grads, conv_output), axis=-1)
heatmap = np.maximum(heatmap, 0)
max_heat = np.max(heatmap)
if max_heat == 0:
    max_heat = 1e-10
heatmap /= max_heat

# heatmap = np.mean(conv_layer_output_value,axis=-1)

# heatmap = np.maximum(heatmap,0)
# heatmap/=np.max(heatmap)

heatmap = heatmap.reshape(14,14)
# print(heatmap)
plt.matshow(heatmap)
plt.savefig("cat_heatmap.jpg")

# print(heatmap.shape)

img = cv2.imread(img_path)
heatmap = cv2.resize(heatmap,(img.shape[1],img.shape[0]))
heatmap = np.uint8(255 * heatmap)
heatmap = cv2.applyColorMap(heatmap,cv2.COLORMAP_JET)
superimposed_img = heatmap * 0.4 + img

cv2.imwrite('vis_cat_res_01.jpg',superimposed_img)

cat.jpeg vis_cat_res_01.jpg