Tags: Tool DeepLearning
Screen Shot 2018-09-17 at 2.31.54 PM.png
深度学习和机器学习的移动端化是未来的趋势，这两年各个大厂也在这方面发力，竞相推出自己移动端的推理框架。Google有Tensorflow Lite, Apple有CoreML，Facebook有Caffe2, Tencent的NCNN最近也是风头正盛，百度有Paddle Mobile， 小米有MACE... 总之各个平台之间不仅性能有差异，光是要把训练好的模型正确的转换过去，就得吐一口血。
扯远了，这篇文章主要记录自己在使用Tensorflow+Keras训练模型，同时将模型转换到移动端的一些经验，会持续更新, 现有的移动端框架主要是Tensorflow Lite和CoreML，未来考虑加入Caffe2和NCNN。
Tensorflow 转 Tensorflow Lite
在Android端Tensorflow Lite 会使用 Android Neural Networks API进行加速，但是需要Android 8.1以上才支持：
Screen Shot 2018-09-17 at 2.39.49 PM.png
Tensorflow提供官方转换工具toco可以直接将tensorflow .pb模型转换为.tflite模型。
Screen Shot 2018-09-17 at 3.33.43 PM.png
toco
usage: toco [-h] --output_file OUTPUT_FILE
(--graph_def_file GRAPH_DEF_FILE | --saved_model_dir SAVED_MODEL_DIR | --keras_model_file KERAS_MODEL_FILE)
[--output_format {TFLITE,GRAPHVIZ_DOT}]
[--inference_type {FLOAT,QUANTIZED_UINT8}]
[--inference_input_type {FLOAT,QUANTIZED_UINT8}]
[--input_arrays INPUT_ARRAYS] [--input_shapes INPUT_SHAPES]
[--output_arrays OUTPUT_ARRAYS]
[--saved_model_tag_set SAVED_MODEL_TAG_SET]
[--saved_model_signature_key SAVED_MODEL_SIGNATURE_KEY]
[--std_dev_values STD_DEV_VALUES] [--mean_values MEAN_VALUES]
[--default_ranges_min DEFAULT_RANGES_MIN]
[--default_ranges_max DEFAULT_RANGES_MAX]
[--quantize_weights QUANTIZE_WEIGHTS] [--drop_control_dependency]
[--reorder_across_fake_quant] [--change_concat_input_ranges]
[--allow_custom_ops] [--dump_graphviz_dir DUMP_GRAPHVIZ_DIR]
[--dump_graphviz_video]
- output_file: 输出模型.tflite后缀
- graph_def_file: 静态图文件, saved_model_dir: 模型保存路径, keras_model_file: keras模型文件, 三者必须有且只能有一个
- output_format: 输出文件类型
- inference_type: 这个参数可以用来对模型进行量化，从而达到不同的accuracy/speed的平衡
- inference_input_type: 输入类型，在使用QUANTIZED_UINT8的时候也需要配置 --std_dev_values --mean_values, --default_ranges_min --default_ranges_max这些参数
我自己试验了下使用inference_type和inference_input_type使用QUANTIZED_UINT8比float要快很多（有时间给一个例子） - input_shape: 输出图片的大小
- input_array: 图输入节点
- output_array: 图输出节点
- allow_custom_ops: 定义了官方不支持的layer或者op的时候需要开启
其他参数可以在用到的时候在官网参考。
从toco的使用提示可以看到，我们需要提供的参数有，静态图.pb文件，生成.tflite文件，输出的格式，输入的节点shape, 输入的节点名称，输出节点名称。
成功实例参考：
toco --graph_def_file=DeeplabV3++_portrait_384_1_05alpha.pb --output_file=DeeplabV3++_portrait_384_1_05alpha.tflite --output_format=TFLITE --input_shape=1,384,384,3 --input_array=input_1 --output_array=output_0 --inference_type=float --allow_custom_ops
下面介绍下如何查看tensorflow模型中节点名称
- 如何查看checkpoints中节点名称
saver = tf.train.import_meta_graph(/path/to/meta/graph)
sess = tf.Session()
saver.restore(sess, /path/to/checkpoints)
graph = sess.graph
print([node.name for node in graph.as_graph_def().node])
- 如何查看静态图.pb节点信息:
以我自己的simplenet 静态图文件为例:
"""FIND GRAPH INFO"""
tf_model_path = "./simplenet_V2_8M.pb"
with open(tf_model_path , 'rb') as f:
serialized = f.read()
tf.reset_default_graph()
original_gdef = tf.GraphDef()
original_gdef.ParseFromString(serialized)
with tf.Graph().as_default() as g:
tf.import_graph_def(original_gdef, name ='')
ops = g.get_operations()
N = len(ops)
for i in [0,1,2,N-3,N-2,N-1]: # for循环设置输出的节点信息
print('\n\nop id {} : op type: "{}"'.format(str(i), ops[i].type))
print('input(s):')
for x in ops[i].inputs:
print("name = {}, shape: {}, ".format(x.name, x.get_shape()))
print('\noutput(s):'),
for x in ops[i].outputs:
print("name = {}, shape: {},".format(x.name, x.get_shape()))
输出如下:
op id 0 : op type: "Placeholder"
input(s):
output(s):
name = input_1:0, shape: (?, 32, 32, 3),
op id 1 : op type: "Const"
input(s):
output(s):
name = block1_conv/kernel:0, shape: (3, 3, 3, 128),
op id 2 : op type: "Identity"
input(s):
name = block1_conv/kernel:0, shape: (3, 3, 3, 128),
output(s):
name = block1_conv/kernel/read:0, shape: (3, 3, 3, 128),
op id 190 : op type: "MatMul"
input(s):
name = global_average_pooling2d_1/Mean:0, shape: (?, 600),
name = dense_1/kernel/read:0, shape: (600, 10),
output(s):
name = dense_1/MatMul:0, shape: (?, 10),
op id 191 : op type: "BiasAdd"
input(s):
name = dense_1/MatMul:0, shape: (?, 10),
name = dense_1/bias/read:0, shape: (10,),
output(s):
name = dense_1/BiasAdd:0, shape: (?, 10),
op id 192 : op type: "Softmax"
input(s):
name = dense_1/BiasAdd:0, shape: (?, 10),
output(s):
name = activation_1/Softmax:0, shape: (?, 10),
得到这些信息之后我就可以给toco提供合适的信息进行转换:
toco --graph_def_file=simplenet_V2_8M.pb --output_file=simplenet_v2_8M.tflite --output_format=TFLITE --input_shape=1,32,32,3 --input_arrays=input_1 --output_arrays=activation_1/Softmax
转换成功之后会生成.tflite文件，就可以用于移动端部署了。
tflite
实际上在直接用于移动端部署之前，你可能需要测试下你的tflite模型的准确度，这就需要直接在Python里面调用tflite:
import numpy as np
import tensorflow as tf
# Load TFLite model and allocate tensors.
interpreter = tf.contrib.lite.Interpreter(model_path="converted_model.tflite")
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Test model on random input data.
input_shape = input_details[0]['shape']
# change the following line to feed into your own data.
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)
Tensorlfow 转 CoreML
TF-COREML
CoreML并不提供将tensorflow模型直接转换为mlmodel的工具，但是有keras接口。此外有第三方转换工具tf-coreml可以用来转换，这里看下如何使用这个工具转换。
首先将我们保存的checkpoints保存为静态图,.pb格式。
然后要输入的参数有：
- tf_model_path: .pb静态图模型路径
- mlmodel_path: 生成的CoreML模型路径地址
- input_name_shape_dict: 网络的输入名称和数据的大小(要根据原始的模型输入确定)
- output_feature_names: 网络输出的名称(要根据原始的模型输出确定)
此外也可以对输入的数据做一些归一化处理:
- image_scale:
- red_bias
- green_bias
- blue_bias
import tensorflow as tf
import tfcoreml
from coremltools.proto import FeatureTypes_pb2 as _FeatureTypes_pb2
import coremltools
""" FIND GRAPH INFO """
tf_model_path = "/tmp//retrained_graph.pb"
with open(tf_model_path , 'rb') as f:
serialized = f.read()
tf.reset_default_graph()
original_gdef = tf.GraphDef()
original_gdef.ParseFromString(serialized)
with tf.Graph().as_default() as g:
tf.import_graph_def(original_gdef, name ='')
ops = g.get_operations()
N = len(ops)
for i in [0,1,2,N-3,N-2,N-1]:
print('\n\nop id {} : op type: "{}"'.format(str(i), ops[i].type))
print('input(s):')
for x in ops[i].inputs:
print("name = {}, shape: {}, ".format(x.name, x.get_shape()))
print('\noutput(s):'),
for x in ops[i].outputs:
print("name = {}, shape: {},".format(x.name, x.get_shape()))
""" CONVERT TF TO CORE ML """
# Model Shape
input_tensor_shapes = {"input:0":[1,224,224,3]}
# Input Name
image_input_name = ['input:0']
# Output CoreML model path
coreml_model_file = '/tmp/myModel.mlmodel'
# Output name
output_tensor_names = ['final_result:0']
# Label file for classification
class_labels = '/tmp/retrained_labels.txt'
#Convert Process
coreml_model = tfcoreml.convert(
tf_model_path=tf_model_path,
mlmodel_path=coreml_model_file,
input_name_shape_dict=input_tensor_shapes,
output_feature_names=output_tensor_names,
image_input_names = image_input_name,
class_labels = class_labels)
# Get image pre-processing parameters of a saved CoreML model
spec = coremltools.models.utils.load_spec(coreml_model_file)
if spec.WhichOneof('Type') == 'neuralNetworkClassifier':
nn = spec.neuralNetworkClassifier
print("neuralNetworkClassifier")
if spec.WhichOneof('Type') == 'neuralNetwork':
nn = spec.neuralNetwork
print("neuralNetwork")
if spec.WhichOneof('Type') == 'neuralNetworkRegressor':
nn = spec.neuralNetworkRegressor
print("neuralNetworkClassifierRegressor")
preprocessing = nn.preprocessing[0].scaler
print('channel scale: ', preprocessing.channelScale)
print('blue bias: ', preprocessing.blueBias)
print('green bias: ', preprocessing.greenBias)
print('red bias: ', preprocessing.redBias)
inp = spec.description.input[0]
if inp.type.WhichOneof('Type') == 'imageType':
colorspace = _FeatureTypes_pb2.ImageFeatureType.ColorSpace.Name(inp.type.imageType.colorSpace)
print('colorspace: ', colorspace)
coreml_model = tfcoreml.convert(
tf_model_path=tf_model_path,
mlmodel_path=coreml_model_file,
input_name_shape_dict=input_tensor_shapes,
output_feature_names=output_tensor_names,
image_input_names = image_input_name,
class_labels = class_labels,
red_bias = -1,
green_bias = -1,
blue_bias = -1,
image_scale = 2.0/255.0)
COREMLTOOLS
下面介绍下苹果官方的转换工具。现在已经发布2.0版本了。支持很多平台：
Screen Shot 2018-09-18 at 10.26.47 AM.png
模型量化:
Screen Shot 2018-09-18 at 10.17.13 AM.png
提供一个Keras 模型转换 coreml模型脚本:
import coremltools
import keras
from keras.models import load_model
from keras.utils.generic_utils import CustomObjectScope
class_labels = []
for i in range(62):
class_labels.append(str(i))
with CustomObjectScope({'relu6': keras.applications.mobilenet.relu6}):
keras_model = load_model('traffic_sign_with_class_weights.h5')
coreml_model = coremltools.converters.keras.convert(keras_model,
input_names=['input_1'],
image_input_names='input_1',
output_names='activation_1',
image_scale=2/255.0,
red_bias=-1,
green_bias=-1,
blue_bias=-1,
class_labels=class_labels)
coreml_model.save('traffic_sign_with_class_weights.mlmodel')
我自己实现的一个人像分割模型，然后使用keras间接移植到IOS设备上，可以实时进行检测，效果还不错哈。
head_segmentation.jpg
如果有想了解深度学习模型移动端移植问题的可以咨询我的微信: ItchHacker。欢迎推荐江浙和上海地区的计算机视觉，数字图像处理岗位。
Reference
- https://qiita.com/hengsokvisal/items/dbb61851a8c76c96c700
- https://www.appcoda.com/coreml2/
- https://www.appcoda.com/coreml-introduction/
- https://www.raywenderlich.com/577-core-ml-and-vision-machine-learning-in-ios-11-tutorial
-
https://sourcediving.com/machine-learning-on-mobile-fc34be69df1a
6.https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/toco - https://blog.csdn.net/u011511601/article/details/80262707
- https://blog.algorithmia.com/machine-learning-and-mobile-deploying-models-on-the-edge/
- https://github.com/tensorflow/tensorflow/issues/15122
网友评论