一、OpenVINO部署与使用
1.简介
OpenVINO™ 工具包是一个综合工具包,用于快速开发解决各种任务的应用程序和解决方案,包括模拟人类视觉、自动语音识别、自然语言处理、推荐系统等。
2018年发布,开源、商用免费。
1)OpenVINO™ 工具包
- 在边缘启用基于 CNN 的深度学习推理
- 支持跨英特尔® CPU、英特尔® 集成显卡、英特尔® 神经计算棒 2 和英特尔® 视觉加速器设计与英特尔® Movidius™ VPU 的异构执行
- 通过易于使用的计算机视觉功能库和预先优化的内核加快上市时间
- 包括对计算机视觉标准的优化调用,包括 OpenCV* 和 OpenCL™
2)OpenVINO™ 工具包组件
- 深度学习模型优化器:一种跨平台命令行工具,用于导入模型并准备它们以使用推理引擎进行最佳执行。模型优化器导入、转换和优化模型,这些模型在流行框架中训练过,例如 Caffe、TensorFlow、MXNet、Kaldi 和 ONNX。
- 优化后的模型在intel硬件平台运行更快。
- 输入:*.pb、*.config
- 输出:中间表达文件(IR文件)
- *.bin:包含模型的权重和偏置值
- *.xml:描述网络拓扑结构
- 深度学习推理引擎:一组统一的 C++/Python API函数,允许在许多硬件类型上进行高性能推理,包括英特尔® CPU、英特尔® 集成显卡、英特尔® 神经计算棒 2、采用英特尔® Movidius™ 视觉处理单元 (VPU) 的英特尔® 视觉加速器设计。
- 推理引擎示例:一组简单的控制台应用程序,演示如何在您的应用程序中使用推理引擎。
- 深度学习工作台:基于 Web 的图形环境,可让您轻松使用各种复杂的 OpenVINO™ 工具包组件。
- 训练后优化工具:用于校准模型然后以 INT8 精度执行它的工具。
- 附加工具:一组用于处理模型的工具,包括Benchmark App、Cross Check Tool、Compile tool。
- Open Model Zoo
- 包括针对各种视觉问题的深度学习解决方案,包括对象识别、人脸识别、姿势估计、文本检测和动作识别
- 附加工具:一组用于处理模型的工具,包括Accuracy Checker Utility和Model Downloader。
- 预训练模型文档:Open Model Zoo github仓库中提供的预训练模型文档。
- Tensorflow预训练模型库
- Deep Learning Streamer (DL Streamer):基于 GStreamer 的流分析框架,用于构建媒体分析组件的图形。DL Streamer 可以通过英特尔® Distribution of OpenVINO™ 工具包安装程序进行安装。
- OpenCV:为英特尔® 硬件编译的 OpenCV 社区版本
- 英特尔® 媒体 SDK:(仅在面向 Linux 的英特尔® OpenVINO™ 工具套件分发版中)
3)OpenVINO™ 工具包工作流程

-
模型准备、转换和优化
- 准备或获取已训练的模型,如:行人检测、人脸检测、车辆检测、车牌识别、头部姿势。
- 通过模型优化器运行训练后的模型,将模型转换为中间表示,其中包含一对用作推理引擎输入的
.xml
和.bin
文件。
image
-
运行和调整推理
- 在应用程序中使用推理引擎API对中间表示(优化模型)运行推理并输出推理结果。
image
-
打包和部署
- 英特尔 OpenVINO™ 工具包分发版为以下设备输出优化的推理运行时:
- 英特尔® CPU
- 英特尔® 处理器显卡
- 英特尔® 神经计算棒 2
- 采用英特尔® Movidius™ VPU 的英特尔® 视觉加速器设计
- 英特尔 OpenVINO™ 工具包分发版为以下设备输出优化的推理运行时:
2.安装OpenVINO
1)环境依赖
- Development Platforms
- Processors
- 6th to 11th generation Intel® Core™ processors
- 1st to 3rd generation of Intel® Xeon® Scalable processors
- Compatible Operating Systems
- Ubuntu* 18.04 LTS (64 bit)
- Ubuntu 20.04 LTS (64 bit) preview support
- Windows® 10 (64 bit)
- CentOS* 7 (64 bit)
- Red Hat* Enterprise Linux* 8 (64 bit)
- macOS* 10.15 (64 bit)
- Processors
- Target System Platforms
- CPU
- Processors
- 6th to 11th generation Intel Core processors
- 1st to 3rd generation of Intel® Xeon® Scalable processors
- Intel Atom® processor with Intel® Streaming SIMD Extensions 4.2 (Intel® SSE4.2)
- Compatible Operating Systems
- Ubuntu 18.04 LTS (64 bit)
- Ubuntu 20.04 LTS (64 bit) preview support
- Windows 10 (64 bit)
- CentOS 7 (64 bit)
- Red Hat* Enterprise Linux* 8 (64 bit)
- macOS 10.15 (64 bit)
- Yocto Project* Poky Zeus v3.0.x (64 bit)
- Processors
- GPU
- Processors
- 6th to 10th generation Intel Core processor with Intel® Iris® Plus and Intel Iris Pro graphics and Intel HD Graphics
- 11th generation Intel Core processor with Xe architecture
- Intel Xeon processor with Intel® Iris® Plus and Intel Iris Pro graphics and Intel HD Graphics (excluding the e5 family which does not include graphics)
- Intel® Iris® Xe MAX Graphics
- Compatible Operating Systems
- Ubuntu 18.04 LTS (64 bit)
- Ubuntu 20.04 LTS (64 bit) preview support
- Windows 10 (64 bit)
- CentOS 7 (64 bit)
- Red Hat* Enterprise Linux* 8 (64 bit)
- Processors
- VPU
- Processor
- Intel® Movidius™ Myriad™ X VPU
- Supported Hardware
- Intel® Neural Compute Stick 2
- Compatible Operating Systems
- Ubuntu 18.04 LTS (64 bit)
- Windows 10 (64 bit)
- CentOS 7 (64 bit)
- macOS (64 bit)
- Red Hat* Enterprise Linux* 8 (64 bit)
- Processor
- CPU
2)下载与安装
-
下载安装包
- Download the appropriate version of the Intel® Distribution of the OpenVINO™ toolkit for your host computer.
2021-08-16 11-04-47 的屏幕截图.png
- 下载文件名:l_openvino_toolkit_p_2021.4.582.tgz
-
执行安装
# Ubuntu 18.04 LTS (64 bit) x86_64 sudo apt-get update sudo apt-get install udev tar -xvf l_openvino_toolkit_p_2021.4.582.tgz cd l_openvino_toolkit_p_2021.4.582 sudo su bash install_openvino_dependencies.sh bash install.sh
- 安装在目录
- 对于 root 或管理员:
/opt/intel/openvino_<version>/
- 对于普通用户:
/home/<USER>/intel/openvino_<version>/
- 对于 root 或管理员:
- 安装在目录
-
配置 Neural Compute Stick USB 驱动
# 设置环境变量 $ source /opt/intel/openvino_2021/bin/setupvars.sh [setupvars.sh] OpenVINO environment initialized # 添加规则97-myriad-usbboot.rules $ cd /opt/intel/openvino_2021/install_dependencies $ ./install_NCS_udev_rules.sh Updating udev rules... Udev rules have been successfully installed.
-
安装文件夹说明
/opt/intel/openvino_2021# tree -L 2 ├── bin │ └── setupvars.sh ├── data_processing │ ├── audio │ ├── dl_streamer │ └── gstreamer ├── deployment_tools │ ├── demo │ ├── inference_engine # openvino应用程序开发环境 │ ├── intel_models -> open_model_zoo/models/intel │ ├── model_optimizer # 模型优化器 │ ├── ngraph │ ├── open_model_zoo # 开源且商用免费的预训练模型库 │ └── tools ├── documentation │ ├── OpenVINO-documentation-online.html │ ├── OpenVINO-GetStarted-Linux-online.html │ ├── OpenVINO-Install-Linux-online.html │ └── OpenVINO-OpenCV-documentation.html ├── inference_engine -> deployment_tools/inference_engine ├── install_dependencies │ ├── install_guide.html │ ├── install_NCS_udev_rules.sh │ ├── install_NEO_OCL_driver.sh │ ├── install_openvino_dependencies.sh │ ├── neo_centos_19.41.14441.sum │ └── neo_centos_20.35.17767.sum ├── licensing │ ├── deployment_tools │ ├── documentation │ ├── EULA.htm │ ├── EULA.rtf │ ├── EULA.txt │ ├── install_dependencies │ ├── Intel_Software_Development_Products.rtf │ ├── Intel_Software_Development_Products.txt │ ├── opencv │ ├── OpenVINOsupport.txt │ └── readme.txt ├── opencv │ ├── bin │ ├── build_params.txt │ ├── cmake │ ├── etc │ ├── include │ ├── lib │ ├── samples │ ├── setupvars.sh │ ├── third-party-programs.txt │ └── version.txt ├── openvino_toolkit_uninstaller │ ├── uninstall │ ├── uninstall_GUI.sh │ └── uninstall.sh └── python ├── python3 ├── python3.6 ├── python3.7 └── requirements.txt
3)openvino demo程序
用于验证openvino toolkit和硬件驱动是否安装成功。
路径:/opt/intel/openvino_2021/deployment_tools/demo
ls demo_* -l
-rwxr-xr-x 1 root root 6879 6月 23 04:29 demo_benchmark_app.sh
-rw-r--r-- 1 root root 149 6月 23 04:29 demo_security_barrier_camera.conf
-rwxr-xr-x 1 root root 6135 6月 23 04:29 demo_security_barrier_camera.sh
-rwxr-xr-x 1 root root 7664 6月 23 04:29 demo_speech_recognition.sh
-rwxr-xr-x 1 root root 6821 6月 23 04:29 demo_squeezenet_download_convert_run.sh
# 基于Core(TM) i7-1165G7 CPU
./demo_benchmark_app.sh -d CPU
target = CPU
target_precision = FP16
Full device name: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
Count: 1000 iterations
Duration: 4255.99 ms
Latency: 13.66 ms
Throughput: 234.96 FPS
# 基于Core(TM) i7-6500U CPU
./demo_benchmark_app.sh -d CPU
target = CPU
target_precision = FP16
Full device name: Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
Count: 1000 iterations
Duration: 9813.88 ms
Latency: 32.30 ms
Throughput: 101.90 FPS
# 基于MYRIAD X
./demo_benchmark_app.sh -d MYRIAD
target = MYRIAD
target_precision = FP16
Total time: 6083 microseconds
Full device name: Intel Movidius Myriad X VPU
Count: 1000 iterations
Duration: 3517.02 ms
Latency: 14.00 ms
Throughput: 284.33 FPS
4)Inference Engine Samples
用于验证openvino toolkit的开发环境是否安装成功。
路径:/opt/intel/openvino_2021/deployment_tools/inference_engine/samples
tree -L 2
.
├── c
│ ├── build_samples.sh
│ ├── CMakeLists.txt
│ ├── common
│ ├── hello_classification
│ ├── hello_nv12_input_classification
│ └── object_detection_sample_ssd
├── cpp
│ ├── benchmark_app
│ ├── build_samples.sh
│ ├── classification_sample_async
│ ├── CMakeLists.txt
│ ├── common
│ ├── hello_classification
│ ├── hello_nv12_input_classification
│ ├── hello_query_device
│ ├── hello_reshape_ssd
│ ├── ngraph_function_creation_sample
│ ├── object_detection_sample_ssd
│ ├── speech_sample
│ ├── style_transfer_sample
│ └── thirdparty
└── python
├── classification_sample_async
├── hello_classification
├── hello_query_device
├── hello_reshape_ssd
├── ngraph_function_creation_sample
├── object_detection_sample_ssd
├── requirements.txt
├── speech_sample
└── style_transfer_sample
编译CPP版本samples
cd cpp && ./build_samples.sh
文件生成路径:/root/inference_engine_cpp_samples_build/intel64/Release
cd /root/inference_engine_cpp_samples_build/intel64/Release
-
编写测试脚本cppsample_test.sh
#!/bin/bash openvino_lib_path=/opt/intel/openvino_2021/deployment_tools/inference_engine/external/tbb/lib/ openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/ openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/ngraph/lib/ export LD_LIBRARY_PATH=$openvino_lib_path ./hello_query_device
-
执行脚本
bash cppsample_test.sh Loading Inference Engine Available devices: CPU SUPPORTED_METRICS: AVAILABLE_DEVICES : [ ] FULL_DEVICE_NAME : Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz OPTIMIZATION_CAPABILITIES : [ FP32 FP16 INT8 BIN ] RANGE_FOR_ASYNC_INFER_REQUESTS : { 1, 1, 1 } RANGE_FOR_STREAMS : { 1, 4 } SUPPORTED_CONFIG_KEYS (default values): CPU_BIND_THREAD : YES CPU_THREADS_NUM : 0 CPU_THROUGHPUT_STREAMS : 1 DUMP_EXEC_GRAPH_AS_DOT : "" DYN_BATCH_ENABLED : NO DYN_BATCH_LIMIT : 0 ENFORCE_BF16 : NO EXCLUSIVE_ASYNC_REQUESTS : NO PERF_COUNT : NO MYRIAD SUPPORTED_METRICS: IMPORT_EXPORT_SUPPORT : true DEVICE_ARCHITECTURE : MYRIAD RANGE_FOR_ASYNC_INFER_REQUESTS : { 3, 6, 1 } OPTIMIZATION_CAPABILITIES : [ FP16 ] DEVICE_THERMAL : EMPTY VALUE FULL_DEVICE_NAME : Intel Movidius Myriad X VPU AVAILABLE_DEVICES : [ 3.1-ma2480 ] SUPPORTED_CONFIG_KEYS (default values): DEVICE_ID : "" EXCLUSIVE_ASYNC_REQUESTS : NO LOG_LEVEL : LOG_NONE VPU_MYRIAD_FORCE_RESET : NO VPU_MYRIAD_PLATFORM : "" VPU_CUSTOM_LAYERS : "" PERF_COUNT : NO VPU_PRINT_RECEIVE_TENSOR_TIME : NO CONFIG_FILE : "" VPU_HW_STAGES_OPTIMIZATION : YES MYRIAD_THROUGHPUT_STREAMS : -1 MYRIAD_ENABLE_FORCE_RESET : NO MYRIAD_ENABLE_RECEIVING_TENSOR_TIME : NO MYRIAD_CUSTOM_LAYERS : "" MYRIAD_ENABLE_HW_ACCELERATION : YES
5)Inference Engine Demo
如果Inference Engine sample和demo编译成功,说明开发环境安装成功。
路径:/opt/intel/openvino_2021/deployment_tools/inference_engine/demos(../../open_model_zoo/demos的软链接)
$ tree -L 1
├── 3d_segmentation_demo
├── action_recognition_demo
├── bert_named_entity_recognition_demo
├── bert_question_answering_demo
├── bert_question_answering_embedding_demo
├── build_demos.sh
├── classification_demo
├── CMakeLists.txt
├── colorization_demo
├── common
├── crossroad_camera_demo
├── deblurring_demo
├── face_detection_mtcnn_demo
├── face_recognition_demo
├── formula_recognition_demo
├── gaze_estimation_demo
├── gesture_recognition_demo
├── handwritten_text_recognition_demo
├── human_pose_estimation_3d_demo
├── human_pose_estimation_demo
├── image_inpainting_demo
├── image_processing_demo
├── image_retrieval_demo
├── image_translation_demo
├── instance_segmentation_demo
├── interactive_face_detection_demo
├── machine_translation_demo
├── mask_rcnn_demo
├── monodepth_demo
├── multi_camera_multi_target_tracking_demo
├── multi_channel_common
├── multi_channel_face_detection_demo
├── multi_channel_human_pose_estimation_demo
├── multi_channel_object_detection_demo_yolov3
├── noise_suppression_demo
├── object_detection_demo
├── pedestrian_tracker_demo
├── place_recognition_demo
├── README.md
├── requirements.txt
├── security_barrier_camera_demo
├── segmentation_demo
├── single_human_pose_estimation_demo
├── smart_classroom_demo
├── social_distance_demo
├── sound_classification_demo
├── speech_recognition_deepspeech_demo
├── speech_recognition_quartznet_demo
├── text_detection_demo
├── text_spotting_demo
├── text_to_speech_demo
├── thirdparty
├── time_series_forecasting_demo
└── whiteboard_inpainting_demo
# 编译模型
$ ./build_demos.sh
# 文件生成路径 :$HOME/omz_demos_build
cd /root/omz_demos_build/intel64/Release
编写测试脚本demo_test.sh
#!/bin/bash
openvino_lib_path=/opt/intel/openvino_2021/deployment_tools/inference_engine/external/tbb/lib/
openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/
openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/ngraph/lib/
export LD_LIBRARY_PATH=$openvino_lib_path
./xxx_demo xxx xxx
# 想要运行程序,还需要下载AI模型和测试图片
3.从Open Model Zoo下载并使用模型
路径:/opt/intel/openvino_2021/deployment_tools/open_model_zoo
工程目录
tree -L 2
├── CONTRIBUTING.md
├── data
│ ├── dataset_classes
│ ├── dataset_definitions.yml
│ ├── datasets.md
│ └── palettes
├── demos # 同inference_engine/demos
│ ├── 3d_segmentation_demo
│ ├── ......
│ └── whiteboard_inpainting_demo
├── intel_models -> models/intel
├── LICENSE
├── licensing
│ └── third-party-programs.txt
├── models # 开源的商用免费AI模型
│ ├── intel
│ └── public
├── README.md
├── tools
│ ├── accuracy_checker
│ └── downloader
│ ├── converter.py # 模型转化工具
│ ├── downloader.py # 模型下载工具
│ ├── info_dumper.py
│ ├── pyproject.toml
│ ├── quantizer.py
│ ├── README.md
│ ├── requirements-caffe2.in
│ ├── requirements.in
│ ├── requirements-pytorch.in
│ ├── requirements-tensorflow.in
│ ├── setup.cfg
│ ├── setup.py
│ └── src
└── version.txt
1)downloader.py使用
模型下载到/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/下各自的AI模型路径中,包含FP32、FP16、FP16-INT8等多种精度的模型。
# 显示所有可以下载的模型
python3 downloader.py --print_all
# 下载所有模型
python3 downloader.py --all
# 下载特定模型,用逗号分割
python3 downloader.py --name human-pose-estimation-0001,face-detection-adas-0001,age-gender-recognition-retail-0013
################|| Downloading human-pose-estimation-0001 ||################
========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP32/human-pose-estimation-0001.xml
... 100%, 150 KB, 264 KB/s, 0 seconds passed
========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP32/human-pose-estimation-0001.bin
... 100%, 16010 KB, 6227 KB/s, 2 seconds passed
========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16/human-pose-estimation-0001.xml
... 100%, 150 KB, 277 KB/s, 0 seconds passed
========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16/human-pose-estimation-0001.bin
... 100%, 8005 KB, 3655 KB/s, 2 seconds passed
========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16-INT8/human-pose-estimation-0001.xml
... 100%, 416 KB, 524 KB/s, 0 seconds passed
========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16-INT8/human-pose-estimation-0001.bin
... 100%, 4072 KB, 6788 KB/s, 0 seconds passed
2)测试AI模型
以 human-pose-estimation-0001为例进行测试
- architecture_type = openpose
- human-pose-estimation-0001
- architecture_type = ae
- human-pose-estimation-0005
- human-pose-estimation-0006
- human-pose-estimation-0007
- architecture_type = higherhrnet
- higher-hrnet-w32-human-pose-estimation
# 下载测试视频
cd /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001
# 将视频 https://github.com/intel-iot-devkit/sample-videos/blob/master/head-pose-face-detection-female-and-male.mp4 下载到当前路径
cd /root/omz_demos_build/intel64/Release
编写测试程序demo_test.sh
#!/bin/bash
openvino_lib_path=/opt/intel/openvino_2021/deployment_tools/inference_engine/external/tbb/lib/
openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/
openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/ngraph/lib/
aipath=/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001
export LD_LIBRARY_PATH=$openvino_lib_path
./human_pose_estimation_demo -at openpose -i $aipath/head-pose-face-detection-female-and-male.mp4 -m $aipath/FP16/human-pose-estimation-0001.xml -d MYRIAD
运行程序
$ bash demo_test.sh
[ INFO ] InferenceEngine: IE version ......... 2021.4
Build ........... 0
[ INFO ] Parsing input parameters
[ INFO ] Reading input
[ INFO ] Loading Inference Engine
[ INFO ] Device info:
[ INFO ] MYRIAD
myriadPlugin version ......... 2021.4
Build ........... 0
Loading network files
[ INFO ] Batch size is forced to 1.
[ INFO ] Loading model to the device
[ INFO ] Metric reports:
Latency: 517.2 ms
FPS: 9.4
[ INFO ]
[ INFO ] The execution has completed successfully
$ bash demo_test.sh
[ INFO ] InferenceEngine: IE version ......... 2021.4
Build ........... 0
[ INFO ] Parsing input parameters
[ INFO ] Reading input
[ INFO ] Loading Inference Engine
[ INFO ] Device info:
[ INFO ] CPU
MKLDNNPlugin version ......... 2021.4
Build ........... 0
Loading network files
[ INFO ] Batch size is forced to 1.
[ INFO ] Loading model to the device
[ INFO ] Metric reports:
Latency: 813.5 ms
FPS: 6.1
[ INFO ]
[ INFO ] The execution has completed successfully
- 备注:需要在桌面版ubutnu执行才能运行成功(ssh远程执行会失败),否则会报错opencv初始化GTK backend失败
- 使用MYRIAD运行,程序CPU占用率47%(系统CPU占用率14%),FPS: 9.4,Latency: 517.2 ms
- 使用CPU( Intel(R) Core(TM) i7-6500U )运行,程序CPU占用率330%(系统CPU占用率100%),FPS: 6.1,Latency: 813.5 ms
网友评论