美文网首页
OpenVINO使用说明

OpenVINO使用说明

作者: Mr_Michael | 来源:发表于2021-10-13 11:14 被阅读0次

一、OpenVINO部署与使用

官方教程

1.简介

OpenVINO™ 工具包是一个综合工具包,用于快速开发解决各种任务的应用程序和解决方案,包括模拟人类视觉、自动语音识别、自然语言处理、推荐系统等。

2018年发布,开源、商用免费。

1)OpenVINO™ 工具包

  • 在边缘启用基于 CNN 的深度学习推理
  • 支持跨英特尔® CPU、英特尔® 集成显卡、英特尔® 神经计算棒 2 和英特尔® 视觉加速器设计与英特尔® Movidius™ VPU 的异构执行
  • 通过易于使用的计算机视觉功能库和预先优化的内核加快上市时间
  • 包括对计算机视觉标准的优化调用,包括 OpenCV* 和 OpenCL™

2)OpenVINO™ 工具包组件

  • 深度学习模型优化器:一种跨平台命令行工具,用于导入模型并准备它们以使用推理引擎进行最佳执行。模型优化器导入、转换和优化模型,这些模型在流行框架中训练过,例如 Caffe、TensorFlow、MXNet、Kaldi 和 ONNX。
    • 优化后的模型在intel硬件平台运行更快。
    • 输入:*.pb、*.config
    • 输出:中间表达文件(IR文件)
      • *.bin:包含模型的权重和偏置值
      • *.xml:描述网络拓扑结构
  • 深度学习推理引擎:一组统一的 C++/Python API函数,允许在许多硬件类型上进行高性能推理,包括英特尔® CPU、英特尔® 集成显卡、英特尔® 神经计算棒 2、采用英特尔® Movidius™ 视觉处理单元 (VPU) 的英特尔® 视觉加速器设计。
  • 推理引擎示例:一组简单的控制台应用程序,演示如何在您的应用程序中使用推理引擎。
  • 深度学习工作台:基于 Web 的图形环境,可让您轻松使用各种复杂的 OpenVINO™ 工具包组件。
  • 训练后优化工具:用于校准模型然后以 INT8 精度执行它的工具。
  • 附加工具:一组用于处理模型的工具,包括Benchmark App、Cross Check Tool、Compile tool。
  • Open Model Zoo
  • Deep Learning Streamer (DL Streamer):基于 GStreamer 的流分析框架,用于构建媒体分析组件的图形。DL Streamer 可以通过英特尔® Distribution of OpenVINO™ 工具包安装程序进行安装。
  • OpenCV:为英特尔® 硬件编译的 OpenCV 社区版本
  • 英特尔® 媒体 SDK:(仅在面向 Linux 的英特尔® OpenVINO™ 工具套件分发版中)

3)OpenVINO™ 工具包工作流程

image
  • 模型准备、转换和优化

    • 准备或获取已训练的模型,如:行人检测、人脸检测、车辆检测、车牌识别、头部姿势。
    • 通过模型优化器运行训练后的模型,将模型转换为中间表示,其中包含一对用作推理引擎输入的.xml.bin文件。
    image
  • 运行和调整推理

    • 在应用程序中使用推理引擎API对中间表示(优化模型)运行推理并输出推理结果。
    image
  • 打包和部署

    • 英特尔 OpenVINO™ 工具包分发版为以下设备输出优化的推理运行时:
      • 英特尔® CPU
      • 英特尔® 处理器显卡
      • 英特尔® 神经计算棒 2
      • 采用英特尔® Movidius™ VPU 的英特尔® 视觉加速器设计

2.安装OpenVINO

1)环境依赖

  • Development Platforms
    • Processors
      • 6th to 11th generation Intel® Core™ processors
      • 1st to 3rd generation of Intel® Xeon® Scalable processors
    • Compatible Operating Systems
      • Ubuntu* 18.04 LTS (64 bit)
      • Ubuntu 20.04 LTS (64 bit) preview support
      • Windows® 10 (64 bit)
      • CentOS* 7 (64 bit)
      • Red Hat* Enterprise Linux* 8 (64 bit)
      • macOS* 10.15 (64 bit)
  • Target System Platforms
    • CPU
      • Processors
        • 6th to 11th generation Intel Core processors
        • 1st to 3rd generation of Intel® Xeon® Scalable processors
        • Intel Atom® processor with Intel® Streaming SIMD Extensions 4.2 (Intel® SSE4.2)
      • Compatible Operating Systems
        • Ubuntu 18.04 LTS (64 bit)
        • Ubuntu 20.04 LTS (64 bit) preview support
        • Windows 10 (64 bit)
        • CentOS 7 (64 bit)
        • Red Hat* Enterprise Linux* 8 (64 bit)
        • macOS 10.15 (64 bit)
        • Yocto Project* Poky Zeus v3.0.x (64 bit)
    • GPU
      • Processors
        • 6th to 10th generation Intel Core processor with Intel® Iris® Plus and Intel Iris Pro graphics and Intel HD Graphics
        • 11th generation Intel Core processor with Xe architecture
        • Intel Xeon processor with Intel® Iris® Plus and Intel Iris Pro graphics and Intel HD Graphics (excluding the e5 family which does not include graphics)
        • Intel® Iris® Xe MAX Graphics
      • Compatible Operating Systems
        • Ubuntu 18.04 LTS (64 bit)
        • Ubuntu 20.04 LTS (64 bit) preview support
        • Windows 10 (64 bit)
        • CentOS 7 (64 bit)
        • Red Hat* Enterprise Linux* 8 (64 bit)
    • VPU
      • Processor
        • Intel® Movidius™ Myriad™ X VPU
      • Supported Hardware
        • Intel® Neural Compute Stick 2
      • Compatible Operating Systems
        • Ubuntu 18.04 LTS (64 bit)
        • Windows 10 (64 bit)
        • CentOS 7 (64 bit)
        • macOS (64 bit)
        • Red Hat* Enterprise Linux* 8 (64 bit)

2)下载与安装

  • 下载安装包

    2021-08-16 11-04-47 的屏幕截图.png
    • 下载文件名:l_openvino_toolkit_p_2021.4.582.tgz
  • 执行安装

    # Ubuntu 18.04 LTS (64 bit) x86_64
    sudo apt-get update
    sudo apt-get install udev
    
    tar -xvf l_openvino_toolkit_p_2021.4.582.tgz
    cd l_openvino_toolkit_p_2021.4.582
    sudo su
    bash install_openvino_dependencies.sh 
    bash install.sh
    
    • 安装在目录
      • 对于 root 或管理员: /opt/intel/openvino_<version>/
      • 对于普通用户: /home/<USER>/intel/openvino_<version>/
  • 配置 Neural Compute Stick USB 驱动

    # 设置环境变量
    $ source /opt/intel/openvino_2021/bin/setupvars.sh
    [setupvars.sh] OpenVINO environment initialized
    
    # 添加规则97-myriad-usbboot.rules
    $ cd /opt/intel/openvino_2021/install_dependencies
    $ ./install_NCS_udev_rules.sh
    Updating udev rules...
    Udev rules have been successfully installed.
    
  • 安装文件夹说明

    /opt/intel/openvino_2021# tree -L 2
    ├── bin
    │   └── setupvars.sh
    ├── data_processing
    │   ├── audio
    │   ├── dl_streamer
    │   └── gstreamer
    ├── deployment_tools
    │   ├── demo
    │   ├── inference_engine  # openvino应用程序开发环境
    │   ├── intel_models -> open_model_zoo/models/intel
    │   ├── model_optimizer       # 模型优化器
    │   ├── ngraph
    │   ├── open_model_zoo    # 开源且商用免费的预训练模型库
    │   └── tools
    ├── documentation
    │   ├── OpenVINO-documentation-online.html
    │   ├── OpenVINO-GetStarted-Linux-online.html
    │   ├── OpenVINO-Install-Linux-online.html
    │   └── OpenVINO-OpenCV-documentation.html
    ├── inference_engine -> deployment_tools/inference_engine
    ├── install_dependencies
    │   ├── install_guide.html
    │   ├── install_NCS_udev_rules.sh
    │   ├── install_NEO_OCL_driver.sh
    │   ├── install_openvino_dependencies.sh
    │   ├── neo_centos_19.41.14441.sum
    │   └── neo_centos_20.35.17767.sum
    ├── licensing
    │   ├── deployment_tools
    │   ├── documentation
    │   ├── EULA.htm
    │   ├── EULA.rtf
    │   ├── EULA.txt
    │   ├── install_dependencies
    │   ├── Intel_Software_Development_Products.rtf
    │   ├── Intel_Software_Development_Products.txt
    │   ├── opencv
    │   ├── OpenVINOsupport.txt
    │   └── readme.txt
    ├── opencv
    │   ├── bin
    │   ├── build_params.txt
    │   ├── cmake
    │   ├── etc
    │   ├── include
    │   ├── lib
    │   ├── samples
    │   ├── setupvars.sh
    │   ├── third-party-programs.txt
    │   └── version.txt
    ├── openvino_toolkit_uninstaller
    │   ├── uninstall
    │   ├── uninstall_GUI.sh
    │   └── uninstall.sh
    └── python
        ├── python3
        ├── python3.6
        ├── python3.7
        └── requirements.txt
    

3)openvino demo程序

用于验证openvino toolkit和硬件驱动是否安装成功。

路径:/opt/intel/openvino_2021/deployment_tools/demo

ls demo_* -l
-rwxr-xr-x 1 root root 6879 6月  23 04:29 demo_benchmark_app.sh
-rw-r--r-- 1 root root  149 6月  23 04:29 demo_security_barrier_camera.conf
-rwxr-xr-x 1 root root 6135 6月  23 04:29 demo_security_barrier_camera.sh
-rwxr-xr-x 1 root root 7664 6月  23 04:29 demo_speech_recognition.sh
-rwxr-xr-x 1 root root 6821 6月  23 04:29 demo_squeezenet_download_convert_run.sh

# 基于Core(TM) i7-1165G7 CPU
./demo_benchmark_app.sh -d CPU
target = CPU
target_precision = FP16
Full device name: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
Count:      1000 iterations
Duration:   4255.99 ms
Latency:    13.66 ms
Throughput: 234.96 FPS

# 基于Core(TM) i7-6500U CPU
./demo_benchmark_app.sh -d CPU
target = CPU
target_precision = FP16
Full device name: Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
Count:      1000 iterations
Duration:   9813.88 ms
Latency:    32.30 ms
Throughput: 101.90 FPS

# 基于MYRIAD X
./demo_benchmark_app.sh -d MYRIAD
target = MYRIAD
target_precision = FP16
Total time: 6083     microseconds
Full device name: Intel Movidius Myriad X VPU
Count:      1000 iterations
Duration:   3517.02 ms
Latency:    14.00 ms
Throughput: 284.33 FPS

4)Inference Engine Samples

用于验证openvino toolkit的开发环境是否安装成功。

路径:/opt/intel/openvino_2021/deployment_tools/inference_engine/samples

tree -L 2
.
├── c
│   ├── build_samples.sh
│   ├── CMakeLists.txt
│   ├── common
│   ├── hello_classification
│   ├── hello_nv12_input_classification
│   └── object_detection_sample_ssd
├── cpp
│   ├── benchmark_app
│   ├── build_samples.sh
│   ├── classification_sample_async
│   ├── CMakeLists.txt
│   ├── common
│   ├── hello_classification
│   ├── hello_nv12_input_classification
│   ├── hello_query_device
│   ├── hello_reshape_ssd
│   ├── ngraph_function_creation_sample
│   ├── object_detection_sample_ssd
│   ├── speech_sample
│   ├── style_transfer_sample
│   └── thirdparty
└── python
    ├── classification_sample_async
    ├── hello_classification
    ├── hello_query_device
    ├── hello_reshape_ssd
    ├── ngraph_function_creation_sample
    ├── object_detection_sample_ssd
    ├── requirements.txt
    ├── speech_sample
    └── style_transfer_sample

编译CPP版本samples

cd cpp && ./build_samples.sh
文件生成路径:/root/inference_engine_cpp_samples_build/intel64/Release
cd /root/inference_engine_cpp_samples_build/intel64/Release
  • 编写测试脚本cppsample_test.sh

    #!/bin/bash
    openvino_lib_path=/opt/intel/openvino_2021/deployment_tools/inference_engine/external/tbb/lib/
    openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/
    openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/ngraph/lib/
    
    export LD_LIBRARY_PATH=$openvino_lib_path
    ./hello_query_device
    
  • 执行脚本

     bash cppsample_test.sh 
    Loading Inference Engine
    Available devices: 
    CPU
      SUPPORTED_METRICS: 
          AVAILABLE_DEVICES : [  ]
          FULL_DEVICE_NAME : Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
          OPTIMIZATION_CAPABILITIES : [ FP32 FP16 INT8 BIN ]
          RANGE_FOR_ASYNC_INFER_REQUESTS : { 1, 1, 1 }
          RANGE_FOR_STREAMS : { 1, 4 }
      SUPPORTED_CONFIG_KEYS (default values): 
          CPU_BIND_THREAD : YES
          CPU_THREADS_NUM : 0
          CPU_THROUGHPUT_STREAMS : 1
          DUMP_EXEC_GRAPH_AS_DOT : ""
          DYN_BATCH_ENABLED : NO
          DYN_BATCH_LIMIT : 0
          ENFORCE_BF16 : NO
          EXCLUSIVE_ASYNC_REQUESTS : NO
          PERF_COUNT : NO
    
    MYRIAD
      SUPPORTED_METRICS: 
          IMPORT_EXPORT_SUPPORT : true
          DEVICE_ARCHITECTURE : MYRIAD
          RANGE_FOR_ASYNC_INFER_REQUESTS : { 3, 6, 1 }
          OPTIMIZATION_CAPABILITIES : [ FP16 ]
          DEVICE_THERMAL : EMPTY VALUE
          FULL_DEVICE_NAME : Intel Movidius Myriad X VPU
          AVAILABLE_DEVICES : [ 3.1-ma2480 ]
      SUPPORTED_CONFIG_KEYS (default values): 
          DEVICE_ID : ""
          EXCLUSIVE_ASYNC_REQUESTS : NO
          LOG_LEVEL : LOG_NONE
          VPU_MYRIAD_FORCE_RESET : NO
          VPU_MYRIAD_PLATFORM : ""
          VPU_CUSTOM_LAYERS : ""
          PERF_COUNT : NO
          VPU_PRINT_RECEIVE_TENSOR_TIME : NO
          CONFIG_FILE : ""
          VPU_HW_STAGES_OPTIMIZATION : YES
          MYRIAD_THROUGHPUT_STREAMS : -1
          MYRIAD_ENABLE_FORCE_RESET : NO
          MYRIAD_ENABLE_RECEIVING_TENSOR_TIME : NO
          MYRIAD_CUSTOM_LAYERS : ""
          MYRIAD_ENABLE_HW_ACCELERATION : YES
    

5)Inference Engine Demo

如果Inference Engine sample和demo编译成功,说明开发环境安装成功。

路径:/opt/intel/openvino_2021/deployment_tools/inference_engine/demos(../../open_model_zoo/demos的软链接)

$ tree -L 1
├── 3d_segmentation_demo
├── action_recognition_demo
├── bert_named_entity_recognition_demo
├── bert_question_answering_demo
├── bert_question_answering_embedding_demo
├── build_demos.sh
├── classification_demo
├── CMakeLists.txt
├── colorization_demo
├── common
├── crossroad_camera_demo
├── deblurring_demo
├── face_detection_mtcnn_demo
├── face_recognition_demo
├── formula_recognition_demo
├── gaze_estimation_demo
├── gesture_recognition_demo
├── handwritten_text_recognition_demo
├── human_pose_estimation_3d_demo
├── human_pose_estimation_demo
├── image_inpainting_demo
├── image_processing_demo
├── image_retrieval_demo
├── image_translation_demo
├── instance_segmentation_demo
├── interactive_face_detection_demo
├── machine_translation_demo
├── mask_rcnn_demo
├── monodepth_demo
├── multi_camera_multi_target_tracking_demo
├── multi_channel_common
├── multi_channel_face_detection_demo
├── multi_channel_human_pose_estimation_demo
├── multi_channel_object_detection_demo_yolov3
├── noise_suppression_demo
├── object_detection_demo
├── pedestrian_tracker_demo
├── place_recognition_demo
├── README.md
├── requirements.txt
├── security_barrier_camera_demo
├── segmentation_demo
├── single_human_pose_estimation_demo
├── smart_classroom_demo
├── social_distance_demo
├── sound_classification_demo
├── speech_recognition_deepspeech_demo
├── speech_recognition_quartznet_demo
├── text_detection_demo
├── text_spotting_demo
├── text_to_speech_demo
├── thirdparty
├── time_series_forecasting_demo
└── whiteboard_inpainting_demo
# 编译模型
$ ./build_demos.sh
    # 文件生成路径 :$HOME/omz_demos_build
cd /root/omz_demos_build/intel64/Release 

编写测试脚本demo_test.sh

#!/bin/bash
openvino_lib_path=/opt/intel/openvino_2021/deployment_tools/inference_engine/external/tbb/lib/
openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/
openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/ngraph/lib/

export LD_LIBRARY_PATH=$openvino_lib_path
./xxx_demo xxx xxx
    # 想要运行程序,还需要下载AI模型和测试图片

3.从Open Model Zoo下载并使用模型

路径:/opt/intel/openvino_2021/deployment_tools/open_model_zoo

工程目录

tree -L 2
├── CONTRIBUTING.md
├── data
│   ├── dataset_classes
│   ├── dataset_definitions.yml
│   ├── datasets.md
│   └── palettes
├── demos   # 同inference_engine/demos
│   ├── 3d_segmentation_demo
│   ├── ......
│   └── whiteboard_inpainting_demo
├── intel_models -> models/intel
├── LICENSE
├── licensing
│   └── third-party-programs.txt
├── models      # 开源的商用免费AI模型
│   ├── intel
│   └── public
├── README.md
├── tools
│   ├── accuracy_checker
│   └── downloader
│       ├── converter.py        # 模型转化工具
│       ├── downloader.py   # 模型下载工具
│       ├── info_dumper.py
│       ├── pyproject.toml
│       ├── quantizer.py
│       ├── README.md
│       ├── requirements-caffe2.in
│       ├── requirements.in
│       ├── requirements-pytorch.in
│       ├── requirements-tensorflow.in
│       ├── setup.cfg
│       ├── setup.py
│       └── src
└── version.txt

1)downloader.py使用

模型下载到/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/下各自的AI模型路径中,包含FP32、FP16、FP16-INT8等多种精度的模型。

# 显示所有可以下载的模型
python3 downloader.py --print_all

# 下载所有模型
python3 downloader.py --all

# 下载特定模型,用逗号分割
python3 downloader.py --name human-pose-estimation-0001,face-detection-adas-0001,age-gender-recognition-retail-0013
################|| Downloading human-pose-estimation-0001 ||################
========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP32/human-pose-estimation-0001.xml
... 100%, 150 KB, 264 KB/s, 0 seconds passed

========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP32/human-pose-estimation-0001.bin
... 100%, 16010 KB, 6227 KB/s, 2 seconds passed

========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16/human-pose-estimation-0001.xml
... 100%, 150 KB, 277 KB/s, 0 seconds passed

========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16/human-pose-estimation-0001.bin
... 100%, 8005 KB, 3655 KB/s, 2 seconds passed

========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16-INT8/human-pose-estimation-0001.xml
... 100%, 416 KB, 524 KB/s, 0 seconds passed

========== Downloading /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001/FP16-INT8/human-pose-estimation-0001.bin
... 100%, 4072 KB, 6788 KB/s, 0 seconds passed

2)测试AI模型

以 human-pose-estimation-0001为例进行测试

  • architecture_type = openpose
    • human-pose-estimation-0001
  • architecture_type = ae
    • human-pose-estimation-0005
    • human-pose-estimation-0006
    • human-pose-estimation-0007
  • architecture_type = higherhrnet
    • higher-hrnet-w32-human-pose-estimation
# 下载测试视频
cd /opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001
# 将视频 https://github.com/intel-iot-devkit/sample-videos/blob/master/head-pose-face-detection-female-and-male.mp4 下载到当前路径

cd /root/omz_demos_build/intel64/Release 

编写测试程序demo_test.sh

#!/bin/bash
openvino_lib_path=/opt/intel/openvino_2021/deployment_tools/inference_engine/external/tbb/lib/
openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/inference_engine/lib/intel64/
openvino_lib_path=$openvino_lib_path:/opt/intel/openvino_2021/deployment_tools/ngraph/lib/

aipath=/opt/intel/openvino_2021.4.582/deployment_tools/open_model_zoo/tools/downloader/intel/human-pose-estimation-0001

export LD_LIBRARY_PATH=$openvino_lib_path
./human_pose_estimation_demo -at openpose -i $aipath/head-pose-face-detection-female-and-male.mp4 -m $aipath/FP16/human-pose-estimation-0001.xml -d MYRIAD

运行程序

$ bash demo_test.sh
[ INFO ] InferenceEngine:   IE version ......... 2021.4
    Build ........... 0
[ INFO ] Parsing input parameters
[ INFO ] Reading input
[ INFO ] Loading Inference Engine
[ INFO ] Device info: 
[ INFO ]    MYRIAD
    myriadPlugin version ......... 2021.4
    Build ........... 0
Loading network files
[ INFO ] Batch size is forced to 1.
[ INFO ] Loading model to the device

[ INFO ] Metric reports:
Latency: 517.2 ms
FPS: 9.4
[ INFO ] 
[ INFO ] The execution has completed successfully

$ bash demo_test.sh
[ INFO ] InferenceEngine:   IE version ......... 2021.4
    Build ........... 0
[ INFO ] Parsing input parameters
[ INFO ] Reading input
[ INFO ] Loading Inference Engine
[ INFO ] Device info: 
[ INFO ]    CPU
    MKLDNNPlugin version ......... 2021.4
    Build ........... 0
Loading network files
[ INFO ] Batch size is forced to 1.
[ INFO ] Loading model to the device

[ INFO ] Metric reports:
Latency: 813.5 ms
FPS: 6.1
[ INFO ] 
[ INFO ] The execution has completed successfully
  • 备注:需要在桌面版ubutnu执行才能运行成功(ssh远程执行会失败),否则会报错opencv初始化GTK backend失败
  • 使用MYRIAD运行,程序CPU占用率47%(系统CPU占用率14%),FPS: 9.4,Latency: 517.2 ms
  • 使用CPU( Intel(R) Core(TM) i7-6500U )运行,程序CPU占用率330%(系统CPU占用率100%),FPS: 6.1,Latency: 813.5 ms

相关文章

网友评论

      本文标题:OpenVINO使用说明

      本文链接:https://www.haomeiwen.com/subject/wuvsoltx.html