美文网首页
paddlepaddle--paddlehub--chinese

paddlepaddle--paddlehub--chinese

作者: 前浪浪奔浪流 | 来源:发表于2023-09-24 17:20 被阅读0次

    第1步:安装Anaconda

    • 说明:使用paddlepaddle需要先安装python环境,这里我们选择python集成环境Anaconda工具包

      • Anaconda是1个常用的python包管理程序;
      • 安装完Anaconda后,可以安装python环境,以及numpy等所需的工具包环境;

      下载Anaconda:

    • 下载地址:https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/?C=M&O=D

      image.png
    • 选择适合您操作系统的版本

      • 可在终端输入uname -m查询系统所用的指令集
    [root@test-web2 /]# uname -m
    x86_64
    
    - 安装Anaconda:
    [root@test-web2 ~]# chmod +x Anaconda3-2023.07-2-Linux-x86_64.sh 
    [root@test-web2 ~]# ll Anaconda3-2023.07-2-Linux-x86_64.sh 
    -rwxr-xr-x 1 root root 1064920017 9月  25 11:17 Anaconda3-2023.07-2-Linux-x86_64.sh
    sh Anaconda3-2023.07-2-Linux-x86_64.sh
    若您下载的是其它版本,则将该命令的文件名替换为您下载的文件名
    按照安装提示安装即可
    查看许可时可输入q来退出
    
    image.png image.png image.png image.png image.png

    将conda加入环境变量

    • 加入环境变量是为了让系统能识别conda命令,若您在安装时已将conda加入环境变量path,则可跳过本步
    • 在终端中打开/etc/profile:
    # 在终端中输入以下命令:
    vim /etc/profile
    ... ...
    export PATH="/root/anaconda3/bin:$PATH"
    
    • 验证是否能识别conda命令:
    source /etc/profile
    conda info --envs   #若能显示当前有base环境,则conda已加入环境变量
    
    image.png

    第2步:创建conda环境

    • 创建新的conda环境
    # 在命令行输入以下命令,创建名为paddle_env的环境
    # 此处为加速下载,使用清华源
    conda create --name paddle_env python=3.10 --channel https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
    
    image.png image.png
    • 该命令会创建1个名为paddle_env、python版本为3.10的可执行环境,根据网络状态,需要花费一段时间
    • 之后命令行中会输出提示信息,输入y并回车继续安装

    激活刚创建的conda环境,在命令行中输入以下命令:

    # 激活paddle_env环境
    conda activate paddle_env
    

    报错:

    # conda activate paddle_env
    usage: conda [-h] [--no-plugins] [-V] COMMAND ...
    conda: error: argument COMMAND: invalid choice: 'activate' (choose from 'clean', 'compare', 'config', 'create', 'info', 'init', 'install', 'list', 'notices', 'package', 'remove', 'uninstall', 'rename', 'run', 'search', 'update', 'upgrade', 'build', 'convert', 'debug', 'develop', 'doctor', 'index', 'inspect', 'metapackage', 'render', 'skeleton', 'pack', 'verify', 'token', 'repo', 'env', 'server', 'content-trust')
    

    解决方法
    查询当前环境位置,试用 source activate 进行激活就可以解决问题。

    #查询环境位置
    conda info | grep -i 'base environment' 
    base environment : /root/anaconda3  (writable)
    # /root/anaconda3 是你自己的输出结果
    source /root/anaconda3/etc/profile.d/conda.sh 
    #激活sh
    conda activate paddle_env 
    #可以运行了
    conda deactivate
    #退出虚拟环境
    conda env remove -n paddle_env
    #删除虚拟环境paddle_env 
    
    image.png
    • 以上anaconda环境和python环境安装完毕

    第3步:安装程序所需要库

    • 使用pip命令在刚激活的环境中安装paddle:
    # 在命令行中输入以下命令:
    # 默认安装CPU版本,安装paddle时建议使用百度源
    pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
    
    (paddle_env) [root@test-web2 ~]# pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
    ... ...
    Installing collected packages: sentencepiece, pytz, pydub, paddle2onnx, jieba, ffmpy, easydict, xxhash, websockets, urllib3, tzdata, typing-extensions, tqdm, toolz, threadpoolctl, six, semantic-version, scipy, safetensors, rpds-py, rarfile, pyzmq, pyyaml, python-multipart, pyparsing, pygments, pycryptodome, pyarrow, psutil, protobuf, packaging, orjson, opencv-python, multidict, mdurl, markupsafe, kiwisolver, joblib, itsdangerous, importlib-resources, future, fsspec, frozenlist, fonttools, filelock, dill, cycler, contourpy, colorlog, colorama, click, charset-normalizer, blinker, Babel, attrs, async-timeout, anyio, annotated-types, aiofiles, yarl, Werkzeug, uvicorn, typer, starlette, scikit-learn, requests, referencing, python-dateutil, pydantic-core, onnx, multiprocess, markdown-it-py, Jinja2, gunicorn, bce-python-sdk, aiosignal, seqeval, rich, pydantic, pandas, paddlefsl, matplotlib, jsonschema-specifications, huggingface-hub, flask, aiohttp, jsonschema, gradio-client, Flask-Babel, fastapi, visualdl, datasets, altair, paddlenlp, gradio, paddlehub
      Attempting uninstall: protobuf
        Found existing installation: protobuf 4.24.3
        Uninstalling protobuf-4.24.3:
          Successfully uninstalled protobuf-4.24.3
      Attempting uninstall: anyio
        Found existing installation: anyio 4.0.0
        Uninstalling anyio-4.0.0:
          Successfully uninstalled anyio-4.0.0
    Successfully installed Babel-2.12.1 Flask-Babel-3.1.0 Jinja2-3.1.2 Werkzeug-2.3.7 aiofiles-23.2.1 aiohttp-3.8.5 aiosignal-1.3.1 altair-5.1.1 annotated-types-0.5.0 anyio-3.7.1 async-timeout-4.0.3 attrs-23.1.0 bce-python-sdk-0.8.90 blinker-1.6.2 charset-normalizer-3.2.0 click-8.1.7 colorama-0.4.6 colorlog-6.7.0 contourpy-1.1.1 cycler-0.11.0 datasets-2.14.5 dill-0.3.4 easydict-1.10 fastapi-0.103.1 ffmpy-0.3.1 filelock-3.12.4 flask-2.3.3 fonttools-4.42.1 frozenlist-1.4.0 fsspec-2023.6.0 future-0.18.3 gradio-3.45.0 gradio-client-0.5.2 gunicorn-21.2.0 huggingface-hub-0.17.3 importlib-resources-6.1.0 itsdangerous-2.1.2 jieba-0.42.1 joblib-1.3.2 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 kiwisolver-1.4.5 markdown-it-py-3.0.0 markupsafe-2.1.3 matplotlib-3.8.0 mdurl-0.1.1 multidict-6.0.4 multiprocess-0.70.12.2 onnx-1.14.1 opencv-python-4.8.0.76 orjson-3.9.7 packaging-23.1 paddle2onnx-1.0.9 paddlefsl-1.1.0 paddlehub-2.4.0 paddlenlp-2.6.1 pandas-2.1.1 protobuf-3.20.2 psutil-5.9.5 pyarrow-13.0.0 pycryptodome-3.19.0 pydantic-2.4.1 pydantic-core-2.10.1 pydub-0.25.1 pygments-2.16.1 pyparsing-3.1.1 python-dateutil-2.8.2 python-multipart-0.0.6 pytz-2023.3.post1 pyyaml-6.0.1 pyzmq-25.1.1 rarfile-4.1 referencing-0.30.2 requests-2.31.0 rich-13.5.3 rpds-py-0.10.3 safetensors-0.3.3.post1 scikit-learn-1.3.1 scipy-1.11.2 semantic-version-2.10.0 sentencepiece-0.1.99 seqeval-1.2.2 six-1.16.0 starlette-0.27.0 threadpoolctl-3.2.0 toolz-0.12.0 tqdm-4.66.1 typer-0.9.0 typing-extensions-4.8.0 tzdata-2023.3 urllib3-2.0.5 uvicorn-0.23.2 visualdl-2.5.3 websockets-11.0.3 xxhash-3.3.0 yarl-1.9.2
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
    (paddle_env) [root@test-web2 ~]#
    

    paddlepaddle-2.5.1

    • 安装完paddle后,继续在paddle_env环境中安装paddlehub:
    # 在命令行中输入以下命令:
    pip install paddlehub -i https://mirror.baidu.com/pypi/simple
    (paddle_env) [root@test-web2 ~]# pip install paddlehub -i https://mirror.baidu.com/pypi/simple
    ... ...
    Successfully installed Babel-2.12.1 Flask-Babel-3.1.0 Jinja2-3.1.2 Werkzeug-2.3.7 aiofiles-23.2.1 aiohttp-3.8.5 aiosignal-1.3.1 altair-5.1.1 annotated-types-0.5.0 anyio-3.7.1 async-timeout-4.0.3 attrs-23.1.0 bce-python-sdk-0.8.90 blinker-1.6.2 charset-normalizer-3.2.0 click-8.1.7 colorama-0.4.6 colorlog-6.7.0 contourpy-1.1.1 cycler-0.11.0 datasets-2.14.5 dill-0.3.4 easydict-1.10 fastapi-0.103.1 ffmpy-0.3.1 filelock-3.12.4 flask-2.3.3 fonttools-4.42.1 frozenlist-1.4.0 fsspec-2023.6.0 future-0.18.3 gradio-3.44.4 gradio-client-0.5.1 gunicorn-21.2.0 huggingface-hub-0.17.2 importlib-resources-6.1.0 itsdangerous-2.1.2 jieba-0.42.1 joblib-1.3.2 jsonschema-4.19.1 jsonschema-specifications-2023.7.1 kiwisolver-1.4.5 markdown-it-py-3.0.0 markupsafe-2.1.3 matplotlib-3.8.0 mdurl-0.1.1 multidict-6.0.4 multiprocess-0.70.12.2 onnx-1.14.1 opencv-python-4.8.0.76 orjson-3.9.7 packaging-23.1 paddle2onnx-1.0.9 paddlefsl-1.1.0 paddlehub-2.4.0 paddlenlp-2.6.1 pandas-2.1.1 protobuf-3.20.2 psutil-5.9.5 pyarrow-13.0.0 pycryptodome-3.19.0 pydantic-2.3.0 pydantic-core-2.6.3 pydub-0.25.1 pygments-2.16.1 pyparsing-3.1.1 python-dateutil-2.8.2 python-multipart-0.0.6 pytz-2023.3.post1 pyyaml-6.0.1 pyzmq-25.1.1 rarfile-4.1 referencing-0.30.2 requests-2.31.0 rich-13.5.3 rpds-py-0.10.3 safetensors-0.3.3.post1 scikit-learn-1.3.1 scipy-1.11.2 semantic-version-2.10.0 sentencepiece-0.1.99 seqeval-1.2.2 six-1.16.0 starlette-0.27.0 threadpoolctl-3.2.0 toolz-0.12.0 tqdm-4.66.1 typer-0.9.0 typing-extensions-4.8.0 tzdata-2023.3 urllib3-2.0.5 uvicorn-0.23.2 visualdl-2.5.3 websockets-11.0.3 xxhash-3.3.0 yarl-1.9.2
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
    (paddle_env) [root@test-web2 ~]# 
    
    
    • 安装paddlehub时会自动安装其它依赖库,可能需要花费一段时间
      paddlehub-2.4.0

    chinese_ocr_db_crnn_mobile模型依赖:

    • paddlepaddle >= 1.7.2
    • paddlehub >= 1.6.0
    • shapely
    • pyclipper
    • 该Module依赖于第三方库shapely和pyclipper,使用该Module之前,请先安装shapely和pyclipper。
    (paddle_env) [root@test-web2 ~]# pip install shapely pyclipper -i https://mirror.baidu.com/pypi/simple
    Looking in indexes: https://mirror.baidu.com/pypi/simple
    Collecting shapely
      Downloading https://mirror.baidu.com/pypi/packages/a8/a5/403728b5614b28083f6424dfdefec5fcf58068495fb03bb08532671c642f/shapely-2.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 7.6 MB/s eta 0:00:00
    Collecting pyclipper
      Downloading https://mirror.baidu.com/pypi/packages/1c/81/4aa8403e587a4c60e00b479c11254a6e3200f3b985dcf4caecf0d8c21261/pyclipper-1.3.0.post5-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (908 kB)
         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 908.3/908.3 kB 4.4 MB/s eta 0:00:00
    Requirement already satisfied: numpy>=1.14 in ./anaconda3/envs/paddle_env/lib/python3.10/site-packages (from shapely) (1.26.0)
    Installing collected packages: pyclipper, shapely
    Successfully installed pyclipper-1.3.0.post5 shapely-2.0.1
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
    

    安装chinese_ocr_db_crnn_mobile模型

    (paddle_env) [root@test-web2 ~]# hub install chinese_ocr_db_crnn_mobile -i https://mirror.baidu.com/pypi/simple
    

    报错:

    (paddle_env) [root@test-web2 ~]# hub install chinese_ocr_db_crnn_mobile -i https://mirror.baidu.com/pypi/simple
    Error: Can not import paddle core while this file exists: /root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/libpaddle.so
    Traceback (most recent call last):
      File "/root/anaconda3/envs/paddle_env/bin/hub", line 5, in <module>
        from paddlehub.commands.utils import execute
      File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddlehub/__init__.py", line 18, in <module>
        import paddle
      File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/__init__.py", line 31, in <module>
        from .framework import monkey_patch_variable
      File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/framework/__init__.py", line 17, in <module>
        from . import random  # noqa: F401
      File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/framework/random.py", line 17, in <module>
        from paddle import fluid
      File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/__init__.py", line 36, in <module>
        from . import framework
      File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/framework.py", line 35, in <module>
        from . import core
      File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/core.py", line 356, in <module>
        raise e
      File "/root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/paddle/fluid/core.py", line 269, in <module>
        from . import libpaddle
    ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory
    

    解决办法:
    Centos7 默认提供的 openssl 版本是 1.0.2 的,如果过低的话就会报错 libssl.so.1.1: cannot open shared object file: No such file or directory,这是由于 openssl 库的位置不正确或未安装 1.1 以上版本的问题;

    想要升级 openssl 版本则需要手动进行编译,报错解决方式很简单,安装正确即可

    #下载 openssl
    wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz
    或者
    wget https://mirrors.cloud.tencent.com/openssl/source/openssl-1.1.1g.tar.gz
    #编译 openssl
    tar -xvf openssl-1.1.1g.tar.gz
    cd openssl-1.1.1g
    ./config shared --openssldir=/usr/local/openssl --prefix=/usr/local/openssl
    make && make install
    

    编译完成后,使用 openssl version 来查看一下当前 openssl 版本号时,你会发现还是 1.0.2,所以这里需要做一些额外的配置工作

    #配置
    mv /usr/bin/openssl  /usr/bin/openssl.old
    ln -s /usr/local/openssl/bin/openssl /usr/bin/openssl
    [root@test-web2 ~]# find / -name libcrypto.so.1.1
    /root/openssl-1.1.1g/libcrypto.so.1.1
    /usr/local/openssl/lib/libcrypto.so.1.1
    
    [root@test-web2 ~]# find / -name libssl.so.1.1
    /root/openssl-1.1.1g/libssl.so.1.1
    /usr/local/openssl/lib/libssl.so.1.1
    [root@test-web2 ~]# ln -s /usr/local/openssl/lib/libssl.so.1.1 /usr/lib64/libssl.so.1.1
    [root@test-web2 ~]# ln -s /usr/local/openssl/lib/libcrypto.so.1.1  /usr/lib64/libcrypto.so.1.1
    
    ldconfig -v
    #退出paddle_env虚拟环境
    conda deactivate
    #重新激活paddle_env
    conda activate paddle_env
    #查看openssl版本
    (base) [root@test-web2 ~]# conda info --envs
    # conda environments:
    #
    base                  *  /root/anaconda3
    
    (base) [root@test-web2 ~]# openssl version
    OpenSSL 1.1.1u  30 May 2023
    
    
    (paddle_env) [root@test-web2 ~]# hub install chinese_ocr_db_crnn_mobile -i https://mirror.baidu.com/pypi/simple
    /root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
      warnings.warn("Setuptools is replacing distutils.")
    usage: hub install [-h] [--ignore_env_mismatch]
    hub install: error: unrecognized arguments: -i
    (paddle_env) [root@test-web2 ~]# hub install chinese_ocr_db_crnn_mobile
    /root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
      warnings.warn("Setuptools is replacing distutils.")
    Download https://bj.bcebos.com/paddlehub/paddlehub_dev/chinese_ocr_db_crnn_mobile_1.2.0.zip
    [##################################################] 100.00%
    Decompress /root/.paddlehub/tmp/tmpphgh428g/chinese_ocr_db_crnn_mobile_1.2.0.zip
    [##################################################] 100.00%
    [2023-09-27 11:43:36,803] [    INFO] - Successfully installed chinese_ocr_db_crnn_mobile-1.2.0
    
    (paddle_env) [root@test-web2 ~]# hub serving start -m chinese_ocr_db_crnn_mobile
    /root/anaconda3/envs/paddle_env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
      warnings.warn("Setuptools is replacing distutils.")
     * Serving Flask app 'paddlehub.serving.app_compat'
     * Debug mode: off
    [2023-09-27 11:46:09,970] [    INFO] _internal.py:187 - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
     * Running on all addresses (0.0.0.0)
     * Running on http://127.0.0.1:8866
     * Running on http://192.168.100.251:8866
    [2023-09-27 11:46:09,970] [    INFO] _internal.py:187 - Press CTRL+C to quit
    
    
    # 直接启动
    hub serving start -m chinese_ocr_db_crnn_server -p 8866 --use_multiprocess --workers=3
    # 后台启动
    nohup hub serving start -m chinese_ocr_db_crnn_server -p 8866 --use_multiprocess --workers=3 &
    #--use_multiprocess 代表启动多线程
    #--workers 代表线程数 一般为2*cpu-1
    #关闭服务
    hub serving stop -p 8866
    

    chinese_ocr_db_crnn_mobile-1.2.0

    另外:

    ImportError: libSM.so.6: cannot open shared object file: No such file or directory
    ImportError: libXrender.so.1: cannot open shared object file: No such file or directory
    原因:
    报错的主要原因是:缺少共享库;
    解决办法
    使用如下命令查看缺少的共享库

    (base) [root@test-web2 ~]# yum whatprovides libSM.so.6
    已加载插件:fastestmirror, langpacks
    Loading mirror speeds from cached hostfile
     * base: mirrors.aliyun.com
     * elrepo: mirrors.tuna.tsinghua.edu.cn
     * epel: mirrors.bfsu.edu.cn
     * extras: mirrors.aliyun.com
     * updates: mirrors.aliyun.com
    libSM-1.2.2-2.el7.i686 : X.Org X11 SM runtime library
    源    :base
    匹配来源:
    提供    :libSM.so.6
    
    yum install libSM-1.2.2-2.el7
    已加载插件:fastestmirror, langpacks
    base                                                                                                                                  | 3.6 kB  00:00:00     
    elrepo                                                                                                                                | 3.0 kB  00:00:00     
    epel/x86_64/metalink                                                                                                                  | 6.1 kB  00:00:00     
    epel                                                                                                                                  | 4.7 kB  00:00:00     
    extras                                                                                                                                | 2.9 kB  00:00:00     
    updates                                                                                                                               | 2.9 kB  00:00:00     
    elrepo/primary_db                                                                                                                     | 357 kB  00:00:00     
    Loading mirror speeds from cached hostfile
     * base: mirrors.aliyun.com
     * elrepo: mirrors.tuna.tsinghua.edu.cn
     * epel: mirrors.bfsu.edu.cn
     * extras: mirrors.aliyun.com
     * updates: mirrors.aliyun.com
    软件包 libSM-1.2.2-2.el7.x86_64 已安装并且是最新版本
    如果发现不能安装 则使用下边的语句
    yum install libSM-1.2.2-2.el7.x86_64 --setopt=protected_multilib=false
    

    可能遇到的问题的参考办法:
    https://www.cnblogs.com/newgold/p/17340518.html
    详细的安装文档
    https://blog.csdn.net/weixin_38997187/article/details/121680809
    https://blog.csdn.net/sika_deer/article/details/120631075

    相关文章

      网友评论

          本文标题:paddlepaddle--paddlehub--chinese

          本文链接:https://www.haomeiwen.com/subject/pyvfbdtx.html