美文网首页我的Python自学之路我爱编程
使用pypiserver快速搭建内网离线pypi仓库实践

使用pypiserver快速搭建内网离线pypi仓库实践

作者: 王奥OX | 来源:发表于2018-04-13 15:48 被阅读26次

    前言

    本文介绍了如何快速搭建一个pypiserver,通过自建pypiserver,我们可以解决网络环境不好,或者离线无法安装python包的问题。如果结合最新的GitLab CI/CD和pipenv我相信各位还可以玩出更多的花样。

    pypiserver - minimal PyPI server for use with pip/easy_install

    更新记录

    2018年04月12日 - 初稿

    阅读原文 - https://wsgzao.github.io/post/pypiserver/

    扩展阅读

    pypiserver - https://github.com/pypiserver/pypiserver

    pypiserver 简介

    pypiserver is a minimal PyPI compatible server for pip or easy_install. It is based on bottle and serves packages from regular directories. Wheels, bdists, eggs and accompanying PGP-signatures can be uploaded either with pip, setuptools, twine, pypi-uploader, or simply copied with scp.

    pypiserver 服务端配置

    如果你的Linux环境缺少Python 2.7可以参考我的文章直接离线升级至最新版本

    Python 2.6 升级至 Python 2.7 的实践心得
    https://wsgzao.github.io/post/python-2-6-update-to-2-7/

    pypiserver > 1.2.x works with python 2.7 and 3.3+ or pypy. Older python-versions may still work, but they are not tested. For legacy python versions, use pypiserver-1.1.x series.

    # 替换pip为阿里云,感概豆瓣的时代已经过去
    tee ~/.pip/pip.conf <<-'EOF'
    [global]
    index-url = https://mirrors.aliyun.com/pypi/simple/
    [install]
    trusted-host= mirrors.aliyun.com
    EOF
    
    # 直接在线安装pypiserver
    pip install pypiserver
    # 离线下载pypiserver
    mkdir /tmp/pypiserver
    pip install -d /tmp/pypiserver/ pypiserver
    
    # Copy packages into this directory.
    mkdir ~/packages
    # Copy some packages into your ~/packages folder and then get your pypiserver up and running:
    pypi-server -p 8080 ~/packages &
    
    

    pypiserver 客户端配置

    ## Download and Install hosted packages.
    pip install  --extra-index-url http://localhost:8080/simple/ ...
    
    # or
    pip install --extra-index-url http://localhost:8080
    
    ## Search hosted packages
    pip search --index http://localhost:8080 ...
    
    # 个人推荐的配置
    tee ~/.pip/pip.conf <<-'EOF'
    [global]
    index-url = http://172.28.70.126/simple
    extra-index-url = https://mirrors.aliyun.com/pypi/simple/
    [install]
    trusted-host = 172.28.70.126
    EOF
    
    

    pypiserver 进阶配置

    pypiserver Running as a systemd service

    https://github.com/pypiserver/pypiserver#running-as-a-systemd-service

    Adjusting the paths and adding this file as pypiserver.service into your systemd/system directory will allow management of the pypiserver process with systemctl, e.g. systemctl start pypiserver.

    More useful information about systemd can be found at https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units

    # 安装需要的包
    yum install nginx -y
    pip install passlib pypiserver gunicorn
    
    # 创建pypiserver服务方便服务启停管理
    tee /usr/lib/systemd/system/pypiserver.service <<-'EOF'
    [Unit]
    Description=gunicorn daemon
    After=network.target
    
    [Service]
    PIDFile=/run/pypiserver.pid
    ExecStart=/usr/local/bin/gunicorn -w16 \
        --pid /run/pypiserver.pid \
        -b :10012 \
        'pypiserver:app(root="/var/www/pypi")'
    ExecReload=/bin/kill -s HUP $MAINPID
    ExecStop=/bin/kill -s TERM $MAINPID
    
    [Install]
    WantedBy=multi-user.target 
    EOF
    
    # Warning: pypiserver.service changed on disk. Run 'systemctl daemon-reload' to reload units.
    systemctl daemon-reload
    
    # 启动pypiserver服务
    systemctl enable pypiserver.service
    systemctl start pypiserver.service
    systemctl status pypiserver.service
    
    # 停止pypiserver服务
    systemctl disable pypiserver.service
    systemctl stop pypiserver.service
    systemctl status pypiserver.service
    
    [root@centos7 run]# systemctl status pypiserver.service
    ● pypiserver.service - gunicorn daemon
       Loaded: loaded (/usr/lib/systemd/system/pypiserver.service; disabled; vendor preset: disabled)
       Active: active (running) since Fri 2018-04-13 15:14:08 CST; 859ms ago
     Main PID: 22524 (gunicorn)
       CGroup: /system.slice/pypiserver.service
               ├─22524 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22530 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22531 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22532 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22533 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22534 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22535 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22536 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22537 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22538 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               ├─22539 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
               └─22540 /usr/local/bin/python /usr/local/bin/gunicorn -w16 --pid /run/pypiserver.pid -b :10012 pypiserver:app(root="/var/www/pypi")
    
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22531] [INFO] Booting worker with pid: 22531
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22532] [INFO] Booting worker with pid: 22532
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22533] [INFO] Booting worker with pid: 22533
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22534] [INFO] Booting worker with pid: 22534
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22535] [INFO] Booting worker with pid: 22535
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22536] [INFO] Booting worker with pid: 22536
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22537] [INFO] Booting worker with pid: 22537
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22538] [INFO] Booting worker with pid: 22538
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22539] [INFO] Booting worker with pid: 22539
    Apr 13 15:14:08 centos7 gunicorn[22524]: [2018-04-13 15:14:08 +0000] [22540] [INFO] Booting worker with pid: 22540
    [root@centos7 run]# netstat -tnlp
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
    tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd           
    tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1517/dnsmasq        
    tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      977/sshd            
    tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      978/cupsd           
    tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1383/master         
    tcp        0      0 127.0.0.1:6011          0.0.0.0:*               LISTEN      19378/sshd: root@pt 
    tcp        0      0 0.0.0.0:10012           0.0.0.0:*               LISTEN      22524/python        
    tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd           
    tcp6       0      0 :::22                   :::*                    LISTEN      977/sshd            
    tcp6       0      0 ::1:631                 :::*                    LISTEN      978/cupsd           
    tcp6       0      0 ::1:25                  :::*                    LISTEN      1383/master         
    tcp6       0      0 ::1:6011                :::*                    LISTEN      19378/sshd: root@pt 
    
    # 检查pypiserver服务
    cd /var/www/pypi
    # 向仓库中添加python package
    [root@centos7 pypi]# pip download pypiserver
    Collecting pypiserver
      Downloading https://mirrors.aliyun.com/pypi/packages/d7/78/5772432dad2b9e754ab92f4d301fa507069b9decc8c943c1b18c2043ff4f/pypiserver-1.2.1-py2.py3-none-any.whl (83kB)
        100% |████████████████████████████████| 92kB 643kB/s 
      Saved ./pypiserver-1.2.1-py2.py3-none-any.whl
    Successfully downloaded pypiserver
    
    [root@centos7 pypi]# ll
    total 84
    -rw-r--r-- 1 root root 83529 Apr 13 14:55 pypiserver-1.2.1-py2.py3-none-any.whl
    
    # 搜索刚才下载的package
    [root@centos7 pypi]# pip search -i http://127.0.0.1:10012 pypiserver
    pypiserver (1.2.1)  - 1.2.1
      INSTALLED: 1.2.1 (latest)
    
    # 配置nginx做反向代理
    tee /etc/nginx/conf.d/pypi.conf <<-'EOF'
    upstream pypiserver {
      server 127.0.0.1:10012;
    }
    
    server {
      listen 10087;
    
      # disable any limits to avoid HTTP 413 for large package uploads
      client_max_body_size 0;
    
      location / {
        proxy_pass http://pypiserver/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    
        # When setting up pypiserver behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.
        proxy_set_header X-Forwarded-Proto $scheme;
    
        proxy_buffering off;
        proxy_request_buffering off;
      }
    
      location /packages/ {
        alias /var/www/pypi; # static file
      }
    }
    EOF
    
    # 启动nginx
    systemctl enable nginx
    systemctl start nginx
    systemctl status nginx
    
    # 检查nginx服务
    pip search -i http://172.28.79.126:10087 pypiserver
    pypiserver (1.2.1)  - 1.2.1
      INSTALLED: 1.2.1 (latest)
    
    

    相关文章

      网友评论

        本文标题:使用pypiserver快速搭建内网离线pypi仓库实践

        本文链接:https://www.haomeiwen.com/subject/kvockftx.html