美文网首页深度学习
Ubuntu 16.04下Anaconda编译安装Caffe

Ubuntu 16.04下Anaconda编译安装Caffe

作者: 逆风g | 来源:发表于2018-11-23 16:05 被阅读1次

    开篇先把我目前电脑上的环境说一下：

    1. 硬件
    • 戴尔工作站
    • 华硕1080ti 11G显存
    1. 软件
    • 已安装好Anoconda 2
    • 已安装好cuda
    • 已安装好cudnn
    • 已安装好TensorFlow、Pytorch，每个都单独有一个虚拟的Anoconda环境

    那么现在想要把caffe安装到一个新的Anoconda虚拟环境中去，每次训练不同平台下的网络时，只需要激活指定的虚拟环境。由于这里的caffe源码别人做了部分修改，运行他的代码，必须编译他修改后的caffe源码。

    安装依赖库

    进入官网http://caffe.berkeleyvision.org/installation.html，可以看到编译caffe需要的依赖库：


    首先进入自己的caffe虚拟环境source activate gcf_caffe,
    由于CUDA已经安装好了，直接跳过，安装各种依赖库：
    sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
    sudo apt-get install --no-install-recommends libboost-all-dev
    接着安装：
    sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
    sudo apt-get install libatlas-dev
    sudo apt-get install liblapack-dev
    sudo apt-get install libatlas-base-dev
    参考地址https://blog.csdn.net/u011974639/article/details/78804299

    假若安装时提示：无法定位软件包，代表需要更新软件源sudo apt-get update，更新软件源时若提示仓库“http://ppa.launchpad.net/fcitx-team/nightly/ubuntu xenial Release”没有Release文件。怎么解决呢？先切换到对应的ppa目录：cd /etc/apt/sources.list.d，再执行mv fcitx-team-ubuntu-nightly-xenial.list fcitx-team-ubuntu-nightly-xenial.list.bak，参考地址https://www.cnblogs.com/wenzheshen/p/6599636.html

    配置caffe的Makefile.config文件

    很多编译问题基本都是这里没有设置好，比如opencv冲突，protobuf冲突，这个文件需要仔细设置，实际内容也并没有多少。
    这里把我电脑上用到的具体配置贴出来：

    ## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#   You should not set this flag if you will be reading LMDBs with any
#   possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \
        -gencode arch=compute_35,code=sm_35 \
        -gencode arch=compute_50,code=sm_50 \
        -gencode arch=compute_52,code=sm_52 \
        -gencode arch=compute_60,code=sm_60 \
        -gencode arch=compute_61,code=sm_61 \
        -gencode arch=compute_61,code=compute_61

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /opt/OpenBLAS/include
# BLAS_LIB := /opt/OpenBLAS/lib

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
        # /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/anaconda2/envs/gcf_caffe
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
        $(ANACONDA_HOME)/include/python2.7 \
        $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
# PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib
LINKFLAGS := -Wl,-rpath,$(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

    注意点有二点：

    1. 我的依赖库都是在caffe虚拟环境gcf_caffe中安装的，这里ANACONDA_HOME := $(HOME)/anaconda2/envs/gcf_caffe
    2. 设置好hdf5的路径，INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
      LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial

    protoc冲突

    这里说一下非常容易遇到的protoc冲突，因为我已经安装了TensorFlow环境，/anaconda/bin/下存在一个3.5的protoc，系统/usr/bin/下也存在一个2.6的protoc，编译caffe时默认使用的是/anaconda/bin下的protoc，提示：error This file was generated by an older version of protoc which is，表示当前使用的protoc版本过高。

    • 安装protobuf的方法有好几种：
      sudo apt-get install libprotobuf-dev protobuf-compiler #Linux系统级的安装
      sudo pip install google protocol #python2.7版本的安装
      sudo pip3 install google protocol #python3.5版本的安装
      conda install protobuf #anaconda版本的安装
    • 查看系统中已安装的protobuf：
      whereis protoc #查看那些路径下安装了protobuf
      which protoc #查看默认选用的protobuf
      protoc --version #查看当前默认的protobuf的版本
      sudo protoc --version #查看系统的protobuf的版本
      参考地址https://blog.csdn.net/m0_38082419/article/details/80117132
      当前protoc情况：

      因为protoc版本过高，那么我只需要使用一个低版本的protoc就行了，修改caffe中的Makefile文件，把如下代码：
    $(Q)protoc --proto_path=$(PROTO_SRC_DIR) --cpp_out=$(PROTO_BUILD_DIR) $<
$(Q)protoc --proto_path=$(PROTO_SRC_DIR) --python_out=$(PY_PROTO_BUILD_DIR) $<

    修改成

    $(Q)/usr/bin/protoc --proto_path=$(PROTO_SRC_DIR) --cpp_out=$(PROTO_BUILD_DIR) $<
$(Q)/usr/bin/protoc --proto_path=$(PROTO_SRC_DIR) --python_out=$(PY_PROTO_BUILD_DIR) $<

    还有个问题，如何查看caffe需要的protoc版本呢？
    打开编译出错的文件caffe.pb.h，里面会有相关错误提示
    编译失败了，重新编译时，记得sudo make clean

    编译

    • sudo make all -j8
    • sudo make test -j4
    • make runtest -j4
    • sudo make pycaffe
    • import caffe
      若果提示ImportError: No module named caffe，需要把caffe下的Python路径导入环境变量中去。sudo vim ~/.bashrc，最后一行加上export PYTHONPATH="/home/ilab-gcf/桌面/CASENet_Codes/caffe/python:$PYTHONPATH"，这里的路径写上你自己的路径。否则的话只能在这个目录下执行Python，导入caffe了。

    报错提示can not find module skimage.io，安装scikit-image，conda install scikit-image

    总结

    这里都是废话，可以不用看。这个caffe编译浪费了我很多时间，遇到错误了就不想弄了，三天打鱼两天筛网，遇到问题还是喜欢逃避，很不好，路要一步步走。最近也有些人生感悟，我经常看不惯别人的习惯，自己还是不够成熟，很多事情站在别人角度来看，就不会觉的别人装逼了，觉得别人行为怪异，也很容易发现自己的那点努力并算不上什么，每个人的生活环境不一样，面对事情的处理方式也不一样，要学会理解。自己承受能力也不够强大，一点点不爽，就喜欢发说出来，别人语气不够友好，就容易怼回去。其实自己也不够强大，也喜欢装清高，比人有求于我时，容易装逼，又不想求别人，拉不下脸，容易自卑，想想各种推销人员，得向他们学习。给自己建议：

    1. 每天制定小目标，一步步脚踏实地。
    2. 多尝试换位思考。
    3. 加强承受能力。
    4. 不清高。

    相关文章

      网友评论

        本文标题：Ubuntu 16.04下Anaconda编译安装Caffe

        本文链接：https://www.haomeiwen.com/subject/ocjjqqtx.html

        延伸阅读

        深度阅读

        • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

        栏目导航

        热点阅读

        深度学习

        关于我们|服务条款|联系我们|Ubuntu 16.04下Anaconda编译安装Caffe|投稿指南|网站地图|RSS订阅|排版工具|手机版

        提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

        Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

        备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

        本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

        所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！