版本更新说明
v1.3 20170630
- 增加Matlab2012b、Matlab2014b、Matlab2016b菜单启动快捷方式
- 安装PyCharm专业版并附有激活服务器,菜单栏可启动
- 激活服务器可激活Jetbrain公司旗下所有所有专业产品
v1.2 20170523
增加Matlab2012,Matlab2014支持,现在已经安装Matlab2012、2014、2016三个版本v1.1 20170511
增加Torch环境的支持v1.0 20170509
主要内容有新用户必读,基础设置,caffe的搭建过程,TensorFlow搭建等
前段时间在机房服务器搭建了基础深度学习平台,搭建全过程见CentOS 7.3 Caffe搭建全过程记录
很多环境与软件已经安装完毕,ssh登录后就会显示相应的信息,如下图所示
系统信息使用之前的几点重要说明:
用户权限问题
为了系统正常运行和其他用户环境的稳定性,不提供root权限给普通用户,如果你需要root权限安装系统级别软件,请联系我或者自己再搭建一台服务器
支持的环境
目前可以为Caffe和TensorFlow学习框架提供支持,两个框架均可以在不需要root权限条件下编译成功并运行,这当然要讲究点技巧和使用规范,下面会有说明。更多框架的支持请留意文章开头的更新说明
软件版本问题
Python目前有2.7.13(默认)和3.6.1两种版本存在,视个人需求而定,这个在创建python虚拟环境是有了更多的选择。Matlab为2016b,只有一个版本,敬请谅解,后期可能会增加2014版,并解决版本冲突问题。
Xmanager可用
大部分同学都需要使用图形话界面。服务器已经配置好,在windows端可直接连接。
关于环境的说明
初始环境已经配置好了PIP的镜像及virtualenvwrapper
,在你的桌面上已经放好了最新的Caffe,接下来就要编译它。
如果使用到了python环境,在开始之前你需要移步Python Virtual Environments虚拟环境导引,找另外一台练习机学会以后再往下读,这点很重要,请务必学会,请务必学会,请务必学会
好,假设你已经会使用python的virtual env 了,那么请往下看
用户以simon
为例,已经创建好
连接到服务器并配置基本环境
在windows端,打开xshell(推荐你使用)或XManager连接服务器后打开shell
登录后会显示系统信息,并要求你更改密码,这个密码必须比较复杂才能被设置。密码更改成功后会强制退出,再次使用ssh登录即可。
显示显示系统信息,紧接着,virtualenvwrapper
初始化
- 配置上网代理
首先我们需要在windows端使用ccproxy这个软件搭建起代理服务器,记住代理服务器的地址和端口号,别忘记设置客户端访问权限。接下来设置远程服务器的代理
vi ~/.bashrc
#修改export http_proxy https_proxy ftp_proxy 为你自己的代理服务器地址和端口
- 创建virtualenvironment ,名字为caffe-master
mkvirtualenv caffe-master # 默认使用2.7版本
好了,我们已经在虚拟环境中了,如下图所示。
image.pngPyCharm的使用
管理员已经为所有用户安装了最新的PyCharm专业版(2017.1.4),可从开始菜单快捷启动
要求激活时,选择服务器激活
点击激活即可
需要注意的是,激活程序切忌不可商用盈利
实验室的同学可使用地址
http://192.168.69.164:9714
http://192.168.69.68:9714
激活服务器可激活Jetbrain公司旗下的所有收费产品,包括PyCharm、PhpStorm、Intelj Idea、WebStorm等,其他产品及详细信息https://www.jetbrains.com/products.html
激活服务器由Lanyu发布,由于涉及到版权及商业利益,暂不公开发布,如果你需要的话,请私信我。
PyCharm的配置过程有兴趣的同学请移步本人博客CentOS 7 为所有用户安装PyCharm并激活
编译caffe
如果你有使用caffe的需求,请参照一下步骤,否则请跳过
解压并修改caffe
cd ~/Desktop
unzip caffe-master.zip
如果你的虚拟环境的python版本是2.7,请执行下面的操作
cd caffe-master/python
vi requirements.txt
#修改ipython>=3.0.0 为 ipython==5.3.0
安装python依赖
cd caffe-master/python #进入python目录
pip install -r requirements.txt
修改Makefile.config文件
cd caffe-master
cp Makefile.config.example Makefile.config
vi Makefile.config
#修改内容如下
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1 ##使用CUDNN
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3 ##指定opencv的版本
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
##删除多余的架构
CUDA_ARCH := -gencode arch=compute_30,code=sm_30
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_52,code=sm_52 \
-gencode arch=compute_60,code=sm_60 \
-gencode arch=compute_61,code=sm_61 \
-gencode arch=compute_61,code=compute_61
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := open ##使用open-blas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /usr/include/openblas ##指定blas的头文件路径
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
MATLAB_DIR := /usr/local/MATLAB/R2016b ## Matlab的路径
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 /usr/lib/python2.7/dist-packages/numpy/core/include
## 修改为你的自己的路径,基本上就是用户名那里变一下
PYTHON_INCLUDE := /home/simon/Envs/caffe-master/include/python2.7 /home/simon/Envs/caffe-master/lib/python2.7/site-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
# $(ANACONDA_HOME)/include/python2.7 \
# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include
# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1 ## 打开pyhton层支持
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
为protoc打补丁
cd caffe-master
protoc src/caffe/proto/caffe.proto --cpp_out=.
mkdir include/caffe/proto
mv src/caffe/proto/caffe.pb.h include/caffe/proto
编译
make -j24
make pycaffe
make matcaffe
make test -j24 #可选操作
make runtest -j8 #可选操作
成功后如下图所示
验证
cd caffe-master/python
ipython
import caffe #如果没问题就成功了
验证caffe
安装TensorFlow(教程太老,已废弃)
TensorFlow是作为python的第三方包发布的,利用包管理工具pip
即可完成安装,只是需要注意版本的选择就可以了。移步CentOS 7 搭建Tensorflow环境 获取更详细的信息
以TensorFlow r0.12版本为例进行说明
去官网查询安装地址
- 官网
-
选择版本号
选择版本号 -
找到地址
就是图中的地址啦
安装
注意确保你在虚拟环境中
pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.0rc1-cp27-none-linux_x86_64.whl
验证
ipython
import tensorflow
安装Torch
一定要确保你在自己创建的Python虚拟环境中,并且已经安装了ipython。
安装
# in a terminal, run the commands WITHOUT sudo
git clone https://github.com/torch/distro.git ~/torch --recursive
cd ~/torch;
./install.sh
source ~/.bashrc
官网网站的指导中还有bash install-deps;
,这句命令的作用是安装Torch7
所需要的依赖,由于管理员已经安装过,所以,普通用户就不用安装了
验证
(simon_main) [zhangxinming@localhost torch]$ th
______ __ | Torch7
/_ __/__ ________/ / | Scientific computing for Lua.
/ / / _ \/ __/ __/ _ \ | Type ? for help
/_/ \___/_/ \__/_//_/ | https://github.com/torch
| http://torch.ch
th> torch.Tensor{1,2,3}
1
2
3
[torch.DoubleTensor of size 3]
th>
更多Torch的用法及知识请移步官方教程
Matlab的使用
启动Matlab非常简单,在图形环境中打开终端,输入
matlab #默认启动2016b版本
matlab2012 #启动2012版本
matlab2014 #启动2014版本
matlab2016 #启动2016版本
v1.3版本后,增加了菜单栏启动的方式
image.png
网友评论