Hadoop编译2018-05-15

1、安装JDK

1.创建文件夹

mkdir  -p /usr/java

cd /usr/java

2.rz上传jdk文件

3.解压缩文件

tar -xzvf jdk-8u45-linux-x64.gz

4.修改权限

chown -R root：root jdk1.8.0_45

5.修改环境变量并生效

vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_45

export PATH=$JAVA_HOME/bin:$PATH

source    /etc/profile

6、测试

java    -version

java version "1.8.0_45"

Java(TM) SE Runtime Environment (build 1.8.0_45-b14)

Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

2、maven安装

1、新建文件夹

[root@hadoop000 ~]# mkdir -p /opt/sourcecode /opt/software

2、上传安装包

cd    /opt/software

rz        apache-maven-3.3.9-bin.zip

3、解压缩文件

unzip  apache-maven-3.3.9-bin.zip

4、配置环境变量并生效

vi /etc/profile

export MAVEN_HOME=/opt/software/apache-maven-3.3.9

export MAVEN_OPTS="-Xms256m -Xmx512m"

export PATH=$MAVEN_HOME/bin:$JAVA_HOME/bin:$PATH

source     /etc/profile

5、测试

mvn     -version

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)

Maven home: /opt/software/apache-maven-3.3.9

Java version: 1.8.0_45, vendor: Oracle Corporation

Java home: /usr/java/jdk1.8.0_45/jre

Default locale: en_US, platform encoding: UTF-8

OS name: "linux", version: "2.6.32-431.el6.x86_64", arch: "amd64", family: "unix"

3、protobuf安装

1、rz protobuf-2.5.0.tar.gz

2、解压缩

tar -xzvf protobuf-2.5.0.tar.gz

3、修改权限

chown    -R root：root     protobuf-2.5.0

4、安装依赖

cd    protobuf-2.5.0

[root@hadoop000 protobuf-2.5.0]# yum install -y gcc gcc-c++ make cmake        

[root@hadoop000 protobuf-2.5.0]# ./configure --prefix=/usr/local/protobuf            编译的结果存放路径

[root@hadoop000 protobuf-2.5.0]# make && make install

5、配置环境变量并生效

vi     /etc/profile

export PROTOC_HOME=/usr/local/protobuf

export PATH=$PROTOC_HOME/bin:

source     /etc/profile

6、测试

[root@hadoop000 protobuf-2.5.0]# protoc --version

libprotoc 2.5.0

4、Findbugs安装

1、rz    findbugs-1.3.9.zip

2、解压    unzip    findbugs-1.3.9.zip

3、配置环境变量及生效

vi     /etc/profile

export FINDBUGS_HOME=/opt/software/findbugs-1.3.9

export PATH=$FINDBUGS_HOME/bin

source     /etc/profile

4、测试

[root@hadoop000 software]# findbugs -version

1.3.9

5、其他依赖

yum install -y openssl openssl-devel svn ncurses-devel zlib-devel libtool

yum install -y snappy snappy-devel bzip2 bzip2-devel lzo lzo-devel lzop autoconf automake

6、编译

cd     /opt/sourcecode

rz    hadoop-2.8.1-src.tar.gz

tar     -xzvf hadoop-2.8.1-src.tar.gz

cd     hadoop-2.8.1-src

[root@hadoop000 hadoop-2.8.1-src]# mvn clean package -Pdist,native -DskipTests -Dtar

死机两次还是用了老师给的.m2包

rz至root家目录后 再重新编译

[INFO] Apache Hadoop Azure support ........................ SUCCESS [ 5.250 s]

[INFO] Apache Hadoop Client ............................... SUCCESS [ 9.194 s]

[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 2.700 s]

[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 21.443 s]

[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 8.359 s]

[INFO] Apache Hadoop Azure Data Lake support .............. SUCCESS [ 8.433 s]

[INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.228 s]

[INFO] Apache Hadoop Distribution ......................... SUCCESS [ 51.624 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------

[INFO] Total time: 17:29 min [INFO] Finished at: 2018-05-15T22:21:41+08:00 [INFO] Final Memory: 187M/494M

[INFO] --------------------------------------------------------------------