0x00 缘由
由于我们从Hadoop的Apache网站上下载的hadoop包是在32位机器上编译的。因此，如果我们的机器的64位的，在使用的时候就会出现问题。所以，我们要在64位机器上重新编译hadoop。
0x01 准备
OS: CentOS6 64位 Hadoop版本：2.6.0
Hadoop2.6.0源码下载地址：
http://archive.apache.org/dist/hadoop/common/hadoop-2.6.0/hadoop-2.6.0-src.tar.gz
0x02 编译前的准备工作
参考BUILDING.txt
hadoop2.6.0目录下，有一个BUILDING.txt文件，这是编译的说明。
Build instructions for Hadoop
------------------------------------------------------------------
Requirements:
* Unix System
* JDK 1.6+
* Maven 3.0 or later
* Findbugs 1.3.9 (if running findbugs)
* ProtocolBuffer 2.5.0
* CMake 2.6 or newer (if compiling native code)
* Zlib devel (if compiling native code)
* openssl devel ( if compiling native hadoop-pipes )
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
------------------------------------------------------------------
把JDK、Maven、ProtocolBuffer2.5.0、Cmake、zlib、openssl-devel先安装好
0x03 编译
依然参考BUILDING.txt
------------------------------------------------------------------
Building distributions:
Create binary distribution without native code and without documentation:
$ mvn package -Pdist -DskipTests -Dtar
Create binary distribution with native code and with documentation:
$ mvn package -Pdist,native,docs -DskipTests -Dtar
Create source distribution:
$ mvn package -Psrc -DskipTests
Create source and binary distributions with native code and documentation:
$ mvn package -Pdist,native,docs,src -DskipTests -Dtar
Create a local staging version of the website (in /tmp/hadoop-site)
$ mvn clean site; mvn site:stage -DstagingDirectory=/tmp/hadoop-site
------------------------------------------------------------------
进入Hadoop2.6.0目录下：
这里我们使用： mvn package -Pdist,native,docs,src -DskipTests -Dtar来编译
注：为了防止在编译的时候出现内存溢出的错误，我们需要手动指定一下maven使用内存的大小
Handling out of memory errors in builds
------------------------------------------------------------------
If the build process fails with an out of memory error, you should be able to fix
it by increasing the memory used by maven -which can be done via the environment
variable MAVEN_OPTS.
Here is an example setting to allocate between 256 and 512 MB of heap space to
Maven
export MAVEN_OPTS="-Xms256m -Xmx512m"
------------------------------------------------------------------
编译顺利的话，一个小时左右，就能完成编译。
编译好生成的hadoop文件在这个目录下：hadoop-2.6.0-src/hadoop-dist/target/
有一个文件：hadoop-2.6.0.tar.gz
就是我们编译好的hadoop2.6.0
0x04 编译时遇到的几个错误
-错误1：
Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-annotations: MavenReportException: Error while creating archive:
[ERROR] Exit code: 1 - /opt/hadoop-2.6.0-src/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java:27: error: unexpected end tag:
解决办法：在编译命令后面加个Dmaven.javadoc.skip=true的参数即可
mvn clean package -Pdist,native,docs,src -DskipTests -Dtar -Dmaven.javadoc.skip=true
-错误2：
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (site) on project hadoop-common: An Ant BuildException has occured: input file /opt/hadoop-2.6.0-src/hadoop-common-project/hadoop-common/target/findbugsXml.xml does not exist
[ERROR] around Ant part ...... @ 44:234 in /opt/hadoop-2.6.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
解决办法：去掉编译命令中的docs参数
mvn clean package -Pdist,native,src -DskipTests -Dtar -Dmaven.javadoc.skip=true
解决了这两个报错，编译应该就没有什么问题了。
我自己编译的时候遇到了这两个报错，编译hadoop2.5.2的方法同上！
hadoop-build-success
可以看到，编译完成用了20多分钟。不同的机器配置，可能耗费的时间会有所不同。
不足之处，请批评指正。
如有问题，请私信联系。
谢谢！
网友评论