环境

系统环境	版本
操作系统	Ubuntu 18.04
Java	openjdk version "1.8.0_212"
Hadoop	3.2.0

1. 更新APT镜像源

添加apt阿里镜像源，加速资源的下载速度，参考官方文档

编辑/etc/apt/sources.list.d/aliyun.list

deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse

deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse

deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse

deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse

deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

执行更新

sudo apt update

2. 安装相关工具

Java JDK推荐使用jdk8，jdk9以上默认禁用了许多javax接口，会导致nodemanager和resourceManager启动失败。

sudo apt install -y openjdk-8-jdk

sudo apt install -y git
sudo apt install -y vim
sudo apt install -y curl
sudo apt install -y zsh
sudo apt install -y openssh-server

3. 安装Zsh（个人爱好）

sh -c "$(curl -fsSL https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"

chsh -s /bin/zsh

4. 新建Hadoop用户

sudo adduser hduser

5. 配置SSH权限（Hadoop用户）

su hduser

ssh-keygen -t rsa
cat ~/.sshid_rsa.pub >> ~/.sshauthorized_keys

测试

ssh localhost

6. 下载Hadoop3.2.0

其余版本链接参考清华大学开源软件镜像站

wget [https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/stable2/hadoop-3.2.0.tar.gz](https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/stable2/hadoop-3.2.0.tar.gz)

7. 修改Hadoop配置文件

为了方便，首先编辑~/.bashrc，配置用户环境变量

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

export HADOOP_HOME=/home/hduser/hadoop-family/hadoop-3.2.0
export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar
  
export PATH=${PATH}:${HADOOP_HOME}/bin/:${HADOOP_HOME}/sbin/

配置hadoop-3.2.0/etc/hadoop/hadoop-env.sh

# The java implementation to use. By default, this environment
export JAVA_HOME=${JAVA_HOME}

配置hadoop-3.2.0/etc/hadoop/core-site.xml

<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://localhost:9000</value>
        </property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>file:/home/hduser/hadoop-family/hadoop-3.2.0/hdfs-tmp</value>
                <description>Abase for other temporary directories.</description>
        </property>
</configuration>

配置hdfs-site.xml

<configuration>
        <property>
                <name>dfs.namenode.secondary.http-address</name>
                <value>localhost:50090</value>
        </property>
        <property>
                <name>dfs.namenode.name.dir</name>
                <value>file:/home/hduser/hadoop-family/hadoop-3.2.0/file-tmp/dfs/name</value>
        </property>
        <property>
                <name>dfs.datanode.data.dir</name>
                <value>file:/home/hduser/hadoop-family/hadoop-3.2.0/file-tmp/dfs/data</value>
        </property>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
</configuration>

配置hadoop-3.2.0/etc/hadoop/mapred-site.xml

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
        <property>
                <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
        </property>
        <property>
            <name>mapreduce.jobhistory.address</name>
            <value>localhost:10020</value>
        </property>
        <property>
            <name>mapreduce.jobhistory.webapp.address</name>
            <value>localhost:19888</value>
        </property>

        <property>
            <name>yarn.app.mapreduce.am.env</name>
            <value>HADOOP_MAPRED_HOME=/home/hduser/hadoop-family/hadoop-3.2.0/hdfs-tmp</value>
        </property>
        <property>
            <name>mapreduce.map.env</name>
            <value>HADOOP_MAPRED_HOME=/home/hduser/hadoop-family/hadoop-3.2.0/hdfs-tmp</value>
        </property>
        <property>
            <name>mapreduce.reduce.env</name>
            <value>HADOOP_MAPRED_HOME=/home/hduser/hadoop-family/hadoop-3.2.0/hdfs-tmp</value>
        </property>
</configuration>

配置hadoop-3.2.0/etc/hadoop/yarn-site.xml

<configuration>
<!-- Site specific YARN configuration properties -->
        <property>
                <name>yarn.resourcemanager.hostname</name>
                <value>localhost</value>
        </property>
        <property>
                <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
        </property>
        <property>
                 <name>yarn.resourcemanager.webapp.address</name>
                 <value>localhost:8088</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.scheduler.address</name>
                 <value>localhost:8030</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.address</name>
                 <value>localhost:8032</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.resource-tracker.address</name>
                 <value>localhost:8031</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.admin.address</name>
                 <value>localhost:8033</value>
         </property>
</configuration>

8. 格式化Hadoop文件系统

hdfs namenode -format

9. 启动Hadoop

Tips:
若出现DataNode未启动的现象，原因可能是由于hdfs-tmp/dfs/name/current/VERSION和hdfs-tmp/dfs/name/current/VERSION中的clusterID不匹配。
可以清空hdfs-tmp，并重新执行hadoop-3.2.0/bin/hdfs namenode -format。
或将hdfs-tmp/dfs/name/current/VERSION的clusterID复制进hdfs-tmp/dfs/name/current/VERSION。