021 单节点集群上 Ubuntu 安装 Hadoop 3.X

作者: 胡巴Lei特 | 来源:发表于2019-08-03 14:01 被阅读0次

021 单节点集群上 Ubuntu 安装 Hadoop 3.X
CentOS6.5安装Hbase1.2.6
Hadoop 入门(二)
Hadoop HA的安装
Ubuntu上安装Hadoop集群
手动部署Hadoop2分布式集群和YARN集群管理命令
hadoop上搭建hive(远程模式+hive服务端、客户端)
Hive环境搭建
sqoop的安装
Apache Spark整合Hive

021 Installation of Hadoop 3.x on Ubuntu on Single Node Cluster

1. Objective

1. 目标

In this tutorial on Installation of Hadoop 3.x on Ubuntu, we are going to learn steps for setting up a pseudo-distributed, single-node Hadoop 3.x cluster on Ubuntu. We will learn steps like how to install java, how to install SSH and configure passwordless SSH, how to download Hadoop, how to setup Hadoop configurations like .bashrc file, hadoop-env.sh, core-site.xml, hdfs-site.xml, mapred-site.xml, YARN-site.xml, how to start the Hadoop cluster and how to stop the Hadoop services.

在本教程中，安装Hadoop3. 个 Ubuntu 的，我们要学习步骤建立伪分布式，单-Hadoop 3. 个集群节点在 Ubuntu.我们将学习如何安装 java 、如何安装 SSH 和配置无密码 SSH 、如何下载 Hadoop 、如何设置 Hadoop 配置等步骤.如何启动 Hadoop 集群以及如何停止 Hadoop 服务，bashrc 文件、 hadoop-env.sh 、 core-site.xml 、 hdfs-site.xml 、 mapred-site.xml 、 YARN-site.xml.

Learn step by step installation of Hadoop 2.7.x on Ubuntu.

在 Ubuntu 上了解 Hadoop 2.7.X 的分步安装.

Installation of Hadoop 3.x on Ubuntu on Single Node Cluster

2. Installation of Hadoop 3.x on Ubuntu

Hadoop 2. 安装 Ubuntu 的 3.x

Before we start with Hadoop 3.x installation on Ubuntu, let us understand key features that have been added in Hadoop 3 that makes the comparison between Hadoop 2 and Hadoop 3.

在我们开始在 Ubuntu 上安装 Hadoop 3.X 之前，让我们了解 Hadoop 3 中添加的关键功能，这些功能使得Hadoop 2 和 3. 的比较.

2.1. Java 8 installation

2.1.安装 Java 8

Hadoop requires working java installation. Let us start with steps for installing java 8:

Hadoop 需要安装 java.让我们从安装 java 8 的步骤开始:

a. Install Python Software Properties

A.安装 Python 软件属性

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">sudo apt-get install python-software-properties</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">安装 python-软件-属性</pre>

b. Add Repository

B.添加存储库

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">sudo add-apt-repository ppa:webupd8team/java</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Sudo add-apt-repository ppa: webupd8team/java</pre>

c. Update the source list

C.更新源列表

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">sudo apt-get update</pre>

d. Install Java 8

安装 Java 8

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">sudo apt-get install oracle-java8-installer</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Sudo apt-get 安装 oracle-java8-installer</pre>

e. Check if java is correctly installed

检查 java 是否安装正确

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">java -version</pre>

2.2. Configure SSH

2.2.配置 SSH

SSH is used for remote login. SSH is required in Hadoop to manage its nodes, i.e. remote machines and local machine if you want to use Hadoop on it. Let us now see SSH installation of Hadoop 3.x on Ubuntu:

远程登录使用 SSH.Hadoop 中需要 SSH 来管理其节点，即如果要在其上使用 Hadoop，则需要远程机器和本地机器.现在让我们在 Ubuntu 上看到 Hadoop 3.X 的 SSH 安装:

a. Installation of passwordless SSH

答: 无密码 SSH 的安装

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">sudo apt-get install ssh
sudo apt-get install pdsh</pre>

b. Generate Key Pairs

生成密钥对

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa</pre>

c. Configure passwordless ssh

配置无密码 ssh

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">cat _{/.ssh/id_rsa.pub>>}/.ssh/authorized_keys</pre>

e. Change the permission of file that contains the key

E.更改包含密钥的文件的权限

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">chmod 0600 ~/.ssh/authorized_keys</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Chmod 0600 ~/.ssh/授权 _ 密钥</pre>

f. ****check ssh to the localhost

F.****检查本地主机的 ssh

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">ssh localhost</pre>

2.3. Install Hadoop

2.3.安装 Hadoop

a. Download Hadoop

A.下载 Hadoop

http://redrockdigimark.com/apachemirror/hadoop/common/hadoop-3.0.0-alpha2/hadoop-3.0.0-alpha2.tar.gz

(Download the latest version of Hadoop hadoop-3.0.0-alpha2.tar.gz)

(下载最新版 Hadoop hadoop-3.0.0-alpha2.tar.gz)

b. Untar Tarball

Untar Tarball

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">tar -xzf hadoop-3.0.0-alpha2.tar.gz</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Tar-xzf hadoop-3.0.0-alpha2.tar.gz</pre>

2.4. Hadoop Setup Configuration

2.4.Hadoop 设置配置

a. Edit .Bashrc
Open .bashrc

A.编辑.Bashrc
打开.Bashrc

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">nano ~/.bashrc</pre>

Edit .bashrc:

编辑.Bashrc:

Edit .bashrc file is located in user’s home directory and adds following parameters:

编辑.Bashrc 文件位于用户的主目录中，并添加了以下参数:

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">export HADOOP_PREFIX="/home/dataflair/hadoop-3.0.0-alpha2"
export PATH= $PATH:$ HADOOP_PREFIX/bin
export PATH= $PATH:$ HADOOP_PREFIX/sbin
export HADOOP_MAPRED_HOME= ${HADOOP_PREFIX} export HADOOP_COMMON_HOME=$ {HADOOP_PREFIX}
export HADOOP_HDFS_HOME= ${HADOOP_PREFIX} export YARN_HOME=$ {HADOOP_PREFIX}</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">导出 hadoop _ 前缀 = "/home/dataflair/hadoop-3.0.0-alpha2"
导出路径 = $PATH:$ hadoop _ 前缀/bin
导出路径 = $PATH:$ HADOOP_PREFIX/sbin
出口 HADOOP_MAPRED_HOME = ${ HADOOP_PREFIX} 出口 HADOOP_COMMON_HOME =$ { HADOOP_PREFIX}
出口 HADOOP_HDFS_HOME = ${ HADOOP_PREFIX} 出口 YARN_HOME =$ { HADOOP_PREFIX}</pre>

Then run

然后跑

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Source ~/.bashrc</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">源 ~/.bashrc</pre>

b. Edit hadoop-env.sh

编辑 hadoop-env.sh

Edit configuration file hadoop-env.sh (located in HADOOP_HOME/etc/hadoop) and set JAVA_HOME:

编辑配置文件 hadoop-env.sh (位于 hadoop _ home/etc/hadoop 中) 并设置 java _ home:

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">export JAVA_HOME=/usr/lib/jvm/java-8-oracle/</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">导出 java _ home =/usr/lib/jvm/java-8-oracle/</pre>

c. Edit core-site.xml

编辑 core-site.xml

Edit configuration file core-site.xml (located in HADOOP_HOME/etc/hadoop) and add following entries:

编辑配置文件 core-site.xml (位于 hadoop _ home/etc/hadoop 中) 并添加以下条目:

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/dataflair/hdata</value>
</property>
</configuration></pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><配置>
<物业>
<Name> fs.defaultFS </name>
Local://本地主机: 9000 </value>
</物业>
<物业>
<Name> hadoop.tmp.dir </name>
<Value>/home/dataflair/hdata </value>
</物业>
</配置></pre>

d. ****Edit hdfs-site.xml

D.****编辑 hdfs-site.xml

Edit configuration file hdfs-site.xml (located in HADOOP_HOME/etc/hadoop) and add following entries:

编辑配置文件 hdfs-site.xml (位于 hadoop _ home/etc/hadoop 中) 并添加以下条目:

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration></pre>

e. Edit mapred-site.xml

E.编辑 mapred-site.xml

If mapred-site.xml file is not available, then use

如果 mapred-site.xml 文件不可用，则使用

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">cp mapred-site.xml.template mapred-site.xml</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Mapred-site.xml.template cp mapred-site.xml</pre>

Edit configuration file mapred-site.xml (located in HADOOP_HOME/etc/hadoop) and add following entries:

编辑配置文件 mapred-site.xml (位于 hadoop _ home/etc/hadoop 中) 并添加以下条目:

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration></pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><配置>
<物业>
<Name> mapreduce.framework.name </name>
纱线 </value>
</物业>
</配置></pre>

f. Yarn-site.xml

F.Yarn-site.xml

Edit configuration file mapred-site.xml (located in HADOOP_HOME/etc/hadoop) and add following entries:

编辑配置文件 mapred-site.xml (位于 hadoop _ home/etc/hadoop 中) 并添加以下条目:

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration></pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"><配置>
<物业>
Aux.nodemanager.aux-服务 </name>
<<价值> mapreduce_shuffle/超值>
</物业>
<物业>
Yarn.nodemanager.Aux-services.mapreduce.shuffle.class </name>
<Value> org.apache.hadoop.mapred.ShuffleHandler </value>
</物业>
</配置></pre>

Test your Hadoop knowledge with this Big data Hadoop quiz.

测试你的 Hadoop 知识Hadoop 大数据问答.

2.5. How to Start the Hadoop services

2.5.Hadoop 服务怎么启动:

Let us now see how to start the Hadoop cluster:

现在让我们来看看如何启动 Hadoop 集群:

The first step to starting up your Hadoop installation is formatting the Hadoop filesystem which is implemented on top of the local filesystem of your “cluster”. This is done as follows:

启动 Hadoop 安装的第一步是格式化 Hadoop 文件系统，该文件系统是在 “集群” 的本地文件系统之上实现的.具体做法如下:

a. Format the namenode

A. 格式名称节点

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">bin/hdfs namenode -format</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Bin/hdfs 名称节点格式</pre>

NOTE: This activity should be done once when you install Hadoop and not for running Hadoop filesystem, else it will delete all your data from HDFS

注意: 当你安装 Hadoop 而不是运行 Hadoop 文件系统时，这个活动应该完成一次，否则它会从 HDFS 中删除你所有的数据

b. ****Start HDFS Services

B.****启动 HDFS 服务

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">sbin/start-dfs.sh</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">/Sbin/start-dfs.sh</pre>

It will give an error at the time of start HDFS services then use:

它会在开始时出错HDFS然后使用服务:

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">echo "ssh" | sudo tee /etc/pdsh/rcmd_default</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Echo “ssh” | sudo tee/etc/pdsh/rcmd _ default</pre>

c. Start YARN Services

开始纱线服务

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">sbin/start-yarn.sh</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">/Sbin/start-yarn.sh</pre>

d. ****Check how many daemons are running

D.****检查运行了多少守护进程

Let us now see whether expected Hadoop processes are running or not:

现在让我们看看预期的 Hadoop 进程是否正在运行:

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">jps
2961 ResourceManager
2482 DataNode
3077 NodeManager
2366 NameNode
2686 SecondaryNameNode
3199 Jps</pre>

<pre class="EnlighterJSRAW" data-enlighter-language="null" style="color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Jps
2961 资源管理器
2482 DataNode
3077 节点管理器
2366 南德
2686 秒
3199 Jps</pre>

Learn How to install Cloudera Hadoop CDH5 on ubuntu from this installation guide.

了解如何在 ubuntu 上安装 Cloudera Hadoop CDH5此安装指南.