51cto赵强HADOOP学习（五）

HDFS的联盟和配置

image.png

修改配置文件

#cd  training/hadoop-2.4.1/etc/hadoop/
#vi core-site.xml
<!--
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://192.168.56.31:9000</value
>
</property>
-->

# vi hdfs-site.xml
<property>
    <name>dfs.nameservices</name>
    <value>ns1,ns2</value>
</property>
<property>
    <name>dfs.namenode.rpc-address.ns1</name>
    <value>192.168.56.31:9000</value>
</property>
<property>
    <name>dfs.namenode.http-address.ns1</name>
    <value>192.168.56.31:50070</value>
</property>
<property>
    <name>dfs.namenode.secondaryhttp-address.ns1</name>
    <value>192.168.56.31:50090</value>
</property>
<property>
    <name>dfs.namenode.rpc-address.ns2</name>
    <value>192.168.56.34:9000</value>
</property>
<property>
    <name>dfs.namenode.http-address.ns2</name>
    <value>192.168.56.34:50070</value>
</property>
<property>
    <name>dfs.namenode.secondaryhttp-address.ns2</name>
    <value>192.168.56.34:50090</value>
</property>
<property>
    <name>dfs.replication</name>
    <value>2</value>
</property>
<property>
    <name>dfs.permissions</name>
    <value>false</value>
</property>

image.png

#cd ~/training
#pwd
/root/training
#scp -r hadoop-2.4.1/ root@hadoop32:/root/training/
#scp -r hadoop-2.4.1/ root@hadoop33:/root/training/
#scp -r hadoop-2.4.1/ root@hadoop34:/root/training/

在hadoop31和hadoop34上面格式化

#hdfs namenode -format -clusterId collen  //hadoop31
#hdfs namenode -format -clusterId collen   //hadoop34
#start-all.sh
#jps
hadoop31:
14075 NameNode
14300 ResourceManager
14557 Jps
hadoop32:
12689 Jps
12466 DataNode
12571 NodeManager
hadoop33:
12689 Jps
12466 DataNode
12571 NodeManager
hadoop34:
11853 NameNode
11919 Jps

HDFS联盟的视图文件系统

#hdfs dfs -ls /

什么是viewFS?

-视图文件系统（View File System，ViewFs）提供了管理多个Hadoop文件系统命名空间的方式，该系统在HDFS联盟的集群中有多个NameNode（因此有多个命名空间）是特别有用。

image.png

hadoop31:

#hdfs dfs -ls /
#stop-all.sh
#pwd
/root/training/hadoop-2.4.1/etc/hadoop
#vi core-site.xml
<configuration xmlns:xi="http://www.w3.org/2001/XIclude">
    <xi:include href="mountTable.xml"/>
    <property>
        <name>fs.default.name</name>
        <value>viewfs://collen</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/root/training/hadoop-2.4.1/tmm</value>
    </property>
</configuration>

把之前的给删掉

#vi mountTable.xml
<configuration>
<property>
    <name>fs.viewfs.mounttable.collen.homedir</name>
    <value>/home</value>
</property>

<property>
    <name>fs.viewfs.mounttable.collen.link./data</name>
    <value>hdfs://192.168.56.31:9000/data</value>
</property>

<property>
    <name>fs.viewfs.mounttable.collen.link./logs</name>
    <value>hdfs://192.168.56.34:9000/logs</value>
</property>

</configuration>

hadoop34

和hadoop31修改的完全一样

#vi core-site.xml
<configuration xmlns:xi="http://www.w3.org/2001/XIclude">
    <xi:include href="mountTable.xml"/>
    <property>
        <name>fs.default.name</name>
        <value>viewfs://collen</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/root/training/hadoop-2.4.1/tmm</value>
    </property>
</configuration>

#vi mountTable.xml
<configuration>
<property>
    <name>fs.viewfs.mounttable.collen.homedir</name>
    <value>/home</value>
</property>

<property>
    <name>fs.viewfs.mounttable.collen.link./data</name>
    <value>hdfs://192.168.56.31:9000/data</value>
</property>

<property>
    <name>fs.viewfs.mounttable.collen.link./logs</name>
    <value>hdfs://192.168.56.34:9000/logs</value>
</property>

</configuration>

hadoop31:

#start-all.sh、
#jps
10577 Jps
10082 NameNode
10310 ResourceManager
hadoop32:
5283 DataNode
5545 Jps
5389 NodeManager
hadoop33:
5283 DataNode
5545 Jps
5389 NodeManager
hadoop34:
3795 NameNode
3869 Jps
#hadoop fs -mkdir hdfs://192.168.56.31:9000/data  //hadoop31
#hadoop fs -mkdir hdfs://192.168.56.34:9000/logs  //hadoop34
#hdfs dfs -ls /
#vi a.txt
#hdfs dfs -put a.txt /data/a.txt
#hdfs dsf -ls /data
#hdfs dfs -cat /data/a.txt