美文网首页
Hadoop 2.x 初级集群安装

Hadoop 2.x 初级集群安装

作者: 壹点零 | 来源:发表于2018-02-04 10:33 被阅读0次

    一、安装文件上传解压

    二、环境变量配置

    vim /etc/profile
    
    ## jdk 环境变量
    export JAVA_HOME=/usr/local/java/jdk1.8
    export JAVA_BIN=$JAVA_HOME/bin
    export JAVA_LIB=$JAVA_HOME/lib
    export CLASSPATH=.:$JAVA_LIB/tools.jar:$JAVA_LIB/dt.jar
    export PATH=$JAVA_BIN:$PATH
    
    ## hadoop 环境变量
    export PATH=/usr/local/hadoop-2.7.5/bin:/usr/local/hadoop-2.7.5/sbin:$PATH
    

    三、hadoop配置文件

    修改/usr/local/hadoop-2.7.5/etc/hadoop/core-site.xml添加配置

    <configuration>
        <property>
            <name>fs.defaultFS</name>
            <value>hdfs://master:9000</value>
        </property>
    </configuration>
    

    修改/usr/local/hadoop-2.7.5/etc/hadoop/slaves添加slave节点

    192.168.1.11
    192.168.1.12
    192.168.1.13
    

    四、启动Hadoop

    1、先格式化 namenode
    hdfs namenode -format

    2、在master上启动namenode
    hadoop-daemon.sh start namenode

    3、在slave上启动datanode
    hadoop-daemon.sh start datanode

    4、在master上关闭namenode
    hadoop-daemon.sh stop namenode

    5、在slave上关闭datanode
    hadoop-daemon.sh stop datanode

    五、验证Hadoop

    hdfs dfsadmin -report
    
    [root@localhost hadoop]# hdfs dfsadmin -report
    Configured Capacity: 55479107584 (51.67 GB)
    Present Capacity: 46706745344 (43.50 GB)
    DFS Remaining: 46706708480 (43.50 GB)
    DFS Used: 36864 (36 KB)
    DFS Used%: 0.00%
    Under replicated blocks: 0
    Blocks with corrupt replicas: 0
    Missing blocks: 0
    Missing blocks (with replication factor 1): 0
    
    -------------------------------------------------
    Live datanodes (3):
    
    Name: 192.168.107.136:50010 (slave136)
    Hostname: slave136
    Decommission Status : Normal
    Configured Capacity: 18238930944 (16.99 GB)
    DFS Used: 12288 (12 KB)
    Non DFS Used: 2737418240 (2.55 GB)
    DFS Remaining: 15501500416 (14.44 GB)
    DFS Used%: 0.00%
    DFS Remaining%: 84.99%
    Configured Cache Capacity: 0 (0 B)
    Cache Used: 0 (0 B)
    Cache Remaining: 0 (0 B)
    Cache Used%: 100.00%
    Cache Remaining%: 0.00%
    Xceivers: 1
    Last contact: Thu Feb 01 00:25:16 PST 2018
    
    
    Name: 192.168.107.133:50010 (slave133)
    Hostname: slave133
    Decommission Status : Normal
    Configured Capacity: 18238930944 (16.99 GB)
    DFS Used: 12288 (12 KB)
    Non DFS Used: 2737139712 (2.55 GB)
    DFS Remaining: 15501778944 (14.44 GB)
    DFS Used%: 0.00%
    DFS Remaining%: 84.99%
    Configured Cache Capacity: 0 (0 B)
    Cache Used: 0 (0 B)
    Cache Remaining: 0 (0 B)
    Cache Used%: 100.00%
    Cache Remaining%: 0.00%
    Xceivers: 1
    Last contact: Thu Feb 01 00:25:16 PST 2018
    
    
    Name: 192.168.107.132:50010 (slave132)
    Hostname: localhost
    Decommission Status : Normal
    Configured Capacity: 19001245696 (17.70 GB)
    DFS Used: 12288 (12 KB)
    Non DFS Used: 3297804288 (3.07 GB)
    DFS Remaining: 15703429120 (14.62 GB)
    DFS Used%: 0.00%
    DFS Remaining%: 82.64%
    Configured Cache Capacity: 0 (0 B)
    Cache Used: 0 (0 B)
    Cache Remaining: 0 (0 B)
    Cache Used%: 100.00%
    Cache Remaining%: 0.00%
    Xceivers: 1
    Last contact: Thu Feb 01 00:25:16 PST 2018
    

    也可通过master提供的web程序进行访问:
    http://192.168.107.131:50070

    相关文章

      网友评论

          本文标题:Hadoop 2.x 初级集群安装

          本文链接:https://www.haomeiwen.com/subject/wjelzxtx.html