Windows配置Hadoop
Hadoop版本:2.9.2
首先,你得安装idea.

安装Maven

下载地址:https://maven.apache.org/download.cgi
然后配置

file->settings->build->build tools->maven
需要配置, maven home, user setting file和 local repository

也可以使用idea默认的maven创建项目. 我是默认创建的项目.

配置依赖

项目创建完毕之后, 在pom.xml进行配置
可以查看HDFS对应的版本进行下载:https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-hdfs

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.xyxj</groupId>
    <artifactId>hadoop</artifactId>
    <version>1.0-SNAPSHOT</version>


    <!--依赖-->
    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-hdfs</artifactId>
            <version>2.9.2</version>
        </dependency>
    </dependencies>

</project>

代码

万事俱备, 开始撸码.
之前我写过命令行的方式介绍,不清楚可以看看我前面的介绍

import org.apache.commons.io.FileSystemUtils;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

/**
 * Created by silen on 2018/12/1 1:02
 * Copyright (c) 2018 in FORETREE
 */
public class Demo {


    public static void main(String[] args) {
        try {
            //1.配置
            Configuration conf = new Configuration();
            conf.set("fs.defaultFS", "hdfs://127.0.0.1:9000/");
            conf.set("dfs.client.use.datanode.hostname", "true");
            //2.文件系统
            FileSystem fs = FileSystem.get(conf);
            //3. 创建文件夹
            fs.mkdirs(new Path("/bbb/ccc"));
            //4.上传文件
            fs.copyFromLocalFile(new Path("D:/word.json"),
                    new Path("/bbb/ccc"));
            fs.rename(new Path("/bbb/ccc/word.json"),
                    new Path("/aaa/1.json"));
            //5.下载文件
            fs.copyToLocalFile(new Path("/aaa/1.json"), new Path("D:/"));


        } catch (Exception e) {
            e.printStackTrace();
        }
    }

}

操作是将D盘的word.json上传到/bbb/ccc中.在改名移动到/aaa中在从/aaa中将改名的1.json的文件进行下载到D盘上.

查看

打开:http://localhost:50070/ 下的Utilities下的brow file system

也可以通过命令行查看

C:\Users\silen>hadoop fs -ls /aaa
Found 2 items
-rw-r--r--   3 silen supergroup   27354320 2018-12-01 01:43 /aaa/1.json
-rw-r--r--   1 silen supergroup   27354320 2018-11-29 23:16 /aaa/word.json