Hadoop学习笔记-HDFS篇（二）

用Java操作HDFS文件

配置jar包

在Eclipse里面导入hadoop-common.jar和hadoop-hdfs.jar这两个jar包

有两种导入方式：内部导入和外部导入

内部导入:在你的工程目录下新建一个lib文件夹，把下载好的jar包复制到lib目录下，右键单击jar包，选择buildpath-->Add to build path.这样做的好处是，jar包会随着工程一起，不论将工程部署到哪，都不会少jar包

外部导入: 直接在工程目录上右键单击，选择buildpath->configure build path->Add external jars.

上传文件:

public void uploadFile(Path localSource,Path hdfsDst) throws IOException{

Configuration conf = new Configuration(); 

conf.set("fs.defaultFS","hdfs://localhost:9000");

conf.set("fs.hdfs.impl","org.apache.hadoop.hdfs.DistributedFileSystem");

FileSystem fs = FileSystem.get(conf);

fs.copyFromLocalFile(localSource, hdfsDst);//具体的上传Api

}

得到文件的状态

public void getFileStatus() throws FileNotFoundException, IllegalArgumentException, IOException{
        Configuration conf = new Configuration();
        conf.set("fs.defaultFS","hdfs://localhost:9000");
        conf.set("fs.hdfs.impl","org.apache.hadoop.hdfs.DistributedFileSystem");
        FileSystem fs = FileSystem.get(conf);
        FileStatus []listStatus = fs.listStatus(new Path("/user/pinjhu/testDir"));
        for(FileStatus filestatus:listStatus){
            System.out.println(filestatus);
        }
    }

读取文件:

        Configuration conf = new Configuration();
        conf.set("fs.defaultFS","hdfs://localhost:9000");
        conf.set("fs.hdfs.impl","org.apache.hadoop.hdfs.DistributedFileSystem");
        FileSystem fs = FileSystem.get(conf);
        FSDataInputStream open = fs.open(new Path("/opt/pinjhu/hadoop/input/yarn-site.xml"));
        IOUtils.copyBytes(open, System.out, conf, true);
    }

删除文件或者文件夹

//删除文件夹
boolean delete = fileSystem.delete(new Path("/myhdfs"), true);
//删除文件
boolean deleteNotExist = fileSystem.delete(new Path("/myhdfs1"), true);