个人总结-Flume

1、书写配置文件

#agent1代表代理名称

Agent1.sources=source1

Agent1.sinks=sink1

Agent1.channels=channel1

#spoolingDirectory是监控指定文件夹中新文件的变化，一旦有新文件出现，就解析该文件，然后写入channle，写入完成后，标记该文件已经完成或者删除

#配置source1

Agent1.sources.source1.type=spooldir

Agent1.sources.source1..spoolDir=/root/hmbbs

Agent1.sources.source1.channels=channel1

Agent1.sources.source1..fileHeader=false

Agent1.sources.source1.interceptors=i1

Agent1.sources.source1.interceptors.i1.type=timestamp

#配置sink1

Agent1.sinks.sink1.type=hdfs

Agent1.sinks.sink1.hdfs.path=hdfs://hadoop0:8020/sinks

Agent1.sinks.sink1.hdfs.fileType=DataStream

Agent1.sinks.sink1.hdfs.writeFormat=TEXT

Agent1.sinks.sink1.hdfs.rollInterval=1

Agent1.sinks.sink1.channel=channel1

Agent1.sinks.sink1..hdfs.filePrefix=%Y-%m-%d

#配置channel1

Agent1.channels.channel1.type=file

Agent1.channels.channel1.checkpointDir=/root/hmbbs_tmp/123

Agent1.channels.channel1.dataDirs/root/hmbbs_tmp/

2、执行任务命令

Bin/flume-gn agent –n agent1 –c conf –fconf/example –Dflume.root.olgger=DEBUGconsole