美文网首页
flume高可用测试

flume高可用测试

作者: wuyuan0127 | 来源:发表于2021-09-06 20:12 被阅读0次

源:

#agent1 name

agent1.channels = c1

agent1.sources = r1

agent1.sinks = k1 k2

#set gruop

agent1.sinkgroups = g1

#set channel

agent1.channels.c1.type = memory

#agent1.channels.c1.capacity = 1000

#agent1.channels.c1.transactionCapacity = 100

#agent1.sources.r1.type = netcat

#agent1.sources.r1.bind = localhost

#agent1.sources.r1.port = 44444

agent1.sources.r1.channels = c1

agent1.sources.r1.type = exec

agent1.sources.r1.command = tail -F /root/flume/runconf/a

# set sink1

agent1.sinks.k1.channel = c1

agent1.sinks.k1.type = avro

agent1.sinks.k1.hostname = localhost

agent1.sinks.k1.port = 52020

# set sink2

agent1.sinks.k2.channel = c1

agent1.sinks.k2.type = avro

agent1.sinks.k2.hostname = localhost

agent1.sinks.k2.port = 52021

#set sink group

agent1.sinkgroups.g1.sinks = k1 k2

#set failover

agent1.sinkgroups.g1.processor.type = failover

agent1.sinkgroups.g1.processor.priority.k1 = 10

agent1.sinkgroups.g1.processor.priority.k2 = 1

agent1.sinkgroups.g1.processor.maxpenalty = 10000

另外的两个配置 

a1.sources = r1

a1.channels = c1

a1.sinks = k1

#set channel

a1.channels.c1.type = memory

#a1.channels.c1.capacity = 1000

#a1.channels.c1.transactionCapacity = 100

# other node,nna to nns

a1.sources.r1.type = avro

a1.sources.r1.bind = localhost

a1.sources.r1.port = 52020

a1.sources.r1.channels = c1

#set sink to hdfs

a1.sinks.k1.type = logger

a1.sinks.k1.channel=c1

----------

a2.sources = r1

a2.channels = c1

a2.sinks = k1

#set channel

a2.channels.c1.type = memory

#a2.channels.c1.capacity = 1000

#a2.channels.c1.transactionCapacity = 100

# other node,nna to nns

a2.sources.r1.type = avro

a2.sources.r1.bind = localhost

a2.sources.r1.port = 52021

a2.sources.r1.channels = c1

#set sink to hdfs

a2.sinks.k1.type = logger

a2.sinks.k1.channel=c1

相关文章

  • flume高可用测试

    源: #agent1 name agent1.channels = c1 agent1.sources = r1 ...

  • 日志采集引擎 —— Flume

    一、什么是 Flume? 1、Flume 是做什么的?   Flume 是 Cloudera 开发的一个高可用、高...

  • Apache Flume

    1 Flume介绍 Flume (flume官网)是 Cloudera 提供的一个高可用的,高可靠的,分布式的...

  • Flume 入门

    Flume 概述 Flume 官网 Flume 是一个分布式的,高可靠的,高可用的服务去高效的收集,聚合并移动大量...

  • 090-BigData-18Flume

    上一篇:089-BigData-17ZooKeeper秒杀和hadoop高可用 一、Flume简介 Flume提供...

  • Flume系列1-Flume 概述

    一. Flume 定义   Flume 是Cloudera 提供的一个高可用的,高可靠的,分布式的海量日志采集、聚...

  • flume机制原理

    what's flume? flume 是系统, 是高可用的,高可靠的,分布式的海量数据日志采集,聚合和传输的系统...

  • 【Flume】Flume 简单理解及使用实例

    一、Flume简介 flume 是一个cloudera提供的 高可用高可靠,分布式的海量日志收集聚合传输系统。原名...

  • flume简介

    参考 Flume架构以及应用介绍 一.简介 Flume是Cloudera提供的一个高可用的,高可靠的,分布式的海量...

  • 数据采集组件:Flume基础用法和Kafka集成

    一、Flume简介 1、基础描述 Flume是Cloudera提供的一个高可用的,高可靠的,分布式的海量日志采集、...

网友评论

      本文标题:flume高可用测试

      本文链接:https://www.haomeiwen.com/subject/dzdywltx.html