美文网首页
双层flume搭建过程中遇到的坑

双层flume搭建过程中遇到的坑

作者: bigdata张凯翔 | 来源:发表于2020-06-27 23:08 被阅读0次
    INFO - Creating /user/hive/warehouse/ods.db/origin_user_behavior/19-04-19/events.1555671386480.tmp
    ERROR - Avro source r2: Unable to process event batch. Exception follows.
    org.apache.flume.ChannelFullException: Space for commit to queue couldn't be acquired. Sinks are likely not keeping up with sources, or the buffer size is 
    

    解决方法:
    先将flume的JVM内存改为512M

    capacity = 1000
    transactionCapacity = 1000
    keep-alive = 30
    
    ERROR - Unable to deliver event. Exception follows.
    org.apache.flume.EventDeliveryException: Failed to send events
        at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:389)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
        at java.lang.Thread.run(Thread.java:748)
    Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoop104, port: 4545 }: Failed to send batch
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:314)
        at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:373)
        ... 3 more
    Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoop104, port: 4545 }: Avro RPC call returned Status: FAILED
        at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:390)
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:373)
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:302)
        ... 4 more
    
    image.png

    请求超时,导致发送event失败
    解决:
    设置request-timeout长一点,默认20秒

    相关文章

      网友评论

          本文标题:双层flume搭建过程中遇到的坑

          本文链接:https://www.haomeiwen.com/subject/sosbfktx.html