flink 启动问题记录

作者: 邵红晓 | 来源:发表于2019-11-25 18:54 被阅读0次

flink 启动问题记录
flink on yarn 启动源码分析
Flink(2) 简单上手
Window10安装Flink1.10.0-大坑
Quickstart
Flink Seesion模式下Dynamic User Cod
Flink on Yarn模式启动流程源代码分析
hadoop 3.x 案例5:Flink SQL运行报错
Flink on YARN 基础架构与启动流程
flink 的部署模式

flink 并行度设置优先级从低到高排序，排序为api>env>p>file，parallelism不能多与slot个数，设置合适的并行度，能提高运算效率
flink 并行度大于kafka分区数，则有任务会空转，解决办法参考
http://yangaidi.com/2019/05/24/flink%E6%B6%88%E8%B4%B9kafka%E6%97%B6%E5%87%BA%E7%8E%B0%E6%95%B0%E6%8D%AE%E5%80%BE%E6%96%9C%E7%9A%84%E5%8E%9F%E5%9B%A0%E5%92%8C%E5%A4%84%E7%90%86%E6%96%B9%E5%BC%8F/
rescale 性能高于Rebalancing(全局性通过网络传输实现) ，但是不能完全解决流式数据倾斜
flink 如果某个任务执行一直报错，那么，会导致整个flink反压会非常高，并且checkpoint 一直失败，参考https://www.jianshu.com/p/539587047ad4
每个算子的一个并行度实例就是一个subtask。那么，带来很多问题，由于flink的taskmanager运行task的时候是每个task采用一个单独的线程，这就会带来很多线程切换开销，进而影响吞吐量。为了减轻这种情况，flink进行了优化，也即对subtask进行chain操作，chain操作结束之后得到的task，再作为一个 调度执行单元，放到一个线程里执行，如下图，一共是6个调度执行单元,需要6个线程来完成任务

image.png

2019-11-25 19:04:35,989 INFO  org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend  - Deleting existing instance base directory /export11/hadoop/yarn/local/usercache/xcardata/appcache/application_1571902955229_72425/flink-io-0f70d123-659f-4d4a-8377-0e8b8820129b/job_c2b9ef1d6bb94e5bb71b85c16ce77b7f_op_WindowOperator_47d89856a1cf553f16e7063d953b7d42__3_8__uuid_1408db12-99ce-4e56-8b31-f4d190b9bfe4.
2019-11-25 19:04:35,990 INFO  org.apache.flink.runtime.taskmanager.Task                     - Window(GlobalWindows(), PurgingTrigger, ScalaProcessWindowFunctionWrapper) (3/8) (92e2021161941cd65e0e010c1bdf8697) switched from RUNNING to FAILED.
java.lang.NullPointerException

org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException: Connection unexpectedly closed by remote task manager 'shyt-hadoop-4024.xxx.com.cn/xx.xx.xx.xx:28212'. This might indicate that the remote task manager was lost.
    at org.apache.flink.runtime.io.network.netty.CreditBasedPartitionRequestClientHandler.channelInactive(CreditBasedPartitionRequestClientHandler.java:136)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
    at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
    at org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
    at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1429)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
    at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:947)
    at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)
    at org.apache.flink.shaded.netty4.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
    at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
    at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
    at java.lang.Thread.run(Thread.java:748)