1、下载flink、flink-shaded
flink 的 release-1.9分支，发现都带了SNAPSHOT就放弃使用了。
https://github.com/apache/flink/archive/release-1.9.0.tar.gz
flink-shaded 包含flink 的很多依赖，直接编译flink1.9运行wordcount例子会报错，找不到shaded包里相关的类，所以要先编译shaded
flink1.9 对应的flink-shaded 版本是 7.0
<flink.shaded.version>7.0</flink.shaded.version>
https://github.com/apache/flink-shaded/archive/release-7.0.tar.gz
2、编译
我用的hadoop是2.7.4所以指定-Dhadoop.version=2.7.4
xh@xh-MacBook-Pro ~/Documents/develop/work-study/flink-shaded-release-7.0 mvn clean install -DskipTests -Dhadoop.version=2.7.4
[INFO] Reactor Summary:
[INFO]
[INFO] flink-shaded 7.0 ................................... SUCCESS [ 3.483 s]
[INFO] flink-shaded-force-shading 7.0 ..................... SUCCESS [ 0.597 s]
[INFO] flink-shaded-asm-6 6.2.1-7.0 ....................... SUCCESS [ 0.627 s]
[INFO] flink-shaded-guava-18 18.0-7.0 ..................... SUCCESS [ 0.994 s]
[INFO] flink-shaded-netty-4 4.1.32.Final-7.0 .............. SUCCESS [ 3.469 s]
[INFO] flink-shaded-netty-tcnative-dynamic 2.0.25.Final-7.0 SUCCESS [ 0.572 s]
[INFO] flink-shaded-jackson-parent 2.9.8-7.0 .............. SUCCESS [ 0.035 s]
[INFO] flink-shaded-jackson-2 2.9.8-7.0 ................... SUCCESS [ 1.479 s]
[INFO] flink-shaded-jackson-module-jsonSchema-2 2.9.8-7.0 . SUCCESS [ 0.857 s]
[INFO] flink-shaded-hadoop-2 2.7.7-7.0 .................... SUCCESS [ 9.132 s]
[INFO] flink-shaded-hadoop-2-uber 2.7.7-7.0 ............... SUCCESS [ 17.173 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 38.612 s
[INFO] Finished at: 2020-11-09T17:04:17+08:00
[INFO] ------------------------------------------------------------------------
编译flink-1.9
xh@xh-MacBook-Pro ~/Documents/develop/work-study/flink-release-1.9.0 mvn clean install -DskipTests -Dhadoop.version=2.7.4
[INFO] force-shading ...................................... SUCCESS [ 4.135 s]
[INFO] flink 1.9.0 ........................................ SUCCESS [ 16.647 s]
[INFO] flink-annotations .................................. SUCCESS [ 10.306 s]
[INFO] flink-shaded-curator ............................... SUCCESS [ 4.131 s]
[INFO] flink-metrics ...................................... SUCCESS [ 0.512 s]
[INFO] flink-metrics-core ................................. SUCCESS [ 3.601 s]
[INFO] flink-test-utils-parent ............................ SUCCESS [ 0.557 s]
[INFO] flink-test-utils-junit ............................. SUCCESS [ 2.363 s]
[INFO] flink-core ......................................... SUCCESS [01:11 min]
[INFO] flink-java ......................................... SUCCESS [ 7.257 s]
[INFO] flink-queryable-state .............................. SUCCESS [ 0.153 s]
[INFO] flink-queryable-state-client-java .................. SUCCESS [ 1.385 s]
[INFO] flink-filesystems .................................. SUCCESS [ 0.205 s]
[INFO] flink-hadoop-fs .................................... SUCCESS [ 2.102 s]
[INFO] flink-runtime ...................................... SUCCESS [02:55 min]
[INFO] flink-scala ........................................ SUCCESS [ 55.924 s]
[INFO] flink-mapr-fs ...................................... SUCCESS [ 1.874 s]
[INFO] flink-filesystems :: flink-fs-hadoop-shaded ........ SUCCESS [ 12.776 s]
[INFO] flink-s3-fs-base ................................... SUCCESS [ 26.949 s]
[INFO] flink-s3-fs-hadoop ................................. SUCCESS [ 18.163 s]
[INFO] flink-s3-fs-presto ................................. SUCCESS [ 20.576 s]
[INFO] flink-swift-fs-hadoop .............................. SUCCESS [ 23.611 s]
[INFO] flink-oss-fs-hadoop ................................ SUCCESS [ 8.018 s]
[INFO] flink-azure-fs-hadoop .............................. SUCCESS [ 9.378 s]
[INFO] flink-optimizer .................................... SUCCESS [ 23.538 s]
[INFO] flink-clients ...................................... SUCCESS [ 2.734 s]
[INFO] flink-streaming-java ............................... SUCCESS [ 14.770 s]
[INFO] flink-test-utils ................................... SUCCESS [ 4.241 s]
[INFO] flink-runtime-web .................................. SUCCESS [02:33 min]
[INFO] flink-examples ..................................... SUCCESS [ 0.315 s]
[INFO] flink-examples-batch ............................... SUCCESS [ 15.594 s]
[INFO] flink-connectors ................................... SUCCESS [ 0.152 s]
[INFO] flink-hadoop-compatibility ......................... SUCCESS [ 5.252 s]
[INFO] flink-state-backends ............................... SUCCESS [ 0.145 s]
[INFO] flink-statebackend-rocksdb ......................... SUCCESS [ 1.744 s]
[INFO] flink-tests ........................................ SUCCESS [ 41.810 s]
[INFO] flink-streaming-scala .............................. SUCCESS [ 40.099 s]
[INFO] flink-table ........................................ SUCCESS [ 0.100 s]
[INFO] flink-table-common ................................. SUCCESS [ 3.692 s]
[INFO] flink-table-api-java ............................... SUCCESS [ 2.296 s]
[INFO] flink-table-api-java-bridge ........................ SUCCESS [ 0.962 s]
[INFO] flink-table-api-scala .............................. SUCCESS [ 6.053 s]
[INFO] flink-table-api-scala-bridge ....................... SUCCESS [ 8.989 s]
[INFO] flink-sql-parser ................................... SUCCESS [ 5.504 s]
[INFO] flink-libraries .................................... SUCCESS [ 0.208 s]
[INFO] flink-cep .......................................... SUCCESS [ 4.134 s]
[INFO] flink-table-planner ................................ SUCCESS [02:38 min]
[INFO] flink-orc .......................................... SUCCESS [ 1.188 s]
[INFO] flink-jdbc ......................................... SUCCESS [ 0.988 s]
[INFO] flink-table-runtime-blink .......................... SUCCESS [ 6.158 s]
[INFO] flink-table-planner-blink .......................... SUCCESS [04:24 min]
[INFO] flink-hbase ........................................ SUCCESS [ 14.734 s]
[INFO] flink-hcatalog ..................................... SUCCESS [ 18.495 s]
[INFO] flink-metrics-jmx .................................. SUCCESS [ 1.962 s]
[INFO] flink-connector-kafka-base ......................... SUCCESS [ 10.477 s]
[INFO] flink-connector-kafka-0.9 .......................... SUCCESS [ 3.296 s]
[INFO] flink-connector-kafka-0.10 ......................... SUCCESS [ 2.469 s]
[INFO] flink-connector-kafka-0.11 ......................... SUCCESS [ 3.185 s]
[INFO] flink-formats ...................................... SUCCESS [ 0.145 s]
[INFO] flink-json ......................................... SUCCESS [ 1.790 s]
[INFO] flink-connector-elasticsearch-base ................. SUCCESS [ 5.220 s]
[INFO] flink-connector-elasticsearch2 ..................... SUCCESS [ 16.131 s]
[INFO] flink-connector-elasticsearch5 ..................... SUCCESS [ 18.528 s]
[INFO] flink-connector-elasticsearch6 ..................... SUCCESS [ 2.826 s]
[INFO] flink-csv .......................................... SUCCESS [ 0.548 s]
[INFO] flink-connector-hive ............................... SUCCESS [ 5.145 s]
[INFO] flink-connector-rabbitmq ........................... SUCCESS [ 0.773 s]
[INFO] flink-connector-twitter ............................ SUCCESS [ 2.345 s]
[INFO] flink-connector-nifi ............................... SUCCESS [ 0.825 s]
[INFO] flink-connector-cassandra .......................... SUCCESS [ 4.264 s]
[INFO] flink-avro ......................................... SUCCESS [ 3.942 s]
[INFO] flink-connector-filesystem ......................... SUCCESS [ 2.263 s]
[INFO] flink-connector-kafka .............................. SUCCESS [ 1.938 s]
[INFO] flink-connector-gcp-pubsub ......................... SUCCESS [ 1.591 s]
[INFO] flink-sql-connector-elasticsearch6 ................. SUCCESS [ 10.079 s]
[INFO] flink-sql-connector-kafka-0.9 ...................... SUCCESS [ 0.583 s]
[INFO] flink-sql-connector-kafka-0.10 ..................... SUCCESS [ 0.803 s]
[INFO] flink-sql-connector-kafka-0.11 ..................... SUCCESS [ 1.142 s]
[INFO] flink-sql-connector-kafka .......................... SUCCESS [ 1.602 s]
[INFO] flink-connector-kafka-0.8 .......................... SUCCESS [ 1.644 s]
[INFO] flink-avro-confluent-registry ...................... SUCCESS [ 1.956 s]
[INFO] flink-parquet ...................................... SUCCESS [ 1.736 s]
[INFO] flink-sequence-file ................................ SUCCESS [ 0.508 s]
[INFO] flink-examples-streaming ........................... SUCCESS [ 21.053 s]
[INFO] flink-examples-table ............................... SUCCESS [ 9.998 s]
[INFO] flink-examples-build-helper ........................ SUCCESS [ 0.207 s]
[INFO] flink-examples-streaming-twitter ................... SUCCESS [ 1.348 s]
[INFO] flink-examples-streaming-state-machine ............. SUCCESS [ 0.842 s]
[INFO] flink-examples-streaming-gcp-pubsub ................ SUCCESS [ 4.719 s]
[INFO] flink-container .................................... SUCCESS [ 0.673 s]
[INFO] flink-queryable-state-runtime ...................... SUCCESS [ 1.054 s]
[INFO] flink-end-to-end-tests ............................. SUCCESS [ 0.111 s]
[INFO] flink-cli-test ..................................... SUCCESS [ 0.288 s]
[INFO] flink-parent-child-classloading-test-program ....... SUCCESS [ 0.352 s]
[INFO] flink-parent-child-classloading-test-lib-package ... SUCCESS [ 0.450 s]
[INFO] flink-dataset-allround-test ........................ SUCCESS [ 0.357 s]
[INFO] flink-dataset-fine-grained-recovery-test ........... SUCCESS [ 0.447 s]
[INFO] flink-datastream-allround-test ..................... SUCCESS [ 2.006 s]
[INFO] flink-batch-sql-test ............................... SUCCESS [ 0.346 s]
[INFO] flink-stream-sql-test .............................. SUCCESS [ 0.346 s]
[INFO] flink-bucketing-sink-test .......................... SUCCESS [ 0.810 s]
[INFO] flink-distributed-cache-via-blob ................... SUCCESS [ 0.302 s]
[INFO] flink-high-parallelism-iterations-test ............. SUCCESS [ 8.435 s]
[INFO] flink-stream-stateful-job-upgrade-test ............. SUCCESS [ 1.052 s]
[INFO] flink-queryable-state-test ......................... SUCCESS [ 2.355 s]
[INFO] flink-local-recovery-and-allocation-test ........... SUCCESS [ 0.656 s]
[INFO] flink-elasticsearch2-test .......................... SUCCESS [ 6.185 s]
[INFO] flink-elasticsearch5-test .......................... SUCCESS [ 6.898 s]
[INFO] flink-elasticsearch6-test .......................... SUCCESS [ 4.051 s]
[INFO] flink-quickstart ................................... SUCCESS [ 1.528 s]
[INFO] flink-quickstart-java .............................. SUCCESS [ 0.849 s]
[INFO] flink-quickstart-scala ............................. SUCCESS [ 0.326 s]
[INFO] flink-quickstart-test .............................. SUCCESS [ 0.547 s]
[INFO] flink-confluent-schema-registry .................... SUCCESS [ 2.300 s]
[INFO] flink-stream-state-ttl-test ........................ SUCCESS [ 4.510 s]
[INFO] flink-sql-client-test .............................. SUCCESS [ 1.438 s]
[INFO] flink-streaming-file-sink-test ..................... SUCCESS [ 0.356 s]
[INFO] flink-state-evolution-test ......................... SUCCESS [ 1.000 s]
[INFO] flink-e2e-test-utils ............................... SUCCESS [ 7.509 s]
[INFO] flink-mesos ........................................ SUCCESS [ 30.197 s]
[INFO] flink-yarn ......................................... SUCCESS [ 2.057 s]
[INFO] flink-gelly ........................................ SUCCESS [ 4.387 s]
[INFO] flink-gelly-scala .................................. SUCCESS [ 21.248 s]
[INFO] flink-gelly-examples ............................... SUCCESS [ 12.097 s]
[INFO] flink-metrics-dropwizard ........................... SUCCESS [ 0.355 s]
[INFO] flink-metrics-graphite ............................. SUCCESS [ 3.052 s]
[INFO] flink-metrics-influxdb ............................. SUCCESS [ 1.344 s]
[INFO] flink-metrics-prometheus ........................... SUCCESS [ 0.560 s]
[INFO] flink-metrics-statsd ............................... SUCCESS [ 0.305 s]
[INFO] flink-metrics-datadog .............................. SUCCESS [ 0.546 s]
[INFO] flink-metrics-slf4j ................................ SUCCESS [ 0.341 s]
[INFO] flink-cep-scala .................................... SUCCESS [ 12.946 s]
[INFO] flink-table-uber ................................... SUCCESS [ 2.824 s]
[INFO] flink-table-uber-blink ............................. SUCCESS [ 3.509 s]
[INFO] flink-sql-client ................................... SUCCESS [ 2.964 s]
[INFO] flink-state-processor-api .......................... SUCCESS [ 1.053 s]
[INFO] flink-python ....................................... SUCCESS [ 0.997 s]
[INFO] flink-scala-shell .................................. SUCCESS [ 14.369 s]
[INFO] flink-dist ......................................... SUCCESS [ 19.683 s]
[INFO] flink-end-to-end-tests-common ...................... SUCCESS [ 2.232 s]
[INFO] flink-metrics-availability-test .................... SUCCESS [ 0.469 s]
[INFO] flink-metrics-reporter-prometheus-test ............. SUCCESS [ 1.199 s]
[INFO] flink-heavy-deployment-stress-test ................. SUCCESS [ 7.551 s]
[INFO] flink-connector-gcp-pubsub-emulator-tests .......... SUCCESS [ 1.453 s]
[INFO] flink-streaming-kafka-test-base .................... SUCCESS [ 0.645 s]
[INFO] flink-streaming-kafka-test ......................... SUCCESS [ 6.752 s]
[INFO] flink-streaming-kafka011-test ...................... SUCCESS [ 7.046 s]
[INFO] flink-streaming-kafka010-test ...................... SUCCESS [ 5.742 s]
[INFO] flink-plugins-test ................................. SUCCESS [ 0.157 s]
[INFO] flink-tpch-test .................................... SUCCESS [ 0.921 s]
[INFO] flink-contrib ...................................... SUCCESS [ 0.133 s]
[INFO] flink-connector-wikiedits .......................... SUCCESS [ 0.742 s]
[INFO] flink-yarn-tests ................................... SUCCESS [ 5.500 s]
[INFO] flink-fs-tests ..................................... SUCCESS [ 0.548 s]
[INFO] flink-docs ......................................... SUCCESS [ 1.019 s]
[INFO] flink-ml-parent .................................... SUCCESS [ 0.086 s]
[INFO] flink-ml-api ....................................... SUCCESS [ 0.395 s]
[INFO] flink-ml-lib 1.9.0 ................................. SUCCESS [ 0.321 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27:48 min
[INFO] Finished at: 2020-11-09T17:04:17+08:00
[INFO] ------------------------------------------------------------------------
3、可能没有与自己hadoop 对应的 flink-shaded-hadoop-2 的包，maven仓库自己下载一下放到对应的位置即可
4、执行wordcount
Executing WordCount example with default input data set.
Use --input to specify file input.
Printing result to stdout. Use --output to specify output path.
(after,1)
(and,12)
(arrows,1)
(ay,1)
(be,4)
(bourn,1)
(cast,1)
(coil,1)
(come,1)
(country,1)
(d,4)
(dread,1)
(er,1)
网友评论