flink1.9.0编译

1、下载flink、flink-shaded

flink 的 release-1.9分支，发现都带了SNAPSHOT就放弃使用了。

https://github.com/apache/flink/archive/release-1.9.0.tar.gz

flink-shaded 包含flink 的很多依赖，直接编译flink1.9运行wordcount例子会报错，找不到shaded包里相关的类，所以要先编译shaded

flink1.9 对应的flink-shaded 版本是 7.0

        <flink.shaded.version>7.0</flink.shaded.version>

https://github.com/apache/flink-shaded/archive/release-7.0.tar.gz

2、编译

我用的hadoop是2.7.4所以指定-Dhadoop.version=2.7.4

xh@xh-MacBook-Pro  ~/Documents/develop/work-study/flink-shaded-release-7.0   mvn clean install -DskipTests -Dhadoop.version=2.7.4

[INFO] Reactor Summary:

[INFO]

[INFO] flink-shaded 7.0 ................................... SUCCESS [  3.483 s]

[INFO] flink-shaded-force-shading 7.0 ..................... SUCCESS [  0.597 s]

[INFO] flink-shaded-asm-6 6.2.1-7.0 ....................... SUCCESS [  0.627 s]

[INFO] flink-shaded-guava-18 18.0-7.0 ..................... SUCCESS [  0.994 s]

[INFO] flink-shaded-netty-4 4.1.32.Final-7.0 .............. SUCCESS [  3.469 s]

[INFO] flink-shaded-netty-tcnative-dynamic 2.0.25.Final-7.0 SUCCESS [  0.572 s]

[INFO] flink-shaded-jackson-parent 2.9.8-7.0 .............. SUCCESS [  0.035 s]

[INFO] flink-shaded-jackson-2 2.9.8-7.0 ................... SUCCESS [  1.479 s]

[INFO] flink-shaded-jackson-module-jsonSchema-2 2.9.8-7.0 . SUCCESS [  0.857 s]

[INFO] flink-shaded-hadoop-2 2.7.7-7.0 .................... SUCCESS [  9.132 s]

[INFO] flink-shaded-hadoop-2-uber 2.7.7-7.0 ............... SUCCESS [ 17.173 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  38.612 s

[INFO] Finished at: 2020-11-09T17:04:17+08:00

[INFO] ------------------------------------------------------------------------

编译flink-1.9

xh@xh-MacBook-Pro  ~/Documents/develop/work-study/flink-release-1.9.0   mvn clean install -DskipTests -Dhadoop.version=2.7.4

[INFO] force-shading ...................................... SUCCESS [ 4.135 s]

[INFO] flink 1.9.0 ........................................ SUCCESS [ 16.647 s]

[INFO] flink-annotations .................................. SUCCESS [ 10.306 s]

[INFO] flink-shaded-curator ............................... SUCCESS [  4.131 s]

[INFO] flink-metrics ...................................... SUCCESS [  0.512 s]

[INFO] flink-metrics-core ................................. SUCCESS [  3.601 s]

[INFO] flink-test-utils-parent ............................ SUCCESS [  0.557 s]

[INFO] flink-test-utils-junit ............................. SUCCESS [  2.363 s]

[INFO] flink-core ......................................... SUCCESS [01:11 min]

[INFO] flink-java ......................................... SUCCESS [  7.257 s]

[INFO] flink-queryable-state .............................. SUCCESS [  0.153 s]

[INFO] flink-queryable-state-client-java .................. SUCCESS [  1.385 s]

[INFO] flink-filesystems .................................. SUCCESS [  0.205 s]

[INFO] flink-hadoop-fs .................................... SUCCESS [  2.102 s]

[INFO] flink-runtime ...................................... SUCCESS [02:55 min]

[INFO] flink-scala ........................................ SUCCESS [ 55.924 s]

[INFO] flink-mapr-fs ...................................... SUCCESS [  1.874 s]

[INFO] flink-filesystems :: flink-fs-hadoop-shaded ........ SUCCESS [ 12.776 s]

[INFO] flink-s3-fs-base ................................... SUCCESS [ 26.949 s]

[INFO] flink-s3-fs-hadoop ................................. SUCCESS [ 18.163 s]

[INFO] flink-s3-fs-presto ................................. SUCCESS [ 20.576 s]

[INFO] flink-swift-fs-hadoop .............................. SUCCESS [ 23.611 s]

[INFO] flink-oss-fs-hadoop ................................ SUCCESS [  8.018 s]

[INFO] flink-azure-fs-hadoop .............................. SUCCESS [  9.378 s]

[INFO] flink-optimizer .................................... SUCCESS [ 23.538 s]

[INFO] flink-clients ...................................... SUCCESS [  2.734 s]

[INFO] flink-streaming-java ............................... SUCCESS [ 14.770 s]

[INFO] flink-test-utils ................................... SUCCESS [  4.241 s]

[INFO] flink-runtime-web .................................. SUCCESS [02:33 min]

[INFO] flink-examples ..................................... SUCCESS [  0.315 s]

[INFO] flink-examples-batch ............................... SUCCESS [ 15.594 s]

[INFO] flink-connectors ................................... SUCCESS [  0.152 s]

[INFO] flink-hadoop-compatibility ......................... SUCCESS [  5.252 s]

[INFO] flink-state-backends ............................... SUCCESS [  0.145 s]

[INFO] flink-statebackend-rocksdb ......................... SUCCESS [  1.744 s]

[INFO] flink-tests ........................................ SUCCESS [ 41.810 s]

[INFO] flink-streaming-scala .............................. SUCCESS [ 40.099 s]

[INFO] flink-table ........................................ SUCCESS [  0.100 s]

[INFO] flink-table-common ................................. SUCCESS [  3.692 s]

[INFO] flink-table-api-java ............................... SUCCESS [  2.296 s]

[INFO] flink-table-api-java-bridge ........................ SUCCESS [  0.962 s]

[INFO] flink-table-api-scala .............................. SUCCESS [  6.053 s]

[INFO] flink-table-api-scala-bridge ....................... SUCCESS [  8.989 s]

[INFO] flink-sql-parser ................................... SUCCESS [  5.504 s]

[INFO] flink-libraries .................................... SUCCESS [  0.208 s]

[INFO] flink-cep .......................................... SUCCESS [  4.134 s]

[INFO] flink-table-planner ................................ SUCCESS [02:38 min]

[INFO] flink-orc .......................................... SUCCESS [  1.188 s]

[INFO] flink-jdbc ......................................... SUCCESS [  0.988 s]

[INFO] flink-table-runtime-blink .......................... SUCCESS [  6.158 s]

[INFO] flink-table-planner-blink .......................... SUCCESS [04:24 min]

[INFO] flink-hbase ........................................ SUCCESS [ 14.734 s]

[INFO] flink-hcatalog ..................................... SUCCESS [ 18.495 s]

[INFO] flink-metrics-jmx .................................. SUCCESS [  1.962 s]

[INFO] flink-connector-kafka-base ......................... SUCCESS [ 10.477 s]

[INFO] flink-connector-kafka-0.9 .......................... SUCCESS [  3.296 s]

[INFO] flink-connector-kafka-0.10 ......................... SUCCESS [  2.469 s]

[INFO] flink-connector-kafka-0.11 ......................... SUCCESS [  3.185 s]

[INFO] flink-formats ...................................... SUCCESS [  0.145 s]

[INFO] flink-json ......................................... SUCCESS [  1.790 s]

[INFO] flink-connector-elasticsearch-base ................. SUCCESS [  5.220 s]

[INFO] flink-connector-elasticsearch2 ..................... SUCCESS [ 16.131 s]

[INFO] flink-connector-elasticsearch5 ..................... SUCCESS [ 18.528 s]

[INFO] flink-connector-elasticsearch6 ..................... SUCCESS [  2.826 s]

[INFO] flink-csv .......................................... SUCCESS [  0.548 s]

[INFO] flink-connector-hive ............................... SUCCESS [  5.145 s]

[INFO] flink-connector-rabbitmq ........................... SUCCESS [  0.773 s]

[INFO] flink-connector-twitter ............................ SUCCESS [  2.345 s]

[INFO] flink-connector-nifi ............................... SUCCESS [  0.825 s]

[INFO] flink-connector-cassandra .......................... SUCCESS [  4.264 s]

[INFO] flink-avro ......................................... SUCCESS [  3.942 s]

[INFO] flink-connector-filesystem ......................... SUCCESS [  2.263 s]

[INFO] flink-connector-kafka .............................. SUCCESS [  1.938 s]

[INFO] flink-connector-gcp-pubsub ......................... SUCCESS [  1.591 s]

[INFO] flink-sql-connector-elasticsearch6 ................. SUCCESS [ 10.079 s]

[INFO] flink-sql-connector-kafka-0.9 ...................... SUCCESS [  0.583 s]

[INFO] flink-sql-connector-kafka-0.10 ..................... SUCCESS [  0.803 s]

[INFO] flink-sql-connector-kafka-0.11 ..................... SUCCESS [  1.142 s]

[INFO] flink-sql-connector-kafka .......................... SUCCESS [  1.602 s]

[INFO] flink-connector-kafka-0.8 .......................... SUCCESS [  1.644 s]

[INFO] flink-avro-confluent-registry ...................... SUCCESS [  1.956 s]

[INFO] flink-parquet ...................................... SUCCESS [  1.736 s]

[INFO] flink-sequence-file ................................ SUCCESS [  0.508 s]

[INFO] flink-examples-streaming ........................... SUCCESS [ 21.053 s]

[INFO] flink-examples-table ............................... SUCCESS [  9.998 s]

[INFO] flink-examples-build-helper ........................ SUCCESS [  0.207 s]

[INFO] flink-examples-streaming-twitter ................... SUCCESS [  1.348 s]

[INFO] flink-examples-streaming-state-machine ............. SUCCESS [  0.842 s]

[INFO] flink-examples-streaming-gcp-pubsub ................ SUCCESS [  4.719 s]

[INFO] flink-container .................................... SUCCESS [  0.673 s]

[INFO] flink-queryable-state-runtime ...................... SUCCESS [  1.054 s]

[INFO] flink-end-to-end-tests ............................. SUCCESS [  0.111 s]

[INFO] flink-cli-test ..................................... SUCCESS [  0.288 s]

[INFO] flink-parent-child-classloading-test-program ....... SUCCESS [  0.352 s]

[INFO] flink-parent-child-classloading-test-lib-package ... SUCCESS [  0.450 s]

[INFO] flink-dataset-allround-test ........................ SUCCESS [  0.357 s]

[INFO] flink-dataset-fine-grained-recovery-test ........... SUCCESS [  0.447 s]

[INFO] flink-datastream-allround-test ..................... SUCCESS [  2.006 s]

[INFO] flink-batch-sql-test ............................... SUCCESS [  0.346 s]

[INFO] flink-stream-sql-test .............................. SUCCESS [  0.346 s]

[INFO] flink-bucketing-sink-test .......................... SUCCESS [  0.810 s]

[INFO] flink-distributed-cache-via-blob ................... SUCCESS [  0.302 s]

[INFO] flink-high-parallelism-iterations-test ............. SUCCESS [  8.435 s]

[INFO] flink-stream-stateful-job-upgrade-test ............. SUCCESS [  1.052 s]

[INFO] flink-queryable-state-test ......................... SUCCESS [  2.355 s]

[INFO] flink-local-recovery-and-allocation-test ........... SUCCESS [  0.656 s]

[INFO] flink-elasticsearch2-test .......................... SUCCESS [  6.185 s]

[INFO] flink-elasticsearch5-test .......................... SUCCESS [  6.898 s]

[INFO] flink-elasticsearch6-test .......................... SUCCESS [  4.051 s]

[INFO] flink-quickstart ................................... SUCCESS [  1.528 s]

[INFO] flink-quickstart-java .............................. SUCCESS [  0.849 s]

[INFO] flink-quickstart-scala ............................. SUCCESS [  0.326 s]

[INFO] flink-quickstart-test .............................. SUCCESS [  0.547 s]

[INFO] flink-confluent-schema-registry .................... SUCCESS [  2.300 s]

[INFO] flink-stream-state-ttl-test ........................ SUCCESS [  4.510 s]

[INFO] flink-sql-client-test .............................. SUCCESS [  1.438 s]

[INFO] flink-streaming-file-sink-test ..................... SUCCESS [  0.356 s]

[INFO] flink-state-evolution-test ......................... SUCCESS [  1.000 s]

[INFO] flink-e2e-test-utils ............................... SUCCESS [  7.509 s]

[INFO] flink-mesos ........................................ SUCCESS [ 30.197 s]

[INFO] flink-yarn ......................................... SUCCESS [  2.057 s]

[INFO] flink-gelly ........................................ SUCCESS [  4.387 s]

[INFO] flink-gelly-scala .................................. SUCCESS [ 21.248 s]

[INFO] flink-gelly-examples ............................... SUCCESS [ 12.097 s]

[INFO] flink-metrics-dropwizard ........................... SUCCESS [  0.355 s]

[INFO] flink-metrics-graphite ............................. SUCCESS [  3.052 s]

[INFO] flink-metrics-influxdb ............................. SUCCESS [  1.344 s]

[INFO] flink-metrics-prometheus ........................... SUCCESS [  0.560 s]

[INFO] flink-metrics-statsd ............................... SUCCESS [  0.305 s]

[INFO] flink-metrics-datadog .............................. SUCCESS [  0.546 s]

[INFO] flink-metrics-slf4j ................................ SUCCESS [  0.341 s]

[INFO] flink-cep-scala .................................... SUCCESS [ 12.946 s]

[INFO] flink-table-uber ................................... SUCCESS [  2.824 s]

[INFO] flink-table-uber-blink ............................. SUCCESS [  3.509 s]

[INFO] flink-sql-client ................................... SUCCESS [  2.964 s]

[INFO] flink-state-processor-api .......................... SUCCESS [  1.053 s]

[INFO] flink-python ....................................... SUCCESS [  0.997 s]

[INFO] flink-scala-shell .................................. SUCCESS [ 14.369 s]

[INFO] flink-dist ......................................... SUCCESS [ 19.683 s]

[INFO] flink-end-to-end-tests-common ...................... SUCCESS [  2.232 s]

[INFO] flink-metrics-availability-test .................... SUCCESS [  0.469 s]

[INFO] flink-metrics-reporter-prometheus-test ............. SUCCESS [  1.199 s]

[INFO] flink-heavy-deployment-stress-test ................. SUCCESS [  7.551 s]

[INFO] flink-connector-gcp-pubsub-emulator-tests .......... SUCCESS [  1.453 s]

[INFO] flink-streaming-kafka-test-base .................... SUCCESS [  0.645 s]

[INFO] flink-streaming-kafka-test ......................... SUCCESS [  6.752 s]

[INFO] flink-streaming-kafka011-test ...................... SUCCESS [  7.046 s]

[INFO] flink-streaming-kafka010-test ...................... SUCCESS [  5.742 s]

[INFO] flink-plugins-test ................................. SUCCESS [  0.157 s]

[INFO] flink-tpch-test .................................... SUCCESS [  0.921 s]

[INFO] flink-contrib ...................................... SUCCESS [  0.133 s]

[INFO] flink-connector-wikiedits .......................... SUCCESS [  0.742 s]

[INFO] flink-yarn-tests ................................... SUCCESS [  5.500 s]

[INFO] flink-fs-tests ..................................... SUCCESS [  0.548 s]

[INFO] flink-docs ......................................... SUCCESS [  1.019 s]

[INFO] flink-ml-parent .................................... SUCCESS [  0.086 s]

[INFO] flink-ml-api ....................................... SUCCESS [  0.395 s]

[INFO] flink-ml-lib 1.9.0 ................................. SUCCESS [  0.321 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 27:48 min

[INFO] Finished at: 2020-11-09T17:04:17+08:00

[INFO] ------------------------------------------------------------------------

3、可能没有与自己hadoop 对应的 flink-shaded-hadoop-2 的包，maven仓库自己下载一下放到对应的位置即可

4、执行wordcount

Executing WordCount example with default input data set.

Use --input to specify file input.

Printing result to stdout. Use --output to specify output path.

(after,1)

(and,12)

(arrows,1)

(ay,1)

(be,4)

(bourn,1)

(cast,1)

(coil,1)

(come,1)

(country,1)

(d,4)

(dread,1)

(er,1)