软件准备
- Azkaban源码包，下载地址：https://github.com/azkaban/azkaban/releases
我这里使用3.76.0版本
环境准备
- JDK-1.8+
- Git：yum install git -y
(1) 解压源码包
# 解压后的目录结构如下：
[hadoop@beh07 azkaban-3.76.0]$ ll
total 92
drwxrwxr-x 3 hadoop hadoop 37 May 31 06:26 az-core
drwxrwxr-x 4 hadoop hadoop 52 May 31 06:26 az-crypto
drwxrwxr-x 5 hadoop hadoop 86 May 31 06:26 az-examples
drwxrwxr-x 3 hadoop hadoop 37 May 31 06:26 az-exec-util
drwxrwxr-x 3 hadoop hadoop 54 May 31 06:26 az-flow-trigger-dependency-plugin
drwxrwxr-x 3 hadoop hadoop 53 May 31 06:26 az-flow-trigger-dependency-type
drwxrwxr-x 3 hadoop hadoop 37 May 31 06:26 az-hadoop-jobtype-plugin
drwxrwxr-x 3 hadoop hadoop 37 May 31 06:26 az-hdfs-viewer
-rw-rw-r-- 1 hadoop hadoop 21925 May 31 06:26 az-intellij-style.xml
drwxrwxr-x 4 hadoop hadoop 49 May 31 06:26 az-jobsummary
drwxrwxr-x 3 hadoop hadoop 55 May 31 06:26 azkaban-common
drwxrwxr-x 3 hadoop hadoop 55 May 31 06:26 azkaban-db
drwxrwxr-x 3 hadoop hadoop 55 May 31 06:26 azkaban-exec-server
drwxrwxr-x 3 hadoop hadoop 37 May 31 06:26 azkaban-hadoop-security-plugin
drwxrwxr-x 3 hadoop hadoop 55 May 31 06:26 azkaban-solo-server
drwxrwxr-x 3 hadoop hadoop 54 May 31 06:26 azkaban-spi
drwxrwxr-x 3 hadoop hadoop 100 May 31 06:26 azkaban-web-server
drwxrwxr-x 3 hadoop hadoop 37 May 31 06:26 az-reportal
-rw-rw-r-- 1 hadoop hadoop 10672 May 31 06:26 build.gradle
-rw-rw-r-- 1 hadoop hadoop 6409 May 31 06:26 CONTRIBUTING.md
drwxrwxr-x 3 hadoop hadoop 4096 May 31 06:26 docs
drwxrwxr-x 3 hadoop hadoop 21 May 31 06:26 gradle
-rw-rw-r-- 1 hadoop hadoop 1488 May 31 06:26 gradle.properties
-rwxrwxr-x 1 hadoop hadoop 5296 May 31 06:26 gradlew
-rw-rw-r-- 1 hadoop hadoop 2260 May 31 06:26 gradlew.bat
-rw-rw-r-- 1 hadoop hadoop 11358 May 31 06:26 LICENSE
-rw-rw-r-- 1 hadoop hadoop 2359 May 31 06:26 NOTICE
-rw-rw-r-- 1 hadoop hadoop 2406 May 31 06:26 README.md
-rw-rw-r-- 1 hadoop hadoop 31 May 31 06:26 requirements.txt
-rw-rw-r-- 1 hadoop hadoop 1170 May 31 06:26 settings.gradle
drwxrwxr-x 6 hadoop hadoop 124 May 31 06:26 test
drwxrwxr-x 2 hadoop hadoop 78 May 31 06:26 tools
(2) 优化编译速度
查看azkaban-3.76.0/gradle/wrapper/gradle-wrapper.properties文件：
distributionUrl=https\://services.gradle.org/distributions/gradle-4.6-all.zip
可以看到依赖的gradle为4.6版本，从https://gradle.org/releases/下载对应版本的gradle的zip包，放到azkaban-3.76.0/gradle/wrapper目录下。
然后修改gradle-wrapper.properties文件的最后一行，改为：
#distributionUrl=https\://services.gradle.org/distributions/gradle-4.6-all.zip
distributionUrl=gradle-4.6-all.zip
(3) 编译源码
在解压目录下有一个可执行文件gradlew，进行以下操作：
step 1
这一步是执行时间最长的，耐心等待
[hadoop@beh07 azkaban-3.76.0]$ ./gradlew build
Downloading file:/opt/beh/core/azkaban-3.76.0/gradle/wrapper/gradle-4.6-all.zip
......
# 当看到BUILD SUCCESSFUL的时候，你会感觉自己真是鸿运当头
BUILD SUCCESSFUL in 5m 6s
105 actionable tasks: 96 executed, 9 from cache
- 可能遇到的报错1：
Could not determine the dependencies of task ':az-flow-trigger-dependency-type:kafka-event-trigger:fatJar'.
> Could not resolve all files for configuration ':az-flow-trigger-dependency-type:kafka-event-trigger:compile'.
> Could not download avro-tools.jar (org.apache.avro:avro-tools:1.8.1)
> Could not get resource 'https://repo.maven.apache.org/maven2/org/apache/avro/avro-tools/1.8.1/avro-tools-1.8.1.jar'.
> Read timed out
> Could not download netty.jar (io.netty:netty:3.10.5.Final)
> Could not get resource 'https://repo.maven.apache.org/maven2/io/netty/netty/3.10.5.Final/netty-3.10.5.Final.jar'.
> Connection reset
解决办法：重新执行./gradlew build，直到所需的jar包下载完成
- 可能遇到的报错2：
> Could not resolve net.jpountz.lz4:lz4:1.2.0.
Required by:
project :az-hadoop-jobtype-plugin > org.apache.spark:spark-core_2.10:1.4.0
> Skipped due to earlier error
解决办法：重新执行./gradlew build，直到所需的jar包下载完成
step 2
[hadoop@beh07 azkaban-3.76.0]$ ./gradlew clean
Parallel execution with configuration on demand is an incubating feature.
BUILD SUCCESSFUL in 2s
19 actionable tasks: 19 executed
step 3
[hadoop@beh07 azkaban-3.76.0]$ ./gradlew installDist
Parallel execution with configuration on demand is an incubating feature.
> Task :azkaban-web-server:npm_install
added 39 packages in 0.835s
BUILD SUCCESSFUL in 11s
53 actionable tasks: 38 executed, 14 from cache, 1 up-to-date
step 4
[hadoop@beh07 azkaban-3.76.0]$ ./gradlew test
Parallel execution with configuration on demand is an incubating feature.
......
BUILD SUCCESSFUL in 1m 59s
68 actionable tasks: 24 executed, 13 from cache, 31 up-to-date
如果遇到报错的情况，多执行几次
step 5
[hadoop@beh07 azkaban-3.76.0]$ ./gradlew build -x test
Parallel execution with configuration on demand is an incubating feature.
......
BUILD SUCCESSFUL in 22s
74 actionable tasks: 31 executed, 1 from cache, 42 up-to-date
执行完以上5步之后，我们就编译好了源码，需要的安装包也已经打包好了，去以下目录中找：
azkaban-3.76.0/azkaban-db/build/distributions
azkaban-3.76.0/azkaban-exec-server/build/distributions
azkaban-3.76.0/azkaban-hadoop-security-plugin/build/distributions
azkaban-3.76.0/azkaban-solo-server/build/distributions
azkaban-3.76.0/azkaban-web-server/build/distributions
找到的.tar.gz文件分别是：
azkaban-db-0.1.0-SNAPSHOT.tar.gz
azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz
azkaban-hadoop-security-plugin-0.1.0-SNAPSHOT.tar.gz
azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz
azkaban-web-server-0.1.0-SNAPSHOT.tar.gz
这样，一份新鲜的azkaban安装包就已经出炉了！
(6) 编译 execute-as-user.c 文件
这个文件后面会使用到，这里也编译一下，提前准备好。
# 找到这个文件的位置
[hadoop@beh07 azkaban-3.76.0]$ find . -name execute-as-user.c
./az-exec-util/src/main/c/execute-as-user.c
# 把文件复制到源码包的根目录下，好找一点
[hadoop@beh07 azkaban-3.76.0]$ cp ./az-exec-util/src/main/c/execute-as-user.c ./
# 编译这个文件，并改名为execute-as-user
[hadoop@beh07 azkaban-3.76.0]$ gcc execute-as-user.c -o execute-as-user
# 检查一下，看看是否有execute-as-user文件产生
[hadoop@beh07 azkaban-3.76.0]$ ll | grep execute-as-user
-rwxrwxr-x 1 hadoop hadoop 13616 Aug 23 10:57 execute-as-user
-rw-rw-r-- 1 hadoop hadoop 3976 Aug 23 10:53 execute-as-user.c
然后把execute-as-user这个文件保存好，我们以后会用的到。
网友评论