问题描述

硬件：Openpower服务器

系统：CentOS 7.6 Power9

软件： HDP 3.0.1

现象：Ambari启动Yarn Registry DNS的时候报如下错误：

resource_management.core.exceptions.ExecutionFailed: Execution of 'ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/xxx/hadoop/libexec && /usr/hdp/xxx/hadoop-yarn/bin/yarn --config /usr/hdp/xxx/hadoop/conf --daemon start registrydns' returned 1. ERROR: Cannot set priority of registrydns process 4603

Yarn Registry DNS始终无法启动。查找错误日志文件cat /var/log/hadoop-yarn/yarn/privileged-root-registrydns-manager.bigdata.err发现如下错误：

Cannot find any VM in Java Home /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le
Cannot locate JVM library file
Service exit with a return value of 1

检查JAVA_HOME配置的路径，完全正确，排除JAVA_HOME的问题。

问题原因

经搜索发现如下链接：[DAEMON-358] PPC64: jsvc fails to find JVM jvm.cfg file and shared objects due to wrong path - ASF JIRA (apache.org)。问题描述和上述问题背景完全相同。可以确认是jsvc在ppc64le架构平台下的bug。

问题解决

因为尝试编译同版本(1.0.3)失败，提示无法识别系统架构。后来决定使用1.3.0版本。编译步骤参考：Daemon – Daemon : Java Service (apache.org)。

首先安装autoconf：

yum install autoconf -y

接下来下载源代码：commons-daemon-1.3.0-native-src.tar.gz。1.3.0版本的源代码本人和jvsc-ppc64.patch比对过，已经解决了jsvc的bug。接下来进入源码目录unix目录。执行./support/buildconf.sh：

[root@manager unix]# ./support/buildconf.sh
./support/buildconf.sh: configure script generated successfully

然后执行./configure：

[root@manager unix]# ./configure
*** Current host ***
checking build system type... powerpc64le-unknown-linux-gnu
checking host system type... powerpc64le-unknown-linux-gnu
checking cached host system type... ok
*** C-Language compilation tools ***
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for ranlib... ranlib
checking for strip... strip
*** Host support ***
checking C flags dependant on host system type... ok
*** Java compilation tools ***
checking for JDK os include directory...  linux
gcc flags added
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking sys/capability.h usability... no
checking sys/capability.h presence... no
checking for sys/capability.h... no
configure: WARNING: cannot find headers for libcap
*** Writing output files ***
configure: creating ./config.status
config.status: creating Makefile
config.status: creating Makedefs
config.status: creating native/Makefile
*** All done ***
Now you can issue "make"

最后执行make：

[root@manager unix]# make
(cd native; make  all)
make[1]: Entering directory `/opt/commons-daemon-1.3.0-native-src/unix/native'
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c jsvc-unix.c -o jsvc-unix.o
jsvc-unix.c: In function ‘run_controller’:
jsvc-unix.c:1311:20: warning: assignment from incompatible pointer type [enabled by default]
     act.sa_handler = controller;
                    ^
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c arguments.c -o arguments.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c debug.c -o debug.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c dso-dlfcn.c -o dso-dlfcn.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c dso-dyld.c -o dso-dyld.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c help.c -o help.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c home.c -o home.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c java.c -o java.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c location.c -o location.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c replace.c -o replace.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c locks.c -o locks.o
gcc -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"ppc64le\" -Wall -Wstrict-prototypes   -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.ppc64le/include/linux -c signals.c -o signals.o
ar cr libservice.a arguments.o debug.o dso-dlfcn.o dso-dyld.o help.o home.o java.o location.o replace.o locks.o signals.o
ranlib libservice.a
gcc   jsvc-unix.o libservice.a -ldl -lpthread -o ../jsvc
make[1]: Leaving directory `/opt/commons-daemon-1.3.0-native-src/unix/native'

到这一步如果没有遇到错误，在unix目录下会生成一个jsvc文件。使用这个文件替换/usr/lib/bigtop-utils/jsvc之后，Yarn Registry DNS启动成功。问题解决。