bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master yarn \
./examples/jars/spark-examples_2.12-2.4.5.jar \
10
yarn 上面相关日志
Log Type: stderr
Log Type: stderr
Log Upload Time: 星期四 七月 16 13:53:51 +0800 2020
Log Length: 6086
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/filecache/10/__spark_libs__388839016180830494.zip/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.1.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2020-07-16 13:53:33,273 INFO util.SignalUtils: Registered signal handler for TERM
2020-07-16 13:53:33,274 INFO util.SignalUtils: Registered signal handler for HUP
2020-07-16 13:53:33,275 INFO util.SignalUtils: Registered signal handler for INT
2020-07-16 13:53:33,498 INFO spark.SecurityManager: Changing view acls to: atguigu
2020-07-16 13:53:33,498 INFO spark.SecurityManager: Changing modify acls to: atguigu
2020-07-16 13:53:33,499 INFO spark.SecurityManager: Changing view acls groups to:
2020-07-16 13:53:33,499 INFO spark.SecurityManager: Changing modify acls groups to:
2020-07-16 13:53:33,500 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(atguigu); groups with view permissions: Set(); users with modify permissions: Set(atguigu); groups with modify permissions: Set()
2020-07-16 13:53:33,720 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2020-07-16 13:53:34,023 INFO yarn.ApplicationMaster: Preparing Local resources
2020-07-16 13:53:34,606 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1594878225045_0001_000001
2020-07-16 13:53:34,911 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.202.103:8030
2020-07-16 13:53:34,925 INFO yarn.YarnRMClient: Registering the ApplicationMaster
2020-07-16 13:53:35,116 INFO client.TransportClientFactory: Successfully created connection to hadoop102/192.168.202.102:44009 after 73 ms (0 ms spent in bootstraps)
2020-07-16 13:53:35,335 INFO yarn.ApplicationMaster:
===============================================================================
YARN executor launch context:
env:
CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark_conf__<CPS>{{PWD}}/__spark_libs__/*<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*<CPS>{{PWD}}/__spark_conf__/__hadoop_conf__
SPARK_YARN_STAGING_DIR -> hdfs://hadoop102:8020/user/atguigu/.sparkStaging/application_1594878225045_0001
SPARK_USER -> atguigu
command:
{{JAVA_HOME}}/bin/java \
-server \
-Xmx1024m \
-Djava.io.tmpdir={{PWD}}/tmp \
'-Dspark.driver.port=44009' \
-Dspark.yarn.app.container.log.dir=<LOG_DIR> \
-XX:OnOutOfMemoryError='kill %p' \
org.apache.spark.executor.CoarseGrainedExecutorBackend \
--driver-url \
spark://CoarseGrainedScheduler@hadoop102:44009 \
--executor-id \
<executorId> \
--hostname \
<hostname> \
--cores \
1 \
--app-id \
application_1594878225045_0001 \
--user-class-path \
file:$PWD/__app__.jar \
1><LOG_DIR>/stdout \
2><LOG_DIR>/stderr
resources:
__spark_libs__ -> resource { scheme: "hdfs" host: "hadoop102" port: 8020 file: "/user/atguigu/.sparkStaging/application_1594878225045_0001/__spark_libs__388839016180830494.zip" } size: 219375098 timestamp: 1594878805504 type: ARCHIVE visibility: PRIVATE
__spark_conf__ -> resource { scheme: "hdfs" host: "hadoop102" port: 8020 file: "/user/atguigu/.sparkStaging/application_1594878225045_0001/__spark_conf__.zip" } size: 312060 timestamp: 1594878805821 type: ARCHIVE visibility: PRIVATE
===============================================================================
2020-07-16 13:53:35,634 INFO conf.Configuration: resource-types.xml not found
2020-07-16 13:53:35,635 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2020-07-16 13:53:35,651 INFO yarn.YarnAllocator: Will request 2 executor container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of overhead)
2020-07-16 13:53:35,676 INFO yarn.YarnAllocator: Submitted 2 unlocalized container requests.
2020-07-16 13:53:35,937 INFO yarn.ApplicationMaster: Started progress reporter thread with (heartbeat : 3000, initial allocation : 200) intervals
2020-07-16 13:53:35,960 INFO yarn.YarnAllocator: Launching container container_1594878225045_0001_01_000002 on host hadoop104 for executor with ID 1
2020-07-16 13:53:35,961 INFO yarn.YarnAllocator: Launching container container_1594878225045_0001_01_000003 on host hadoop102 for executor with ID 2
2020-07-16 13:53:35,963 INFO yarn.YarnAllocator: Received 2 containers from YARN, launching executors on 2 of them.
2020-07-16 13:53:49,897 INFO yarn.YarnAllocator: Driver requested a total number of 0 executor(s).
2020-07-16 13:53:49,991 INFO yarn.ApplicationMaster$AMEndpoint: Driver terminated or disconnected! Shutting down. hadoop102:44009
2020-07-16 13:53:49,991 INFO yarn.ApplicationMaster$AMEndpoint: Driver terminated or disconnected! Shutting down. hadoop102:44009
2020-07-16 13:53:49,994 INFO yarn.ApplicationMaster: Final app status: SUCCEEDED, exitCode: 0
2020-07-16 13:53:50,014 INFO yarn.ApplicationMaster: Unregistering ApplicationMaster with SUCCEEDED
2020-07-16 13:53:50,036 INFO impl.AMRMClientImpl: Waiting for application to be successfully unregistered.
2020-07-16 13:53:50,141 INFO yarn.ApplicationMaster: Deleting staging directory hdfs://hadoop102:8020/user/atguigu/.sparkStaging/application_1594878225045_0001
2020-07-16 13:53:50,166 INFO util.ShutdownHookManager: Shutdown hook called
Log Type: launch_container.sh
Log Type: launch_container.sh
Log Upload Time: 星期四 七月 16 13:53:51 +0800 2020
Log Length: 4848
#!/bin/bash
set -o pipefail -e
export PRELAUNCH_OUT="/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/prelaunch.out"
exec >"${PRELAUNCH_OUT}"
export PRELAUNCH_ERR="/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/prelaunch.err"
exec 2>"${PRELAUNCH_ERR}"
echo "Setting up env variables"
export JAVA_HOME=${JAVA_HOME:-"/opt/module/jdk1.8.0_212"}
export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/opt/module/hadoop-3.1.3"}
export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/opt/module/hadoop-3.1.3"}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/opt/module/hadoop-3.1.3/etc/hadoop"}
export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/opt/module/hadoop-3.1.3"}
export HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/opt/module/hadoop-3.1.3"}
export HADOOP_TOKEN_FILE_LOCATION="/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/appcache/application_1594878225045_0001/container_1594878225045_0001_01_000001/container_tokens"
export CONTAINER_ID="container_1594878225045_0001_01_000001"
export NM_PORT="34708"
export NM_HOST="hadoop103"
export NM_HTTP_PORT="8042"
export LOCAL_DIRS="/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/appcache/application_1594878225045_0001"
export LOCAL_USER_DIRS="/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/"
export LOG_DIRS="/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001"
export USER="atguigu"
export LOGNAME="atguigu"
export HOME="/home/"
export PWD="/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/appcache/application_1594878225045_0001/container_1594878225045_0001_01_000001"
export JVM_PID="$$"
export MALLOC_ARENA_MAX="4"
export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
export SPARK_YARN_STAGING_DIR="hdfs://hadoop102:8020/user/atguigu/.sparkStaging/application_1594878225045_0001"
export APPLICATION_WEB_PROXY_BASE="/proxy/application_1594878225045_0001"
export CLASSPATH="$PWD:$PWD/__spark_conf__:$PWD/__spark_libs__/*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:$PWD/__spark_conf__/__hadoop_conf__"
export APP_SUBMIT_TIME_ENV="1594878807012"
export SPARK_USER="atguigu"
echo "Setting up job resources"
ln -sf -- "/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/filecache/10/__spark_libs__388839016180830494.zip" "__spark_libs__"
ln -sf -- "/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/filecache/11/__spark_conf__.zip" "__spark_conf__"
echo "Copying debugging information"
# Creating copy of launch script
cp "launch_container.sh" "/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/launch_container.sh"
chmod 640 "/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/launch_container.sh"
# Determining directory contents
echo "ls -l:" 1>"/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/directory.info"
ls -l 1>>"/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/directory.info"
echo "find -L . -maxdepth 5 -ls:" 1>>"/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/directory.info"
find -L . -maxdepth 5 -ls 1>>"/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/directory.info"
echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/directory.info"
find -L . -maxdepth 5 -type l -ls 1>>"/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/directory.info"
echo "Launching container"
exec /bin/bash -c "$JAVA_HOME/bin/java -server -Xmx512m -Djava.io.tmpdir=$PWD/tmp -Dspark.yarn.app.container.log.dir=/opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001 org.apache.spark.deploy.yarn.ExecutorLauncher --arg 'hadoop102:44009' --properties-file $PWD/__spark_conf__/__spark_conf__.properties 1> /opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/stdout 2> /opt/module/hadoop-3.1.3/logs/userlogs/application_1594878225045_0001/container_1594878225045_0001_01_000001/stderr"
Log Type: directory.info
Log Type: directory.info
Log Upload Time: 星期四 七月 16 13:53:51 +0800 2020
Log Length: 22910
ls -l:
total 32
lrwxrwxrwx 1 atguigu atguigu 92 Jul 16 13:53 __spark_conf__ -> /opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/filecache/11/__spark_conf__.zip
lrwxrwxrwx 1 atguigu atguigu 110 Jul 16 13:53 __spark_libs__ -> /opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/atguigu/filecache/10/__spark_libs__388839016180830494.zip
-rw-r--r-- 1 atguigu atguigu 74 Jul 16 13:53 container_tokens
-rwx------ 1 atguigu atguigu 778 Jul 16 13:53 default_container_executor.sh
-rwx------ 1 atguigu atguigu 723 Jul 16 13:53 default_container_executor_session.sh
-rwx------ 1 atguigu atguigu 4848 Jul 16 13:53 launch_container.sh
drwx--x--- 2 atguigu atguigu 4096 Jul 16 13:53 tmp
find -L . -maxdepth 5 -ls:
2890413 4 drwx--x--- 3 atguigu atguigu 4096 Jul 16 13:53 .
2890415 4 drwx--x--- 2 atguigu atguigu 4096 Jul 16 13:53 ./tmp
2890423 4 -rw-r--r-- 1 atguigu atguigu 16 Jul 16 13:53 ./.default_container_executor.sh.crc
2890417 4 -rw-r--r-- 1 atguigu atguigu 12 Jul 16 13:53 ./.container_tokens.crc
2890421 4 -rw-r--r-- 1 atguigu atguigu 16 Jul 16 13:53 ./.default_container_executor_session.sh.crc
2890422 4 -rwx------ 1 atguigu atguigu 778 Jul 16 13:53 ./default_container_executor.sh
2890214 12 drwx------ 2 atguigu atguigu 12288 Jul 16 13:53 ./__spark_libs__
2890216 40 -r-x------ 1 atguigu atguigu 40086 Jul 16 13:53 ./__spark_libs__/json4s-jackson_2.12-3.5.3.jar
2890217 364 -r-x------ 1 atguigu atguigu 370119 Jul 16 13:53 ./__spark_libs__/lz4-java-1.4.0.jar
2890218 1300 -r-x------ 1 atguigu atguigu 1330219 Jul 16 13:53 ./__spark_libs__/netty-3.9.9.Final.jar
2890219 176 -r-x------ 1 atguigu atguigu 176285 Jul 16 13:53 ./__spark_libs__/automaton-1.11-8.jar
2890220 8 -r-x------ 1 atguigu atguigu 4596 Jul 16 13:53 ./__spark_libs__/jul-to-slf4j-1.7.16.jar
2890221 1468 -r-x------ 1 atguigu atguigu 1502280 Jul 16 13:53 ./__spark_libs__/htrace-core4-4.1.0-incubating.jar
2890222 20 -r-x------ 1 atguigu atguigu 18098 Jul 16 13:53 ./__spark_libs__/jersey-container-servlet-2.22.2.jar
2890223 16 -r-x------ 1 atguigu atguigu 14766 Jul 16 13:53 ./__spark_libs__/aopalliance-repackaged-2.4.0-b34.jar
2890224 316 -r-x------ 1 atguigu atguigu 323158 Jul 16 13:53 ./__spark_libs__/spark-kubernetes_2.12-2.4.5.jar
2890225 268 -r-x------ 1 atguigu atguigu 273370 Jul 16 13:53 ./__spark_libs__/commons-net-3.1.jar
2890226 172 -r-x------ 1 atguigu atguigu 174351 Jul 16 13:53 ./__spark_libs__/stream-2.7.0.jar
2890227 120 -r-x------ 1 atguigu atguigu 118973 Jul 16 13:53 ./__spark_libs__/hk2-utils-2.4.0-b34.jar
2890228 932 -r-x------ 1 atguigu atguigu 951701 Jul 16 13:53 ./__spark_libs__/jersey-server-2.22.2.jar
2890229 1256 -r-x------ 1 atguigu atguigu 1282424 Jul 16 13:53 ./__spark_libs__/ivy-2.4.0.jar
2890230 344 -r-x------ 1 atguigu atguigu 351802 Jul 16 13:53 ./__spark_libs__/jackson-module-scala_2.12-2.6.7.1.jar
2890231 396 -r-x------ 1 atguigu atguigu 404970 Jul 16 13:53 ./__spark_libs__/univocity-parsers-2.7.3.jar
2890232 708 -r-x------ 1 atguigu atguigu 723203 Jul 16 13:53 ./__spark_libs__/parquet-format-2.4.0.jar
2890233 200 -r-x------ 1 atguigu atguigu 201965 Jul 16 13:53 ./__spark_libs__/curator-framework-2.13.0.jar
2890234 72 -r-x------ 1 atguigu atguigu 69702 Jul 16 13:53 ./__spark_libs__/spire-macros_2.12-0.13.0.jar
2890235 68 -r-x------ 1 atguigu atguigu 66270 Jul 16 13:53 ./__spark_libs__/jersey-container-servlet-core-2.22.2.jar
2890236 20 -r-x------ 1 atguigu atguigu 20235 Jul 16 13:53 ./__spark_libs__/osgi-resource-locator-1.0.1.jar
2890237 36 -r-x------ 1 atguigu atguigu 33650 Jul 16 13:53 ./__spark_libs__/machinist_2.12-0.6.1.jar
2890238 504 -r-x------ 1 atguigu atguigu 512742 Jul 16 13:53 ./__spark_libs__/woodstox-core-5.0.3.jar
2890239 1168 -r-x------ 1 atguigu atguigu 1194003 Jul 16 13:53 ./__spark_libs__/arpack_combined_all-0.1.jar
2890240 32 -r-x------ 1 atguigu atguigu 30057 Jul 16 13:53 ./__spark_libs__/spark-sketch_2.12-2.4.5.jar
2890241 164 -r-x------ 1 atguigu atguigu 164422 Jul 16 13:53 ./__spark_libs__/core-1.1.2.jar
2890242 3596 -r-x------ 1 atguigu atguigu 3678534 Jul 16 13:53 ./__spark_libs__/scala-reflect-2.12.10.jar
2890243 80 -r-x------ 1 atguigu atguigu 80420 Jul 16 13:53 ./__spark_libs__/hadoop-yarn-server-web-proxy-3.1.3.jar
2890244 17460 -r-x------ 1 atguigu atguigu 17878574 Jul 16 13:53 ./__spark_libs__/hadoop-client-api-3.1.3.jar
2890245 1076 -r-x------ 1 atguigu atguigu 1097799 Jul 16 13:53 ./__spark_libs__/parquet-column-1.10.1.jar
2890246 128 -r-x------ 1 atguigu atguigu 128414 Jul 16 13:53 ./__spark_libs__/re2j-1.1.jar
2890247 784 -r-x------ 1 atguigu atguigu 801369 Jul 16 13:53 ./__spark_libs__/janino-3.0.9.jar
2890248 10804 -r-x------ 1 atguigu atguigu 11061771 Jul 16 13:53 ./__spark_libs__/kubernetes-model-4.6.1.jar
2890249 52 -r-x------ 1 atguigu atguigu 52037 Jul 16 13:53 ./__spark_libs__/arrow-format-0.10.0.jar
2890250 44 -r-x------ 1 atguigu atguigu 42858 Jul 16 13:53 ./__spark_libs__/jackson-module-paranamer-2.7.9.jar
2890251 264 -r-x------ 1 atguigu atguigu 269295 Jul 16 13:53 ./__spark_libs__/snakeyaml-1.15.jar
2890252 9436 -r-x------ 1 atguigu atguigu 9660469 Jul 16 13:53 ./__spark_libs__/spire_2.12-0.13.0.jar
2890253 100 -r-x------ 1 atguigu atguigu 99555 Jul 16 13:53 ./__spark_libs__/xz-1.5.jar
2890254 16 -r-x------ 1 atguigu atguigu 15490 Jul 16 13:53 ./__spark_libs__/spark-tags_2.12-2.4.5.jar
2890255 888 -r-x------ 1 atguigu atguigu 906708 Jul 16 13:53 ./__spark_libs__/zookeeper-3.4.13.jar
2890256 220 -r-x------ 1 atguigu atguigu 225042 Jul 16 13:53 ./__spark_libs__/scala-parser-combinators_2.12-1.1.0.jar
2890257 176 -r-x------ 1 atguigu atguigu 178947 Jul 16 13:53 ./__spark_libs__/hk2-api-2.4.0-b34.jar
2890258 288 -r-x------ 1 atguigu atguigu 292606 Jul 16 13:53 ./__spark_libs__/spark-mesos_2.12-2.4.5.jar
2890259 4 -r-x------ 1 atguigu atguigu 3180 Jul 16 13:53 ./__spark_libs__/macro-compat_2.12-1.1.1.jar
2890260 748 -r-x------ 1 atguigu atguigu 764569 Jul 16 13:53 ./__spark_libs__/jtransforms-2.4.0.jar
2890261 1292 -r-x------ 1 atguigu atguigu 1318940 Jul 16 13:53 ./__spark_libs__/arrow-vector-0.10.0.jar
2890262 12 -r-x------ 1 atguigu atguigu 12244 Jul 16 13:53 ./__spark_libs__/slf4j-log4j12-1.7.25.jar
2890263 56 -r-x------ 1 atguigu atguigu 54391 Jul 16 13:53 ./__spark_libs__/objenesis-2.5.1.jar
2890264 80 -r-x------ 1 atguigu atguigu 79283 Jul 16 13:53 ./__spark_libs__/arrow-memory-0.10.0.jar
2890265 1128 -r-x------ 1 atguigu atguigu 1154604 Jul 16 13:53 ./__spark_libs__/spark-streaming_2.12-2.4.5.jar
2890266 10424 -r-x------ 1 atguigu atguigu 10672015 Jul 16 13:53 ./__spark_libs__/scala-compiler-2.12.10.jar
2890267 4 -r-x------ 1 atguigu atguigu 3956 Jul 16 13:53 ./__spark_libs__/kubernetes-model-common-4.6.1.jar
2890268 1532 -r-x------ 1 atguigu atguigu 1565700 Jul 16 13:53 ./__spark_libs__/orc-core-1.5.5-nohive.jar
2890269 22220 -r-x------ 1 atguigu atguigu 22750297 Jul 16 13:53 ./__spark_libs__/hadoop-client-runtime-3.1.3.jar
2890270 132 -r-x------ 1 atguigu atguigu 132989 Jul 16 13:53 ./__spark_libs__/avro-ipc-1.8.2.jar
2890271 7172 -r-x------ 1 atguigu atguigu 7343426 Jul 16 13:53 ./__spark_libs__/mesos-1.4.0-shaded-protobuf.jar
2890272 5156 -r-x------ 1 atguigu atguigu 5276900 Jul 16 13:53 ./__spark_libs__/scala-library-2.12.10.jar
2890273 16 -r-x------ 1 atguigu atguigu 15824 Jul 16 13:53 ./__spark_libs__/metrics-json-3.1.5.jar
2890274 36 -r-x------ 1 atguigu atguigu 34654 Jul 16 13:53 ./__spark_libs__/paranamer-2.8.jar
2890275 136 -r-x------ 1 atguigu atguigu 138823 Jul 16 13:53 ./__spark_libs__/hadoop-auth-3.1.3.jar
2890276 40 -r-x------ 1 atguigu atguigu 39283 Jul 16 13:53 ./__spark_libs__/metrics-jvm-3.1.5.jar
2890277 56 -r-x------ 1 atguigu atguigu 57071 Jul 16 13:53 ./__spark_libs__/spark-kvstore_2.12-2.4.5.jar
2890278 164 -r-x------ 1 atguigu atguigu 167421 Jul 16 13:53 ./__spark_libs__/jersey-client-2.22.2.jar
2890279 1976 -r-x------ 1 atguigu atguigu 2021167 Jul 16 13:53 ./__spark_libs__/snappy-java-1.1.7.3.jar
2890280 120 -r-x------ 1 atguigu atguigu 120465 Jul 16 13:53 ./__spark_libs__/metrics-core-3.1.5.jar
2890281 536 -r-x------ 1 atguigu atguigu 548430 Jul 16 13:53 ./__spark_libs__/scala-xml_2.12-1.0.5.jar
2890282 2340 -r-x------ 1 atguigu atguigu 2393456 Jul 16 13:53 ./__spark_libs__/spark-network-common_2.12-2.4.5.jar
2890283 360 -r-x------ 1 atguigu atguigu 365552 Jul 16 13:53 ./__spark_libs__/commons-compress-1.8.1.jar
2890284 12 -r-x------ 1 atguigu atguigu 10166 Jul 16 13:53 ./__spark_libs__/flatbuffers-1.2.0-3f79e055.jar
2890285 1140 -r-x------ 1 atguigu atguigu 1166637 Jul 16 13:53 ./__spark_libs__/jackson-databind-2.6.7.3.jar
2890286 1024 -r-x------ 1 atguigu atguigu 1048171 Jul 16 13:53 ./__spark_libs__/parquet-jackson-1.10.1.jar
2890287 316 -r-x------ 1 atguigu atguigu 320444 Jul 16 13:53 ./__spark_libs__/jackson-dataformat-yaml-2.6.7.jar
2890288 24 -r-x------ 1 atguigu atguigu 21247 Jul 16 13:53 ./__spark_libs__/metrics-graphite-3.1.5.jar
2890289 96 -r-x------ 1 atguigu atguigu 94995 Jul 16 13:53 ./__spark_libs__/parquet-common-1.10.1.jar
2890291 8 -r-x------ 1 atguigu atguigu 5950 Jul 16 13:53 ./__spark_libs__/javax.inject-2.4.0-b34.jar
2890292 832 -r-x------ 1 atguigu atguigu 848750 Jul 16 13:53 ./__spark_libs__/parquet-encoding-1.10.1.jar
2890293 2280 -r-x------ 1 atguigu atguigu 2333186 Jul 16 13:53 ./__spark_libs__/zstd-jni-1.3.2-2.jar
2890294 13008 -r-x------ 1 atguigu atguigu 13319481 Jul 16 13:53 ./__spark_libs__/breeze_2.12-0.13.2.jar
2890295 280 -r-x------ 1 atguigu atguigu 284184 Jul 16 13:53 ./__spark_libs__/commons-codec-1.10.jar
2890296 28 -r-x------ 1 atguigu atguigu 26366 Jul 16 13:53 ./__spark_libs__/javax.annotation-api-1.2.jar
2890297 84 -r-x------ 1 atguigu atguigu 83376 Jul 16 13:53 ./__spark_libs__/json4s-ast_2.12-3.5.3.jar
2890298 672 -r-x------ 1 atguigu atguigu 686815 Jul 16 13:53 ./__spark_libs__/kubernetes-client-4.6.1.jar
2890299 96 -r-x------ 1 atguigu atguigu 94796 Jul 16 13:53 ./__spark_libs__/pyrolite-4.13.jar
2890300 280 -r-x------ 1 atguigu atguigu 284220 Jul 16 13:53 ./__spark_libs__/commons-lang-2.6.jar
2890301 40 -r-x------ 1 atguigu atguigu 37871 Jul 16 13:53 ./__spark_libs__/commons-compiler-3.0.9.jar
2890302 124 -r-x------ 1 atguigu atguigu 125330 Jul 16 13:53 ./__spark_libs__/breeze-macros_2.12-0.13.2.jar
2890303 700 -r-x------ 1 atguigu atguigu 714194 Jul 16 13:53 ./__spark_libs__/javassist-3.18.1-GA.jar
2890304 256 -r-x------ 1 atguigu atguigu 258919 Jul 16 13:53 ./__spark_libs__/jackson-core-2.6.7.jar
2890305 88 -r-x------ 1 atguigu atguigu 88732 Jul 16 13:53 ./__spark_libs__/okio-1.15.0.jar
2890306 416 -r-x------ 1 atguigu atguigu 422786 Jul 16 13:53 ./__spark_libs__/okhttp-3.12.0.jar
2890307 576 -r-x------ 1 atguigu atguigu 588337 Jul 16 13:53 ./__spark_libs__/commons-collections-3.2.2.jar
2890308 480 -r-x------ 1 atguigu atguigu 489884 Jul 16 13:53 ./__spark_libs__/log4j-1.2.17.jar
2890309 952 -r-x------ 1 atguigu atguigu 971310 Jul 16 13:53 ./__spark_libs__/jersey-guava-2.22.2.jar
2890310 48 -r-x------ 1 atguigu atguigu 46986 Jul 16 13:53 ./__spark_libs__/jackson-annotations-2.6.7.jar
2890311 4 -r-x------ 1 atguigu atguigu 4028 Jul 16 13:53 ./__spark_libs__/shims-0.7.45.jar
2890312 76 -r-x------ 1 atguigu atguigu 75932 Jul 16 13:53 ./__spark_libs__/spark-launcher_2.12-2.4.5.jar
2890313 52 -r-x------ 1 atguigu atguigu 51164 Jul 16 13:53 ./__spark_libs__/spark-repl_2.12-2.4.5.jar
2890314 16 -r-x------ 1 atguigu atguigu 12486 Jul 16 13:53 ./__spark_libs__/logging-interceptor-3.12.0.jar
2890315 72 -r-x------ 1 atguigu atguigu 72733 Jul 16 13:53 ./__spark_libs__/jersey-media-jaxb-2.22.2.jar
2890316 72 -r-x------ 1 atguigu atguigu 70897 Jul 16 13:53 ./__spark_libs__/spark-network-shuffle_2.12-2.4.5.jar
2890317 480 -r-x------ 1 atguigu atguigu 489399 Jul 16 13:53 ./__spark_libs__/json4s-core_2.12-3.5.3.jar
2890318 20 -r-x------ 1 atguigu atguigu 19827 Jul 16 13:53 ./__spark_libs__/opencsv-2.3.jar
2890319 26772 -r-x------ 1 atguigu atguigu 27413468 Jul 16 13:53 ./__spark_libs__/hadoop-client-minicluster-3.1.3.jar
2890320 684 -r-x------ 1 atguigu atguigu 698375 Jul 16 13:53 ./__spark_libs__/jersey-common-2.22.2.jar
2890321 116 -r-x------ 1 atguigu atguigu 115534 Jul 16 13:53 ./__spark_libs__/javax.ws.rs-api-2.0.1.jar
2890322 6944 -r-x------ 1 atguigu atguigu 7108871 Jul 16 13:53 ./__spark_libs__/spark-catalyst_2.12-2.4.5.jar
2890323 3988 -r-x------ 1 atguigu atguigu 4080249 Jul 16 13:53 ./__spark_libs__/netty-all-4.1.42.Final.jar
2890324 1632 -r-x------ 1 atguigu atguigu 1671083 Jul 16 13:53 ./__spark_libs__/hppc-0.7.2.jar
2890325 8492 -r-x------ 1 atguigu atguigu 8694617 Jul 16 13:53 ./__spark_libs__/spark-core_2.12-2.4.5.jar
2890326 5640 -r-x------ 1 atguigu atguigu 5772768 Jul 16 13:53 ./__spark_libs__/spark-sql_2.12-2.4.5.jar
2890327 1524 -r-x------ 1 atguigu atguigu 1556863 Jul 16 13:53 ./__spark_libs__/avro-1.8.2.jar
2890328 192 -r-x------ 1 atguigu atguigu 195811 Jul 16 13:53 ./__spark_libs__/chill_2.12-0.9.3.jar
2890329 132 -r-x------ 1 atguigu atguigu 134595 Jul 16 13:53 ./__spark_libs__/commons-crypto-1.0.0.jar
2890330 320 -r-x------ 1 atguigu atguigu 326224 Jul 16 13:53 ./__spark_libs__/spark-yarn_2.12-2.4.5.jar
2890331 44 -r-x------ 1 atguigu atguigu 41203 Jul 16 13:53 ./__spark_libs__/slf4j-api-1.7.25.jar
2890332 472 -r-x------ 1 atguigu atguigu 479881 Jul 16 13:53 ./__spark_libs__/commons-lang3-3.5.jar
2890333 280 -r-x------ 1 atguigu atguigu 283653 Jul 16 13:53 ./__spark_libs__/curator-recipes-2.13.0.jar
2890334 100 -r-x------ 1 atguigu atguigu 101649 Jul 16 13:53 ./__spark_libs__/spark-mllib-local_2.12-2.4.5.jar
2890335 184 -r-x------ 1 atguigu atguigu 187052 Jul 16 13:53 ./__spark_libs__/avro-mapred-1.8.2-hadoop2.jar
2890336 8 -r-x------ 1 atguigu atguigu 5711 Jul 16 13:53 ./__spark_libs__/minlog-1.3.0.jar
2890337 28 -r-x------ 1 atguigu atguigu 27745 Jul 16 13:53 ./__spark_libs__/orc-shims-1.5.5.jar
2890338 64 -r-x------ 1 atguigu atguigu 65261 Jul 16 13:53 ./__spark_libs__/oro-2.0.8.jar
2890339 320 -r-x------ 1 atguigu atguigu 325335 Jul 16 13:53 ./__spark_libs__/RoaringBitmap-0.7.45.jar
2890340 12 -r-x------ 1 atguigu atguigu 8837 Jul 16 13:53 ./__spark_libs__/spark-tags_2.12-2.4.5-tests.jar
2890341 60 -r-x------ 1 atguigu atguigu 58633 Jul 16 13:53 ./__spark_libs__/chill-java-0.9.3.jar
2890342 280 -r-x------ 1 atguigu atguigu 282930 Jul 16 13:53 ./__spark_libs__/xbean-asm6-shaded-4.8.jar
2890343 280 -r-x------ 1 atguigu atguigu 285732 Jul 16 13:53 ./__spark_libs__/parquet-hadoop-1.10.1.jar
2890344 404 -r-x------ 1 atguigu atguigu 410874 Jul 16 13:53 ./__spark_libs__/kryo-shaded-4.0.2.jar
2890345 160 -r-x------ 1 atguigu atguigu 161867 Jul 16 13:53 ./__spark_libs__/stax2-api-3.1.4.jar
2890346 80 -r-x------ 1 atguigu atguigu 79845 Jul 16 13:53 ./__spark_libs__/compress-lzf-1.0.3.jar
2890347 36 -r-x------ 1 atguigu atguigu 35518 Jul 16 13:53 ./__spark_libs__/zjsonpatch-0.3.0.jar
2890348 2368 -r-x------ 1 atguigu atguigu 2423157 Jul 16 13:53 ./__spark_libs__/curator-client-2.13.0.jar
2890349 1024 -r-x------ 1 atguigu atguigu 1045744 Jul 16 13:53 ./__spark_libs__/leveldbjni-all-1.8.jar
2890350 328 -r-x------ 1 atguigu atguigu 334662 Jul 16 13:53 ./__spark_libs__/antlr4-runtime-4.7.jar
2890351 32 -r-x------ 1 atguigu atguigu 32612 Jul 16 13:53 ./__spark_libs__/jackson-module-jaxb-annotations-2.6.7.jar
2890352 120 -r-x------ 1 atguigu atguigu 122774 Jul 16 13:53 ./__spark_libs__/py4j-0.10.7.jar
2890353 616 -r-x------ 1 atguigu atguigu 627814 Jul 16 13:53 ./__spark_libs__/joda-time-2.9.3.jar
2890354 2764 -r-x------ 1 atguigu atguigu 2829843 Jul 16 13:53 ./__spark_libs__/shapeless_2.12-2.3.2.jar
2890355 2684 -r-x------ 1 atguigu atguigu 2747878 Jul 16 13:53 ./__spark_libs__/guava-27.0-jre.jar
2890356 64 -r-x------ 1 atguigu atguigu 63777 Jul 16 13:53 ./__spark_libs__/validation-api-1.1.0.Final.jar
2890357 52 -r-x------ 1 atguigu atguigu 49930 Jul 16 13:53 ./__spark_libs__/spark-unsafe_2.12-2.4.5.jar
2890358 5184 -r-x------ 1 atguigu atguigu 5305175 Jul 16 13:53 ./__spark_libs__/spark-mllib_2.12-2.4.5.jar
2890359 20 -r-x------ 1 atguigu atguigu 16430 Jul 16 13:53 ./__spark_libs__/jcl-over-slf4j-1.7.16.jar
2890360 340 -r-x------ 1 atguigu atguigu 346628 Jul 16 13:53 ./__spark_libs__/json4s-scalap_2.12-3.5.3.jar
2890361 96 -r-x------ 1 atguigu atguigu 95806 Jul 16 13:53 ./__spark_libs__/javax.servlet-api-3.1.0.jar
2890362 420 -r-x------ 1 atguigu atguigu 429742 Jul 16 13:53 ./__spark_libs__/spark-graphx_2.12-2.4.5.jar
2890363 180 -r-x------ 1 atguigu atguigu 181271 Jul 16 13:53 ./__spark_libs__/hk2-locator-2.4.0-b34.jar
2890364 1988 -r-x------ 1 atguigu atguigu 2035066 Jul 16 13:53 ./__spark_libs__/commons-math3-3.4.1.jar
2890365 796 -r-x------ 1 atguigu atguigu 812313 Jul 16 13:53 ./__spark_libs__/orc-mapreduce-1.5.5-nohive.jar
2890366 36 -r-x------ 1 atguigu atguigu 33015 Jul 16 13:53 ./__spark_libs__/jsr305-1.3.9.jar
2890367 16 -r-x------ 1 atguigu atguigu 14395 Jul 16 13:53 ./__spark_libs__/generex-1.0.2.jar
2890368 132 -r-x------ 1 atguigu atguigu 134044 Jul 16 13:53 ./__spark_libs__/aircompressor-0.10.jar
2890420 4 -rwx------ 1 atguigu atguigu 723 Jul 16 13:53 ./default_container_executor_session.sh
2890418 8 -rwx------ 1 atguigu atguigu 4848 Jul 16 13:53 ./launch_container.sh
2890370 4 drwx------ 3 atguigu atguigu 4096 Jul 16 13:53 ./__spark_conf__
2890405 180 -r-x------ 1 atguigu atguigu 183460 Jul 16 13:53 ./__spark_conf__/__spark_hadoop_conf__.xml
2890372 4 drwx------ 2 atguigu atguigu 4096 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__
2890398 4 -r-x------ 1 atguigu atguigu 21 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/httpfs-signature.secret
2890381 4 -r-x------ 1 atguigu atguigu 30 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/workers
2890384 4 -r-x------ 1 atguigu atguigu 2250 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/yarn-env.cmd
2890374 16 -r-x------ 1 atguigu atguigu 13326 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/log4j.properties
2890393 4 -r-x------ 1 atguigu atguigu 3414 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/hadoop-user-functions.sh.example
2890392 4 -r-x------ 1 atguigu atguigu 2642 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/user_ec_policies.xml.template
2890373 4 -r-x------ 1 atguigu atguigu 2591 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/yarnservice-log4j.properties
2890397 4 -r-x------ 1 atguigu atguigu 1484 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/httpfs-env.sh
2890377 4 -r-x------ 1 atguigu atguigu 1657 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/httpfs-log4j.properties
2890395 4 -r-x------ 1 atguigu atguigu 1351 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/kms-env.sh
2890385 12 -r-x------ 1 atguigu atguigu 10006 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/capacity-scheduler.xml
2890399 4 -r-x------ 1 atguigu atguigu 812 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/tez-site.xml
2890394 4 -r-x------ 1 atguigu atguigu 2316 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/ssl-client.xml.example
2890390 4 -r-x------ 1 atguigu atguigu 1940 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/container-executor.cfg
2890382 4 -r-x------ 1 atguigu atguigu 2211 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/core-site.xml
2890383 4 -r-x------ 1 atguigu atguigu 1747 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/kms-log4j.properties
2890388 8 -r-x------ 1 atguigu atguigu 6056 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/yarn-env.sh
2890380 4 -r-x------ 1 atguigu atguigu 2465 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/yarn-site.xml
2890375 4 -r-x------ 1 atguigu atguigu 1146 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/mapred-site.xml
2890402 4 -r-x------ 1 atguigu atguigu 2697 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/ssl-server.xml.example
2890391 4 -r-x------ 1 atguigu atguigu 1764 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/mapred-env.sh
2890396 12 -r-x------ 1 atguigu atguigu 11392 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/hadoop-policy.xml
2890404 8 -r-x------ 1 atguigu atguigu 4113 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/mapred-queues.xml.template
2890376 16 -r-x------ 1 atguigu atguigu 15903 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/hadoop-env.sh
2890400 4 -r-x------ 1 atguigu atguigu 620 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/httpfs-site.xml
2890379 4 -r-x------ 1 atguigu atguigu 3999 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/hadoop-env.cmd
2890401 4 -r-x------ 1 atguigu atguigu 951 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/mapred-env.cmd
2890386 4 -r-x------ 1 atguigu atguigu 3518 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/kms-acls.xml
2890389 4 -r-x------ 1 atguigu atguigu 897 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/hdfs-site.xml
2890378 4 -r-x------ 1 atguigu atguigu 3323 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/hadoop-metrics2.properties
2890403 4 -r-x------ 1 atguigu atguigu 682 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/kms-site.xml
2890387 4 -r-x------ 1 atguigu atguigu 1335 Jul 16 13:53 ./__spark_conf__/__hadoop_conf__/configuration.xsl
2890371 16 -r-x------ 1 atguigu atguigu 13326 Jul 16 13:53 ./__spark_conf__/log4j.properties
2890406 4 -r-x------ 1 atguigu atguigu 774 Jul 16 13:53 ./__spark_conf__/__spark_conf__.properties
2890419 4 -rw-r--r-- 1 atguigu atguigu 48 Jul 16 13:53 ./.launch_container.sh.crc
2890416 4 -rw-r--r-- 1 atguigu atguigu 74 Jul 16 13:53 ./container_tokens
broken symlinks(find -L . -maxdepth 5 -type l -ls):
网友评论