美文网首页
Spark 错误日志 java.lang.NoClassDefF

Spark 错误日志 java.lang.NoClassDefF

作者: 李飞_fd28 | 来源:发表于2019-12-26 18:46 被阅读0次
    
    Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
    19/12/26 17:43:21 ERROR SpringApplication: Application run failed
    java.lang.NoClassDefFoundError: org/codehaus/janino/InternalCompilerException
        at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:1370)
        at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1466)
        at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1463)
        at org.sparkproject.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
        at org.sparkproject.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
        at org.sparkproject.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)
        at org.sparkproject.guava.cache.LocalCache$Segment.get(LocalCache.java:2257)
        at org.sparkproject.guava.cache.LocalCache.get(LocalCache.java:4000)
        at org.sparkproject.guava.cache.LocalCache.getOrLoad(LocalCache.java:4004)
        at org.sparkproject.guava.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)
        at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.compile(CodeGenerator.scala:1317)
        at org.apache.spark.sql.execution.WholeStageCodegenExec.liftedTree1$1(WholeStageCodegenExec.scala:692)
        at org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:691)
        at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:189)
        at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:227)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
        at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:224)
        at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:185)
        at org.apache.spark.sql.execution.SparkPlan.getByteArrayRdd(SparkPlan.scala:329)
        at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:421)
        at org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:47)
        at org.apache.spark.sql.Dataset.collectFromPlan(Dataset.scala:3417)
        at org.apache.spark.sql.Dataset.$anonfun$head$1(Dataset.scala:2516)
        at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3407)
        at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$4(SQLExecution.scala:100)
        at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:160)
        at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:87)
        at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3403)
        at org.apache.spark.sql.Dataset.head(Dataset.scala:2516)
        at org.apache.spark.sql.Dataset.take(Dataset.scala:2723)
        at org.apache.spark.sql.Dataset.getRows(Dataset.scala:297)
        at org.apache.spark.sql.Dataset.showString(Dataset.scala:334)
        at org.apache.spark.sql.Dataset.show(Dataset.scala:799)
        at org.apache.spark.sql.Dataset.show(Dataset.scala:758)
        at org.apache.spark.sql.Dataset.show(Dataset.scala:767)
        at com.aeotrade.datacenter.dec.spark.DecSparkService.doStat(DecSparkService.java:86)
        at com.aeotrade.datacenter.DatacenterDecSparkApplication.run(DatacenterDecSparkApplication.java:36)
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:784)
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:768)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:322)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
        at com.aeotrade.datacenter.DatacenterDecSparkApplication.main(DatacenterDecSparkApplication.java:23)
    Caused by: java.lang.ClassNotFoundException: org.codehaus.janino.InternalCompilerException
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 43 more
    19/12/26 17:43:21 INFO SparkUI: Stopped Spark web UI at http://lifeidembp:4040
    19/12/26 17:43:21 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
    

    解决方法:

     <dependency>
          <groupId>org.codehaus.janino</groupId>
          <artifactId>janino</artifactId>
          <version>3.0.8</version>
        </dependency>
    

    相关文章

      网友评论

          本文标题:Spark 错误日志 java.lang.NoClassDefF

          本文链接:https://www.haomeiwen.com/subject/aqsloctx.html