美文网首页大数据开发
Spark 运行警告:spark.debug.maxToStri

Spark 运行警告:spark.debug.maxToStri

作者: 晨冉1688 | 来源:发表于2020-05-01 11:55 被阅读0次

    spark运行报错

    Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.

    可以设置一个参数.config("spark.debug.maxToStringFields", "100")

    SparkSession
            .builder()
            .master("local[*]")
            .appName("report")
            .config("spark.debug.maxToStringFields", "100")
            .getOrCreate();
    

    相关文章

      网友评论

        本文标题:Spark 运行警告:spark.debug.maxToStri

        本文链接:https://www.haomeiwen.com/subject/tonfghtx.html