美文网首页大数据开发
Spark 运行警告:spark.debug.maxToStri

Spark 运行警告:spark.debug.maxToStri

作者: 晨冉1688 | 来源:发表于2020-05-01 11:55 被阅读0次

spark运行报错

Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.

可以设置一个参数.config("spark.debug.maxToStringFields", "100")

SparkSession
        .builder()
        .master("local[*]")
        .appName("report")
        .config("spark.debug.maxToStringFields", "100")
        .getOrCreate();

相关文章

网友评论

    本文标题:Spark 运行警告:spark.debug.maxToStri

    本文链接:https://www.haomeiwen.com/subject/tonfghtx.html