美文网首页
Exception: Python in worker has

Exception: Python in worker has

作者: zh_harry | 来源:发表于2020-11-04 15:11 被阅读0次
    Exception: Python in worker has different version 3.7 than that in driver 3.5, PySpark cannot run with different minor versions.Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set.
    
        at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.handlePythonException(PythonRunner.scala:456)
        at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:592)
        at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:575)
        at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:410)
        at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
        at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
        at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
        at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
        at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
    

    第一种方法:

    image image

    到这里,配置以后就可以了。

    第二种方法:以上这种方法只是针对前工程,如果你想对所有工程都起作用,需要到环境变量配置PYSPARK_PYTHON如下所示:

    image

    相关文章

      网友评论

          本文标题:Exception: Python in worker has

          本文链接:https://www.haomeiwen.com/subject/tbegvktx.html