相关疑难解决方法(0)

我正在使用spark over emr并编写一个pyspark脚本,我在尝试时遇到错误

from pyspark import SparkContext
sc = SparkContext()

这是错误

File "pyex.py", line 5, in <module>
    sc = SparkContext()   File "/usr/local/lib/python3.4/site-packages/pyspark/context.py", line 118, in __init__
    conf, jsc, profiler_cls)   File "/usr/local/lib/python3.4/site-packages/pyspark/context.py", line 195, in _do_init
    self._encryption_enabled = self._jvm.PythonUtils.getEncryptionEnabled(self._jsc)   File "/usr/local/lib/python3.4/site-packages/py4j/java_gateway.py", line 1487, in __getattr__
    "{0}.{1} does not exist in the JVM".format(self._fqn, name)) py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM

我发现这个答案说明我需要导入sparkcontext,但这也不起作用.

我正在尝试在 jupyter 笔记本中创建 SparkContext 但出现以下错误：

Py4JError：JVM中不存在org.apache.spark.api.python.PythonUtils.getPythonAuthSocketTimeout

这是我的代码

from pyspark import SparkContext, SparkConf
conf = SparkConf().setMaster("local").setAppName("Groceries")
sc = SparkContext(conf = conf)


Py4JError                                 Traceback (most recent call last)
<ipython-input-20-5058f350f58a> in <module>
      1 conf = SparkConf().setMaster("local").setAppName("My App")
----> 2 sc = SparkContext(conf = conf)

~/Documents/python38env/lib/python3.8/site-packages/pyspark/context.py in __init__(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls)
    144         SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
    145         try:
--> 146             self._do_init(master, appName, sparkHome, pyFiles, environment, batchSize, serializer,
    147                           conf, jsc, profiler_cls)
    148         except:

~/Documents/python38env/lib/python3.8/site-packages/pyspark/context.py in _do_init(self, master, …