我尝试从本地计算机上的笔记本连接到远程 Spark Master。
当我尝试创建 SparkContext 时
sc = pyspark.SparkContext(master = "spark://remote-spark-master-hostname:7077",
appName="jupyter notebook_test"),
Run Code Online (Sandbox Code Playgroud)
我收到以下异常:
/opt/.venv/lib/python3.7/site-packages/pyspark/context.py in __init__(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls)
134 try:
135 self._do_init(master, appName, sparkHome, pyFiles, environment, batchSize, serializer,
--> 136 conf, jsc, profiler_cls)
137 except:
138 # If an error occurs, clean up in order to allow future SparkContext creation:
/opt/.venv/lib/python3.7/site-packages/pyspark/context.py in _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, jsc, profiler_cls)
196
197 # Create the Java SparkContext …Run Code Online (Sandbox Code Playgroud)