jupyter notebook NameError：未定义名称“sc”

Question

我使用了 jupyter notebook，pyspark，然后，我的第一个命令是：

rdd = sc.parallelize([2, 3, 4])

然后，它表明

NameError Traceback (most recent call last)
<ipython-input-1-c540c4a1d203> in <module>()
----> 1 rdd = sc.parallelize([2, 3, 4])

NameError: name 'sc' is not defined.

未定义如何修复此错误“sc”。

Answer 1

Have you initialized the SparkContext?

You could try this:

#Initializing PySpark
from pyspark import SparkContext, SparkConf

# #Spark Config
conf = SparkConf().setAppName("sample_app")
sc = SparkContext(conf=conf)