我正在尝试按照 AWS文档在我的 ubuntu 虚拟机上设置 AWS Glue 环境。
我已经完成了必要的操作,例如下载 awsglue libs、spark 包并按照建议设置 Spark home。之后,我无法初始化粘合上下文并面临以下错误。
from awsglue.context import GlueContext
from pyspark.context import SparkContext
glueContext = GlueContext(SparkContext.getOrCreate())
or
glueContext = GlueContext(sc)
Run Code Online (Sandbox Code Playgroud)
错误:
TypeError Traceback (most recent call last)
<ipython-input-15-0798793d4033> in <module>
----> 1 glueContext = GlueContext(SparkContext.getOrCreate())
~/aws-glue-libs-glue-1.0/PyGlue.zip/awsglue/context.py in __init__(self, sparkContext, **options)
43 super(GlueContext, self).__init__(sparkContext)
44 register(sparkContext)
---> 45 self._glue_scala_context = self._get_glue_scala_context(**options)
46 self.create_dynamic_frame = DynamicFrameReader(self)
47 self.write_dynamic_frame = DynamicFrameWriter(self)
~/aws-glue-libs-glue-1.0/PyGlue.zip/awsglue/context.py in _get_glue_scala_context(self, **options)
64
65 if min_partitions is None:
---> 66 …Run Code Online (Sandbox Code Playgroud)