小编Pan*_*chu的帖子

使用count()和first()时,iPython笔记本中的PySpark会引发Py4JJavaError

我在iPython笔记本(python v.3.6)中使用PySpark(v.2.1.0)而不是我的Mac(Sierra 10.12.3 Beta)中的virtualenv.

1.我在终端拍摄推出了iPython笔记本 -

 PYSPARK_PYTHON=python3 PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS="notebook" /Applications/spark-2.1.0-bin-hadoop2.7/bin/pyspark
Run Code Online (Sandbox Code Playgroud)

2.将我的文件加载到Spark Context并确保其加载 -

>>>lines = sc.textFile("/Users/PanchusMac/Dropbox/Learn_py/Virtual_Env/pyspark/README.md") 

>>>for i in lines.collect(): 
    print(i)
Run Code Online (Sandbox Code Playgroud)

它工作正常并将结果打印在我的控制台上,如下所示:

# Apache Spark

Spark is a fast and general cluster computing system for Big Data. It provides
high-level APIs in Scala, Java, Python, and R, and an optimized engine that
supports general computation graphs for data analysis. It also supports a
rich set of higher-level tools including Spark SQL for SQL and DataFrames,
MLlib for machine learning, GraphX …
Run Code Online (Sandbox Code Playgroud)

python virtualenv ipython-notebook apache-spark pyspark

13
推荐指数
3
解决办法
2万
查看次数

无法从 Mac 终端执行 jshell

我目前在我的 Mac Sierra 上使用 java 1.8。

java -version
java version "1.8.0_51"
Java(TM) SE Runtime Environment (build 1.8.0_51-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)
Run Code Online (Sandbox Code Playgroud)

现在我想运行“jshell”来直接从终端执行简单的java命令。但我最终得到了以下错误消息 -

Unable to locate an executable at "/Library/Java/JavaVirtualMachines/jdk1.8.0_51.jdk/Contents/Home/bin/jshell" (-1)
Run Code Online (Sandbox Code Playgroud)

我需要采取什么解决方法来避免出现此错误消息?

java macos terminal java-9 jshell

5
推荐指数
1
解决办法
1688
查看次数