相关疑难解决方法(0)

我已经在线学习了一些教程,但他们不能Spark 1.5.1在OS X El Capitan上工作(10.11)

基本上我已经运行了这个命令下载 apache-spark

brew update
brew install scala
brew install apache-spark

更新了.bash_profile

# For a ipython notebook and pyspark integration
if which pyspark > /dev/null; then
  export SPARK_HOME="/usr/local/Cellar/apache-spark/1.5.1/libexec/"
  export PYSPARK_SUBMIT_ARGS="--master local[2]"
fi

跑

ipython profile create pyspark

创建了~/.ipython/profile_pyspark/startup/00-pyspark-setup.py以这种方式配置的启动文件

# Configure the necessary Spark environment
import os
import sys

# Spark home
spark_home = os.environ.get("SPARK_HOME")

# If Spark V1.4.x is detected, then add ' pyspark-shell' to
# the end of the 'PYSPARK_SUBMIT_ARGS' environment …