org.apache.spark.SparkException：写入行时任务失败

Question

我收到以下代码，试图将 DataFrame 保存到提供的路径

df.write.mode(SaveMode.Append).parquet("/path/year=2018")

使用上述代码时出现以下异常。我正在使用 Spark 2.2.0。我是 Spark 的新手。

org.apache.spark.SparkException: Task failed while writing rows at
    org.apache.spark.sql.execution.datasources.FileFormatWriter$.org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask(FileFormatWriter.scala:270)
    at
    org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$write$1$$anonfun$apply$mcV$sp$1.apply(FileFormatWriter.scala:189)
    at
    org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$write$1$$anonfun$apply$mcV$sp$1.apply(FileFormatWriter.scala:188)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at
    org.apache.spark.scheduler.Task.run(Task.scala:108) at
    org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:338) at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745) 

Caused by:  java.lang.NullPointerException at
        org.apache.spark.sql.SparkSession$$anonfun$3.apply(SparkSession.scala:468)
        at
        org.apache.spark.sql.SparkSession$$anonfun$3.apply(SparkSession.scala:468)
        at scala.collection.Iterator$$anon$11.next(Iterator.scala:409) at
        scala.collection.Iterator$$anon$11.next(Iterator.scala:409) at
        org.apache.spark.sql.execution.datasources.FileFormatWriter$SingleDirectoryWriteTask.execute(FileFormatWriter.scala:324)
        at
        org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask$3.apply(FileFormatWriter.scala:256)
        at
        org.apache.spark.sql.execution.datasources.FileFormatWriter$$anonfun$org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask$3.apply(FileFormatWriter.scala:254)
        at
        org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1371)
        at
        org.apache.spark.sql.execution.datasources.FileFormatWriter$.org$apache$spark$sql$execution$datasources$FileFormatWriter$$executeTask(FileFormatWriter.scala:259)