那些遇到过的问题

如何在本地模式下运行 Spark SQL Thrift Server 并使用 JDBC 连接到 Delta

Jef*_*uti 5 apache-spark apache-spark-sql spark-thriftserver delta-lake

我想使用 JDBC 连接到 Delta，并想在本地模式下运行 Spark Thrift Server (STS) 以进行测试。

我使用以下命令启动 STS：

$SPARK_HOME/sbin/start-thriftserver.sh \
  --conf spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension \
  --conf spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog \
  --packages 'io.delta:delta-core_2.12:1.0.0'
Run Code Online (Sandbox Code Playgroud)

尝试使用 beeline 连接时出现以下错误：

Beeline version 2.3.7 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000

Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000:
Enter password for jdbc:hive2://localhost:10000: 

Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000: Failed to open new session: org.apache.spark.SparkException: Cannot find catalog plugin class for catalog 'spark_catalog': org.apache.spark.sql.delta.catalog.DeltaCatalog (state=08S01,code=0)
Run Code Online (Sandbox Code Playgroud)

查看 thrift 服务器日志，我没有看到任何明显的错误，并且 delta jar 文件的加载也没有任何错误。

Spark Command: /Users/sandbox/.sdkman/candidates/java/8.0.282.j9-adpt/bin/java -cp /Users/sandbox/projects/delta/spark-3.1.2-bin-hadoop3.2/conf/:/Users/sandbox/projects/delta/spark-3.1.2-bin-hadoop3.2/jars/* -Xmx1g org.apache.spark.deploy.SparkSubmit --conf spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension --conf spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --name Thrift JDBC/ODBC Server --packages io.delta:delta-core_2.12:1.0.0 spark-internal
========================================
:: loading settings :: url = jar:file:/Users/sandbox/projects/delta/spark-3.1.2-bin-hadoop3.2/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
Ivy Default Cache set to: /Users/sandbox/.ivy2/cache
The jars for the packages stored in: /Users/sandbox/.ivy2/jars
io.delta#delta-core_2.12 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-8f65b233-668d-46ea-98d6-0da629b12d1e;1.0
        confs: [default]
        found io.delta#delta-core_2.12;1.0.0 in central
        found org.antlr#antlr4;4.7 in central
        found org.antlr#antlr4-runtime;4.7 in central
        found org.antlr#antlr-runtime;3.5.2 in central
        found org.antlr#ST4;4.0.8 in central
        found org.abego.treelayout#org.abego.treelayout.core;1.0.3 in central
        found org.glassfish#javax.json;1.0.4 in central
        found com.ibm.icu#icu4j;58.2 in central
:: resolution report :: resolve 385ms :: artifacts dl 13ms
        :: modules in use:
        com.ibm.icu#icu4j;58.2 from central in [default]
        io.delta#delta-core_2.12;1.0.0 from central in [default]
        org.abego.treelayout#org.abego.treelayout.core;1.0.3 from central in [default]
        org.antlr#ST4;4.0.8 from central in [default]
        org.antlr#antlr-runtime;3.5.2 from central in [default]
        org.antlr#antlr4;4.7 from central in [default]
        org.antlr#antlr4-runtime;4.7 from central in [default]
        org.glassfish#javax.json;1.0.4 from central in [default]
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent-8f65b233-668d-46ea-98d6-0da629b12d1e
        confs: [default]
        0 artifacts copied, 8 already retrieved (0kB/13ms)
21/11/06 07:28:30 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
21/11/06 07:28:30 INFO HiveThriftServer2: Started daemon with process name: 75591@MacBook-Pro
21/11/06 07:28:30 INFO SignalUtils: Registering signal handler for TERM
21/11/06 07:28:30 INFO SignalUtils: Registering signal handler for HUP
21/11/06 07:28:30 INFO SignalUtils: Registering signal handler for INT
21/11/06 07:28:30 INFO HiveThriftServer2: Starting SparkContext
21/11/06 07:28:31 INFO HiveConf: Found configuration file null
21/11/06 07:28:31 INFO SparkContext: Running Spark version 3.1.2
21/11/06 07:28:31 INFO ResourceUtils: ==============================================================
21/11/06 07:28:31 INFO ResourceUtils: No custom resources configured for spark.driver.
21/11/06 07:28:31 INFO ResourceUtils: ==============================================================
21/11/06 07:28:31 INFO SparkContext: Submitted application: Thrift JDBC/ODBC Server
21/11/06 07:28:31 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 1, script: , vendor: , memory -> name: memory, amount: 1024, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)
21/11/06 07:28:31 INFO ResourceProfile: Limiting resource is cpu
21/11/06 07:28:31 INFO ResourceProfileManager: Added ResourceProfile id: 0
...
21/11/06 07:28:31 INFO Utils: Successfully started service 'sparkDriver' on port 55507.
...
21/11/06 07:28:32 INFO Utils: Successfully started service 'SparkUI' on port 4040.
21/11/06 07:28:32 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://macbook-pro:4040
21/11/06 07:28:32 INFO SparkContext: Added JAR file:///Users/sandbox/.ivy2/jars/io.delta_delta-core_2.12-1.0.0.jar at spark://macbook-pro:55507/jars/io.delta_delta-core_2.12-1.0.0.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO SparkContext: Added JAR file:///Users/sandbox/.ivy2/jars/org.antlr_antlr4-4.7.jar at spark://macbook-pro:55507/jars/org.antlr_antlr4-4.7.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO SparkContext: Added JAR file:///Users/sandbox/.ivy2/jars/org.antlr_antlr4-runtime-4.7.jar at spark://macbook-pro:55507/jars/org.antlr_antlr4-runtime-4.7.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO SparkContext: Added JAR file:///Users/sandbox/.ivy2/jars/org.antlr_antlr-runtime-3.5.2.jar at spark://macbook-pro:55507/jars/org.antlr_antlr-runtime-3.5.2.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO SparkContext: Added JAR file:///Users/sandbox/.ivy2/jars/org.antlr_ST4-4.0.8.jar at spark://macbook-pro:55507/jars/org.antlr_ST4-4.0.8.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO SparkContext: Added JAR file:///Users/sandbox/.ivy2/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar at spark://macbook-pro:55507/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO SparkContext: Added JAR file:///Users/sandbox/.ivy2/jars/org.glassfish_javax.json-1.0.4.jar at spark://macbook-pro:55507/jars/org.glassfish_javax.json-1.0.4.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO SparkContext: Added JAR file:///Users/sandbox/.ivy2/jars/com.ibm.icu_icu4j-58.2.jar at spark://macbook-pro:55507/jars/com.ibm.icu_icu4j-58.2.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO SparkContext: Added file file:///Users/sandbox/.ivy2/jars/io.delta_delta-core_2.12-1.0.0.jar at file:///Users/sandbox/.ivy2/jars/io.delta_delta-core_2.12-1.0.0.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: Copying /Users/sandbox/.ivy2/jars/io.delta_delta-core_2.12-1.0.0.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/io.delta_delta-core_2.12-1.0.0.jar
21/11/06 07:28:32 INFO SparkContext: Added file file:///Users/sandbox/.ivy2/jars/org.antlr_antlr4-4.7.jar at file:///Users/sandbox/.ivy2/jars/org.antlr_antlr4-4.7.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: Copying /Users/sandbox/.ivy2/jars/org.antlr_antlr4-4.7.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr4-4.7.jar
21/11/06 07:28:32 INFO SparkContext: Added file file:///Users/sandbox/.ivy2/jars/org.antlr_antlr4-runtime-4.7.jar at file:///Users/sandbox/.ivy2/jars/org.antlr_antlr4-runtime-4.7.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: Copying /Users/sandbox/.ivy2/jars/org.antlr_antlr4-runtime-4.7.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr4-runtime-4.7.jar
21/11/06 07:28:32 INFO SparkContext: Added file file:///Users/sandbox/.ivy2/jars/org.antlr_antlr-runtime-3.5.2.jar at file:///Users/sandbox/.ivy2/jars/org.antlr_antlr-runtime-3.5.2.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: Copying /Users/sandbox/.ivy2/jars/org.antlr_antlr-runtime-3.5.2.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr-runtime-3.5.2.jar
21/11/06 07:28:32 INFO SparkContext: Added file file:///Users/sandbox/.ivy2/jars/org.antlr_ST4-4.0.8.jar at file:///Users/sandbox/.ivy2/jars/org.antlr_ST4-4.0.8.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: Copying /Users/sandbox/.ivy2/jars/org.antlr_ST4-4.0.8.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_ST4-4.0.8.jar
21/11/06 07:28:32 INFO SparkContext: Added file file:///Users/sandbox/.ivy2/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar at file:///Users/sandbox/.ivy2/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: Copying /Users/sandbox/.ivy2/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar
21/11/06 07:28:32 INFO SparkContext: Added file file:///Users/sandbox/.ivy2/jars/org.glassfish_javax.json-1.0.4.jar at file:///Users/sandbox/.ivy2/jars/org.glassfish_javax.json-1.0.4.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: Copying /Users/sandbox/.ivy2/jars/org.glassfish_javax.json-1.0.4.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.glassfish_javax.json-1.0.4.jar
21/11/06 07:28:32 INFO SparkContext: Added file file:///Users/sandbox/.ivy2/jars/com.ibm.icu_icu4j-58.2.jar at file:///Users/sandbox/.ivy2/jars/com.ibm.icu_icu4j-58.2.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: Copying /Users/sandbox/.ivy2/jars/com.ibm.icu_icu4j-58.2.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/com.ibm.icu_icu4j-58.2.jar
21/11/06 07:28:32 INFO Executor: Starting executor ID driver on host macbook-pro
21/11/06 07:28:32 INFO Executor: Fetching file:///Users/sandbox/.ivy2/jars/org.antlr_antlr4-runtime-4.7.jar with timestamp 1636183711298
21/11/06 07:28:32 INFO Utils: /Users/sandbox/.ivy2/jars/org.antlr_antlr4-runtime-4.7.jar has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr4-runtime-4.7.jar
21/11/06 07:28:32 INFO Executor: Fetching file:///Users/sandbox/.ivy2/jars/com.ibm.icu_icu4j-58.2.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: /Users/sandbox/.ivy2/jars/com.ibm.icu_icu4j-58.2.jar has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/com.ibm.icu_icu4j-58.2.jar
21/11/06 07:28:33 INFO Executor: Fetching file:///Users/sandbox/.ivy2/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: /Users/sandbox/.ivy2/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar
21/11/06 07:28:33 INFO Executor: Fetching file:///Users/sandbox/.ivy2/jars/org.glassfish_javax.json-1.0.4.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: /Users/sandbox/.ivy2/jars/org.glassfish_javax.json-1.0.4.jar has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.glassfish_javax.json-1.0.4.jar
21/11/06 07:28:33 INFO Executor: Fetching file:///Users/sandbox/.ivy2/jars/org.antlr_antlr-runtime-3.5.2.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: /Users/sandbox/.ivy2/jars/org.antlr_antlr-runtime-3.5.2.jar has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr-runtime-3.5.2.jar
21/11/06 07:28:33 INFO Executor: Fetching file:///Users/sandbox/.ivy2/jars/org.antlr_antlr4-4.7.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: /Users/sandbox/.ivy2/jars/org.antlr_antlr4-4.7.jar has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr4-4.7.jar
21/11/06 07:28:33 INFO Executor: Fetching file:///Users/sandbox/.ivy2/jars/org.antlr_ST4-4.0.8.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: /Users/sandbox/.ivy2/jars/org.antlr_ST4-4.0.8.jar has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_ST4-4.0.8.jar
21/11/06 07:28:33 INFO Executor: Fetching file:///Users/sandbox/.ivy2/jars/io.delta_delta-core_2.12-1.0.0.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: /Users/sandbox/.ivy2/jars/io.delta_delta-core_2.12-1.0.0.jar has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/io.delta_delta-core_2.12-1.0.0.jar
21/11/06 07:28:33 INFO Executor: Fetching spark://macbook-pro:55507/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO TransportClientFactory: Successfully created connection to MacBook-Pro/192.168.1.207:55507 after 53 ms (0 ms spent in bootstraps)
21/11/06 07:28:33 INFO Utils: Fetching spark://macbook-pro:55507/jars/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp7179925225828426556.tmp
21/11/06 07:28:33 INFO Utils: /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp7179925225828426556.tmp has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar
21/11/06 07:28:33 INFO Executor: Adding file:/private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.abego.treelayout_org.abego.treelayout.core-1.0.3.jar to class loader
21/11/06 07:28:33 INFO Executor: Fetching spark://macbook-pro:55507/jars/org.glassfish_javax.json-1.0.4.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: Fetching spark://macbook-pro:55507/jars/org.glassfish_javax.json-1.0.4.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp6953292115063484853.tmp
21/11/06 07:28:33 INFO Utils: /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp6953292115063484853.tmp has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.glassfish_javax.json-1.0.4.jar
21/11/06 07:28:33 INFO Executor: Adding file:/private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.glassfish_javax.json-1.0.4.jar to class loader
21/11/06 07:28:33 INFO Executor: Fetching spark://macbook-pro:55507/jars/org.antlr_ST4-4.0.8.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: Fetching spark://macbook-pro:55507/jars/org.antlr_ST4-4.0.8.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp7481622034847054241.tmp
21/11/06 07:28:33 INFO Utils: /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp7481622034847054241.tmp has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_ST4-4.0.8.jar
21/11/06 07:28:33 INFO Executor: Adding file:/private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_ST4-4.0.8.jar to class loader
21/11/06 07:28:33 INFO Executor: Fetching spark://macbook-pro:55507/jars/org.antlr_antlr4-runtime-4.7.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: Fetching spark://macbook-pro:55507/jars/org.antlr_antlr4-runtime-4.7.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp8137443686268894230.tmp
21/11/06 07:28:33 INFO Utils: /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp8137443686268894230.tmp has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr4-runtime-4.7.jar
21/11/06 07:28:33 INFO Executor: Adding file:/private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr4-runtime-4.7.jar to class loader
21/11/06 07:28:33 INFO Executor: Fetching spark://macbook-pro:55507/jars/org.antlr_antlr4-4.7.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: Fetching spark://macbook-pro:55507/jars/org.antlr_antlr4-4.7.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp7474775280611839356.tmp
21/11/06 07:28:33 INFO Utils: /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp7474775280611839356.tmp has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr4-4.7.jar
21/11/06 07:28:33 INFO Executor: Adding file:/private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr4-4.7.jar to class loader
21/11/06 07:28:33 INFO Executor: Fetching spark://macbook-pro:55507/jars/io.delta_delta-core_2.12-1.0.0.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: Fetching spark://macbook-pro:55507/jars/io.delta_delta-core_2.12-1.0.0.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp7899823112197045994.tmp
21/11/06 07:28:33 INFO Utils: /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp7899823112197045994.tmp has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/io.delta_delta-core_2.12-1.0.0.jar
21/11/06 07:28:33 INFO Executor: Adding file:/private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/io.delta_delta-core_2.12-1.0.0.jar to class loader
21/11/06 07:28:33 INFO Executor: Fetching spark://macbook-pro:55507/jars/org.antlr_antlr-runtime-3.5.2.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: Fetching spark://macbook-pro:55507/jars/org.antlr_antlr-runtime-3.5.2.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp1438945171356535536.tmp
21/11/06 07:28:33 INFO Executor: Adding file:/private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/org.antlr_antlr-runtime-3.5.2.jar to class loader
21/11/06 07:28:33 INFO Executor: Fetching spark://macbook-pro:55507/jars/com.ibm.icu_icu4j-58.2.jar with timestamp 1636183711298
21/11/06 07:28:33 INFO Utils: Fetching spark://macbook-pro:55507/jars/com.ibm.icu_icu4j-58.2.jar to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp836117221049857229.tmp
21/11/06 07:28:33 INFO Utils: /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/fetchFileTemp836117221049857229.tmp has been previously copied to /private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/com.ibm.icu_icu4j-58.2.jar
21/11/06 07:28:33 INFO Executor: Adding file:/private/var/folders/qp/g1z81/T/spark-1270d/userFiles-2c8e44aa/com.ibm.icu_icu4j-58.2.jar to class loader
21/11/06 07:28:33 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 55509.
21/11/06 07:28:33 INFO NettyBlockTransferService: Server created on macbook-pro:55509
...
21/11/06 07:28:33 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/Users/sandbox/projects/delta/spark-warehouse/').
21/11/06 07:28:33 INFO SharedState: Warehouse path is 'file:/Users/sandbox/projects/delta/spark-warehouse/'.
21/11/06 07:28:34 INFO HiveUtils: Initializing HiveMetastoreConnection ve

Jee*_*ali 3

一旦您可以将 io.delta:delta-core_2.12:1.0.0 JAR 文件复制到 $SPARK_HOME/lib 并重新启动，此错误就会消失。

归档时间：

查看次数：

3161 次

最近记录：

1 年，11 月 前
相关归档
如何充分利用集群中的所有Spark节点？ 14
更新apache镶木地板文件中的值 14
如何将整列的大小写改为小写？ 14
本地类不兼容异常:从IDE运行spark standalone时 12
在Python中使用Spark DataFrame创建labeledPoints 11
Pyspark 使用架构读取 csv、标头检查并存储损坏的记录 10
Spark Streaming 2.2.0的NoSuchMethodError.和卡夫卡0.8 6
Spark：在不更改列的可为空属性的情况下转换小数 6
如何在 Spark 中将字符串解析为数组？ 3
将拆分与数据帧一起使用时出现未找到的错误 2
难疑归档
如何有效地迭代Java Map中的每个条目？ 3113
不区分大小写'包含(字符串)' 2785
如何修复android.os.NetworkOnMainThreadException？ 2308
如何在Python中连接两个列表？ 2250
使用jQuery从下拉列表(选择框)中获取所选文本 2240
Git拒绝合并关于rebase的无关历史 1862
JavaScript切断/切片/修剪字符串中的最后一个字符 1840
你什么时候使用git rebase而不是git merge？ 1461
Markdown中的评论 1277
如何使用$ scope.$ watch和$ scope.$在AngularJS中申请？ 1076