我在针对本地运行的 Spark 独立集群启动 spark shell 时遇到问题。有任何想法吗?我在 spark 3.1.0-SNAPSHOT 上运行它。
在本地模式下启动 shell 或常规应用程序工作正常,但都失败并显示以下命令。
$ ./bin/spark-shell --master spark://localhost:8080
...
20/04/05 00:34:47 WARN StandaloneAppClient$ClientEndpoint: Could not connect to localhost:8080: java.lang.IllegalArgumentException: Too large frame: 5211883372140375593
20/04/05 00:34:47 WARN StandaloneAppClient$ClientEndpoint: Failed to connect to master localhost:8080
org.apache.spark.SparkException: Exception thrown in awaitResult:
at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:303)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:101)
at org.apache.spark.rpc.RpcEnv.setupEndpointRef(RpcEnv.scala:109)
at org.apache.spark.deploy.client.StandaloneAppClient$ClientEndpoint$$anon$1.run(StandaloneAppClient.scala:106)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Too large frame: 5211883372140375593
at org.sparkproject.guava.base.Preconditions.checkArgument(Preconditions.java:119)
at org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:148)
at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:98)
at …Run Code Online (Sandbox Code Playgroud) apache-spark ×1