小编tam*_*ama的帖子

由async-http-client引起的Spark中的IllegalAccessError

上下文:我正在开发一个Spark流工作,使用这个库将数据写入InfluxDB .这是环境.

  • 斯卡拉2.11.8
  • Spark 2.1.0(Dockerized独立集群)

相关依赖:

"org.apache.spark" %% "spark-core" % "2.1.0" % "provided",
"org.apache.spark" %% "spark-streaming" % "2.1.0" % "provided",
"org.apache.spark" %% "spark-streaming-kafka-0-8" % "2.1.0",
"com.paulgoldbaum" %% "scala-influxdb-client" % "0.5.2" // which uses "org.asynchttpclient" % "async-http-client" % "2.0.24"
Run Code Online (Sandbox Code Playgroud)

一切都在我的本地计算机上编译并运行良好,但是当我将程序集jar提交给Spark集群时,我在驱动程序中收到此错误:

Exception in thread "main" java.lang.reflect.InvocationTargetException
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at org.apache.spark.deploy.worker.DriverWrapper$.main(DriverWrapper.scala:58)
  at org.apache.spark.deploy.worker.DriverWrapper.main(DriverWrapper.scala)
Caused by: java.lang.IllegalAccessError: tried to access field io.netty.handler.ssl.JdkSslContext.SUPPORTED_CIPHERS from class io.netty.handler.ssl.NettySslPackageAccessor
  at io.netty.handler.ssl.NettySslPackageAccessor.jdkSupportedCipherSuites(NettySslPackageAccessor.java:24)
  at org.asynchttpclient.config.AsyncHttpClientConfigDefaults.defaultEnabledCipherSuites(AsyncHttpClientConfigDefaults.java:85)
  at org.asynchttpclient.DefaultAsyncHttpClientConfig$Builder.<init>(DefaultAsyncHttpClientConfig.java:635)
  at org.asynchttpclient.DefaultAsyncHttpClient.<init>(DefaultAsyncHttpClient.java:67)
  at …
Run Code Online (Sandbox Code Playgroud)

dependencies asynchttpclient apache-spark

6
推荐指数
1
解决办法
1609
查看次数