我在三个节点上成功安装了spark。我可以访问spark Web UI并找到每个工作节点和主节点处于活动状态。
我可以成功运行SparkPi示例。
我的集群信息:10.45.10.33(master&worker,hadoop-master,hadoop-slave)10.45.10.34(worker,hadoop-slave)10.45.10.35(worker,hadoop-slave)
但是当我尝试运行“ spark-shell --master yarn”时,它给出了例外:
16/09/12 19:50:29 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:85)
at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:62)
at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:149)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:500)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2256)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply(SparkSession.scala:831)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply(SparkSession.scala:823)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:823)
at org.apache.spark.repl.Main$.createSparkSession(Main.scala:101)
at $line3.$read$$iw$$iw.<init>(<console>:15)
at $line3.$read$$iw.<init>(<console>:31)
at $line3.$read.<init>(<console>:33)
at $line3.$read$.<init>(<console>:37)
at $line3.$read$.<clinit>(<console>)
at $line3.$eval$.$print$lzycompute(<console>:7)
at $line3.$eval$.$print(<console>:6)
at $line3.$eval.$print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786) …Run Code Online (Sandbox Code Playgroud) 我已阅读stackoverflow 中的相关问题。但是,答案仅告诉如何配置身份验证检查。我需要在 Spring Boot 中使用 rsocket 加密所有传输的数据。如何在带有 SSL/TLS 的 Spring Boot 中使用 tls。当我初始化 rsocket 客户端时,我找不到任何支持的方法,如下所示,尽管我知道 rsocket 本身可以支持 SSL/TLS。我找到了一些例子:例子
this.rsocketRequester = rsocketRequesterBuilder.setupRoute("sidecar-client")
.setupData("test_data")
.rsocketConnector(connector -> connector.acceptor(responder))
.connectTcp("localhost", 7000)
.block();
Run Code Online (Sandbox Code Playgroud)
Summary of my question:
1. Does rsocket in springboot support SSL/TLS ?
2. If spring boot support rsocket with SSL/TLS, are there any examples can be referenced
Run Code Online (Sandbox Code Playgroud) 我按照 spring-rsocket-demo 项目来完成我的代码。我在服务器端代码中添加了一些身份验证逻辑,如下所示。您可以看到我在身份验证逻辑“CLIENT.add(requester)”之后抛出了一个异常。我在 spring-rsocket-demo 项目中执行名为“testRequestGetsResponse”的测试方法,发现无法收到异常。如何处理 @ConnectMapping 注释方法中的设置有效负载并在需要时返回 RejectedSetupException。
requester.rsocket()
.onClose()
.doFirst(() -> {
// Add all new clients to a client list
log.info("Client: {} CONNECTED.", client);
//throw Exception during process setup payload
//my authentication code.
try {
// CLIENTS.add(requester);
throw new RuntimeException();
} catch (Exception exception) {
throw exception;
}
})
.doOnError(error -> {
// Warn when channels are closed by clients
log.warn("Channel to client {} CLOSED", client);
})
.doFinally(consumer -> {
// Remove disconnected clients from the client list …Run Code Online (Sandbox Code Playgroud) 我整天都被这个问题困住了.
我已经仔细阅读了github上的django-ckeditor的文档,但它不够详细.而给定的演示太复杂了.有没有更简单的方法来实现上传和浏览.如果可以通过向views.py添加几行来完成,urls.py和setting.py将使我很容易实现它.
PS:我是django的新人.
urls.py
from django.conf.urls import include, url
urlpatterns = [
url(r'^$', 'guitar.views.home', name='home'),
url(r'^ckeditor/', include('ckeditor_uploader.urls')),
]
Run Code Online (Sandbox Code Playgroud)
settings.py
import os
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATIC_URL = '/static/'
STATICFILES_DIRS = (
os.path.join(BASE_DIR, "static"),
)
MEDIA_URL = "image_upload/"
MEDIA_ROOT = os.path.join(BASE_DIR, "image_upload")
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/1.8/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = ')7sn#y%9dgm!pxl#c$ws_jld!%2lbvj5tg*jlqnaelc)+$bp*%'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
ALLOWED_HOSTS …Run Code Online (Sandbox Code Playgroud) rsocket ×2
spring-boot ×2
apache-spark ×1
ckeditor ×1
django ×1
hadoop ×1
hdfs ×1
python ×1
rsocket-java ×1