Avi*_*van 7 authentication kerberos hadoop-yarn apache-spark hadoop2
安装信息:-
Hadoop 版本:- 2.6.5 Spark 版本:- 2.1.0 和 Kerberos
我正在尝试使用 kerberos 身份验证在纱线模式下获取 spark 上下文,但低于异常。
代码 :-
public static void main(String[] args) {
Configuration conf = new Configuration();
conf.set("fs.defaultFS", "hdfs://localhost:9000");
conf.set("hadoop.security.authentication", "kerberos");
conf.set("hadoop.security.authorization", "true");
UserGroupInformation.setConfiguration(conf);
System.out.println("Security enabled " + UserGroupInformation.isSecurityEnabled());
SparkConf sparkConf = new SparkConf().setAppName("Spark shell").setMaster("yarn");
SparkSession sparkSession = SparkSession.builder().config(sparkConf).getOrCreate();
System.out.println(sparkSession.version() + " : " + sparkSession.sparkContext());
}
Run Code Online (Sandbox Code Playgroud)
其打印安全性已启用,例如:- 安全性已启用 true
例外 :-
org.apache.hadoop.security.AccessControlException:未启用简单身份验证。可用:[TOKEN, KERBEROS] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingImpl5.javaAccessor4Accessor) java.lang.reflect.Constructor.newInstance(Constructor.java:423) 在 org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
请建议我解决此错误。