Kerberos身份验证错误 - 从SharedPath加载Hadoop配置文件时

Question

我正在开发一个Java应用程序,这个应用程序正在将结果数据保存到HDFS.java应用程序应该在我的Windows机器上运行.

我们使用Kerberos身份验证,并将密钥表文件放在NAS驱动器中.我们将Hadoop配置文件保存在同一个NAS驱动器中.

我的问题是当我从NAS驱动器加载Hadoop配置文件时,它给我一些验证错误,但是如果我从本地文件系统加载配置文件,我的应用程序运行正常(我还在C:\ Hadoop中保存了配置文件) )

下面是我的工作代码片段.(NAS中的keytab文件,本地文件系统中的Hadoop配置文件)

static String KeyTabPath = "\\\\path\\2\\keytabfile\\name.keytab"
Configuration config = new Configuration();
        config.set("fs.defaultFS", "hdfs://xxx.xx.xx.com:8020");
        config.addResource(new Path("C:\\Hadoop\\core-site.xml"));
        config.addResource(new Path("C:\\Hadoop\\hdfs-site.xml"));
        config.addResource(new Path("C:\\Hadoop\\mapred-site.xml"));
        config.addResource(new Path("C:\\Hadoop\\yarn-site.xml"));
        config.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
        config.set("fs.file.impl",org.apache.hadoop.fs.LocalFileSystem.class.getName());
        // Kerberos Authentication
        config.set("hadoop.security.authentication", "Kerberos");
        UserGroupInformation.setConfiguration(config);
        UserGroupInformation.loginUserFromKeytab("name@xx.xx.COM",KeyTabPath);

我也尝试从NAS驱动器加载配置文件,但是获得了kerberos身份验证错误.下面是抛出错误的代码片段(NAS中的Keytab文件和NAS中的Hadoop配置文件)

static String KeyTabPath = "\\\\path\\2\\keytabfile\\name.keytab"
Configuration config = new Configuration();
        config.set("fs.defaultFS", "hdfs://xxx.xx.xx.com:8020");
        config.addResource(new Path("\\\\NASDrive\\core-site.xml"));
        config.addResource(new Path("\\\\NASDrive\\hdfs-site.xml"));
        config.addResource(new Path("\\\\NASDrive\\mapred-site.xml"));
        config.addResource(new Path("\\\\NASDrive\\yarn-site.xml"));
        config.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
        config.set("fs.file.impl",org.apache.hadoop.fs.LocalFileSystem.class.getName());
        // Kerberos Authentication
        config.set("hadoop.security.authentication", "Kerberos");
        UserGroupInformation.setConfiguration(config);
        UserGroupInformation.loginUserFromKeytab("name@xx.xx.COM",KeyTabPath);

以下是错误消息

java.io.IOException: Login failure for name@XX.XX.COM from keytab \\NASdrive\name.keytab: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:962)
    at Appname.ldapLookupLoop(Appname.java:111)
    at Appname.main(Appname.java:70)
Caused by: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM
    at org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at javax.security.auth.login.LoginContext.invoke(Unknown Source)
    at javax.security.auth.login.LoginContext.access$000(Unknown Source)
    at javax.security.auth.login.LoginContext$4.run(Unknown Source)
    at javax.security.auth.login.LoginContext$4.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(Unknown Source)
    at javax.security.auth.login.LoginContext.login(Unknown Source)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:953)
    ... 2 more
Caused by: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM
    at org.apache.hadoop.security.User.<init>(User.java:51)
    at org.apache.hadoop.security.User.<init>(User.java:43)
    at org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:197)
    ... 14 more
Caused by: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM
    at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:389)
    at org.apache.hadoop.security.User.<init>(User.java:48)
    ... 16 more
Jul 06, 2016 4:29:14 PM com.XX.it.logging.JdkMapper info
INFO:  IO Exception occured: java.io.IOException: Login failure for name@XX.XX.COM from keytab \\NASdrive\name.keytab: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name name@XX.XX.COM: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to name@XX.XX.COM

所以问题似乎是加载配置文件.我的应用程序从NAS驱动器读取keytab文件,但不是Hadoop配置文件.可能是什么问题.我检查了所有NAS驱动器权限和文件权限.Everthing很好.我不知道问题出在哪里.请任何人帮我找出问题.

Answer 1

您缺少auth_to_localkerberos 主体名称转换的“DEFAULT”规则。

org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule：没有规则应用于

请参阅此处的示例 -

https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SecureMode.html#Mapping_from_Kerberos_principals_to_OS_user_accounts

hadoop.security.auth_to_local所以基本上只需在 .txt 文件的末尾添加“DEFAULT”一词即可core-site.xml。

另请auth_to_local参阅Kerberos 文档。

附言。如果您有兴趣深入研究这个主题，这里是Hadoop 代码库中发生此异常的地方。