专题 › Hadoop|YARN › hadoop使用kerbero后webhdfs的配置问题

pengsuyun 发表于 2014-10-24 15:59:51

hadoop使用kerbero后webhdfs的配置问题

本帖最后由 pengsuyun 于 2014-10-24 15:59 编辑

如题，在配置如上应用的时候，遇到一个问题“javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store”，导致namenode启动不了，其中有个比较奇怪的问题，我为namenode配置的principal nn/admin@psy.com登录却是没有问题。
core-site.xml中的关键配置：<property>
                <name>hadoop.security.authentication</name>
                <value>kerberos</value>
      </property>
      <property>
                <name>hadoop.security.authorization</name>
                <value>true</value>
      </property>
hdfs-site.xml中的配置<property>
                   <name>dfs.webhdfs.enabled</name>
                   <value>true</value>
           </property>
        <property>
                   <name>dfs.web.authentication.kerberos.principal</name>
                   <value>http/admin@psy.com</value>
           </property>
        <property>
                   <name>dfs.web.authentication.kerberos.keytab</name>
                   <value>/hadoop-data/etc/hadoop/http.service.keytab</value>
           </property>
问题报错信息：
2014-10-24 00:43:00,384 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Login using keytab /hadoop-data/etc/hadoop/http.service.keytab, for principal http/admin@psy.com
2014-10-24 00:43:00,406 WARN org.mortbay.log: failed SpnegoFilter: javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store
2014-10-24 00:43:00,407 WARN org.mortbay.log: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@4f98b5c{/,file:/opt/hadoop-2.4.1/share/hadoop/hdfs/webapps/hdfs}
javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:203)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:150)
        at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
        at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:224)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:796)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:132)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:666)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:557)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
Caused by: javax.security.auth.login.LoginException: No key to store
        at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1072)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:596)
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:187)
        ... 23 more
2014-10-24 00:43:00,418 WARN org.mortbay.log: Nested in javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store:
javax.security.auth.login.LoginException: No key to store
        at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1072)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:596)
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:187)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:150)
        at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
        at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:224)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:796)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:132)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:666)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:557)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
2014-10-24 00:43:00,447 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50070
2014-10-24 00:43:00,447 INFO org.mortbay.log: Stopped SelectChannelConnector@0.0.0.0:50070
2014-10-24 00:43:00,454 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system...
2014-10-24 00:43:00,455 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped.
2014-10-24 00:43:00,455 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete.
2014-10-24 00:43:00,455 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join
java.io.IOException: Unable to initialize WebAppContext
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:818)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:132)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:666)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:557)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
Caused by: javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:203)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:150)
        at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
        at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:224)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:796)
        ... 7 more
Caused by: javax.security.auth.login.LoginException: No key to store
        at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1072)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:596)
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:187)
        ... 23 more
2014-10-24 00:43:00,464 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2014-10-24 00:43:00,466 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:

bioger_hit 发表于 2014-10-24 18:08:49

建议按照下面配置，改一下

<!-- General HDFS security config -->
<property>
<name>dfs.block.access.token.enable</name>
<value>true</value>
</property>

<!-- NameNode security config -->
<property>
<name>dfs.https.address</name>
<value><fully qualified domain name of NN>:50470</value>
</property>
<property>
<name>dfs.https.port</name>
<value>50470</value>
</property>
<property>
<name>dfs.namenode.keytab.file</name>
<value>/usr/local/hadoop/conf/hdfs.keytab</value> <!-- path to the HDFS keytab -->
</property>
<property>
<name>dfs.namenode.kerberos.principal</name>
<value>hdfs/_HOST@YOUR-REALM.COM</value>
</property>
<property>
<name>dfs.namenode.kerberos.https.principal</name>
<value>host/_HOST@YOUR-REALM.COM</value>
</property>

<!-- Secondary NameNode security config -->
<property>
<name>dfs.secondary.https.address</name>
<value><fully qualified domain name of 2NN>:50495</value>
</property>
<property>
<name>dfs.secondary.https.port</name>
<value>50495</value>
</property>
<property>
<name>dfs.secondary.namenode.keytab.file</name>
<value>/usr/local/hadoop/conf/hdfs.keytab</value> <!-- path to the HDFS keytab -->
</property>
<property>
<name>dfs.secondary.namenode.kerberos.principal</name>
<value>hdfs/_HOST@YOUR-REALM.COM</value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.https.principal</name>
<value>host/_HOST@YOUR-REALM.COM</value>
</property>

<!-- DataNode security config -->
<property>
<name>dfs.datanode.data.dir.perm</name>
<value>700</value>
</property>
<property>
<name>dfs.datanode.address</name>
<value>0.0.0.0:1004</value>
</property>
<property>
<name>dfs.datanode.http.address</name>
<value>0.0.0.0:1006</value>
</property>
<property>
<name>dfs.datanode.keytab.file</name>
<value>/usr/local/hadoop/conf/hdfs.keytab</value> <!-- path to the HDFS keytab -->
</property>
<property>
<name>dfs.datanode.kerberos.principal</name>
<value>hdfs/_HOST@YOUR-REALM.COM</value>
</property>
<property>
<name>dfs.datanode.kerberos.https.principal</name>
<value>host/_HOST@YOUR-REALM.COM</value>
</property>

desehawk 发表于 2014-10-24 18:33:29

感觉你的配置文档有些问题，可以参考这个
hadoop 添加kerberos认证

pengsuyun 发表于 2014-10-24 23:21:08

desehawk 发表于 2014-10-24 18:33
感觉你的配置文档有些问题，可以参考这个
hadoop 添加kerberos认证

你的这篇帖子我之前也看过了，可能我的能力有限理解得不太好，但是我想理解其中的这个原理是什么？方便留下你的qq吗？我好直接找你请教问题。

desehawk 发表于 2014-10-25 09:25:44

刚开始学习，如果不能理解，就先参照别人的来学习，等时间长了，熟悉了，自然理解了

pengsuyun 发表于 2014-11-10 08:46:31

确实是这样，好吧！

xbings 发表于 2015-4-27 14:27:40

请问这个问题最终怎么解决的呢？

z812620498 发表于 2015-9-17 11:46:55

估计是没有解决吧

z812620498 发表于 2015-9-17 11:47:34

我也遇到类似的问题 无奈

linqingrui 发表于 2016-3-2 13:42:54

应该是时间同步的问题

查看完整版本: hadoop使用kerbero后webhdfs的配置问题