Hbase shell 启动报错，使用status命令报 master is initializing

[mw_shl_code=shell,true]2016-12-20 09:55:59,579 WARN  [master:16000.activeMasterManager] master.AssignmentManager: Failed assignment of hbase:meta,,1.1588230740 to slave1,16020,1482198847112, trying to assign elsewhere instead; try=5 of 10
java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupConnection(RpcClientImpl.java:416)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:722)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
        at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.openRegion(AdminProtos.java:23125)
        at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:765)
        at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2155)
        at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1601)
        at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1579)
        at org.apache.hadoop.hbase.master.AssignmentManager.assignMeta(AssignmentManager.java:2759)
        at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:941)
        at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:744)
        at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:186)
        at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1762)
        at java.lang.Thread.run(Thread.java:745)
2016-12-20 09:55:59,580 INFO  [master:16000.activeMasterManager] master.AssignmentManager: Assigning hbase:meta,,1.1588230740 to slave1,16020,1482198847112
2016-12-20 09:55:59,580 WARN  [master:16000.activeMasterManager] master.AssignmentManager: Failed assignment of hbase:meta,,1.1588230740 to slave1,16020,1482198847112, trying to assign elsewhere instead; try=6 of 10
org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: slave1/10.162.30.162:16020
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:707)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
        at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
        at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.openRegion(AdminProtos.java:23125)
        at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:765)
        at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:2155)
        at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1601)
        at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1579)
        at org.apache.hadoop.hbase.master.AssignmentManager.assignMeta(AssignmentManager.java:2759)
        at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:941)
        at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:744)
        at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:186)
        at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1762)
        at java.lang.Thread.run(Thread.java:745)
2016-12-20 09:55:59,580 INFO  [master:16000.activeMasterManager] master.AssignmentManager: Trying to re-assign hbase:meta,,1.1588230740 to the same failed server.
z^C[/mw_shl_code]

查看之后 发现时间不一致，不过还有如下错误：
1，zookeeper 日志：
[mw_shl_code=shell,true]        at org.apache.zookeeper.common.AtomicFileOutputStream.<init>(AtomicFileOutputStream.java:59)
        at org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(QuorumPeer.java:1183)
        at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:476)
        at org.apache.zookeeper.server.quorum.QuorumPeer.getLastLoggedZxid(QuorumPeer.java:590)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.getInitLastLoggedZxid(FastLeaderElection.java:765)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:812)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2016-12-20 10:34:38,340 WARN  [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181] quorum.QuorumPeer: Unexpected exception
java.lang.RuntimeException: Unable to run quorum server
        at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:498)
        at org.apache.zookeeper.server.quorum.QuorumPeer.getLastLoggedZxid(QuorumPeer.java:590)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.getInitLastLoggedZxid(FastLeaderElection.java:765)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:812)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
Caused by: java.io.FileNotFoundException: /tmp/hbase-hadoop/zookeeper/version-2/currentEpoch.tmp (No such file or directory)
        at java.io.FileOutputStream.open0(Native Method)
        at java.io.FileOutputStream.open(FileOutputStream.java:270)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
        at org.apache.zookeeper.common.AtomicFileOutputStream.<init>(AtomicFileOutputStream.java:59)
        at org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(QuorumPeer.java:1183)
        at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:476)
        ... 4 more
2016-12-20 10:34:38,340 INFO  [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181] quorum.QuorumPeer: LOOKING
2016-12-20 10:34:38,340 INFO  [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181] quorum.FastLeaderElection: New election. My id =  0, proposed zxid=0x0
2016-12-20 10:34:38,340 INFO  [WorkerReceiver[myid=0]] quorum.FastLeaderElection: Notification: 1 (message format version), 0 (n.leader), 0x0 (n.zxid), 0xb (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-12-20 10:34:38,341 INFO  [WorkerReceiver[myid=0]] quorum.FastLeaderElection: Notification: 1 (message format version), 2 (n.leader), 0xf00000033 (n.zxid), 0x9 (n.round), FOLLOWING (n.state), 1 (n.sid), 0x10 (n.peerEpoch) LOOKING (my state)
2016-12-20 10:34:38,341 INFO  [WorkerReceiver[myid=0]] quorum.FastLeaderElection: Notification: 1 (message format version), 2 (n.leader), 0xf00000033 (n.zxid), 0x9 (n.round), LEADING (n.state), 2 (n.sid), 0x10 (n.peerEpoch) LOOKING (my state)
2016-12-20 10:34:38,341 INFO  [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181] quorum.QuorumPeer: FOLLOWING
2016-12-20 10:34:38,341 INFO  [QuorumPeer[myid=0]/0:0:0:0:0:0:0:0:2181] server.ZooKeeperServer: Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 90000 datadir /tmp/hbase-hadoop/zookeeper/version-2 snapdir /tmp/hbase-hadoop/zookeeper/version-2

[/mw_shl_code]



[mw_shl_code=shell,true]hbase(main):003:0> status

ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
        at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2293)
        at org.apache.hadoop.hbase.master.MasterRpcServices.getClusterStatus(MasterRpcServices.java:777)
        at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55652)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2178)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
        at java.lang.Thread.run(Thread.java:745)

Here is some help for this command:
Show cluster status. Can be 'summary', 'simple', 'detailed', or 'replication'. The
default is 'summary'. Examples:

  hbase> status
  hbase> status 'simple'
  hbase> status 'summary'
  hbase> status 'detailed'
  hbase> status 'replication'
  hbase> status 'replication', 'source'
  hbase> status 'replication', 'sink'[/mw_shl_code]


目前master 和slave时间差几乎一秒之内， 且master和slave进程正常

Wyy_Ck 发表于 2016-12-20 10:41
查看之后 发现时间不一致，不过还有如下错误：
1，zookeeper 日志：
[mw_shl_code=shell,true]        a ...

存在不少的问题。首先说第一个问题
java.net.NoRouteToHostException: No route to host

这个原因是由于网络造成的。
具体原因，比如hosts是否都一致，127.0.1.1是否注释掉等，还有就是集群的所有机器防火墙是否都关闭，还有端口是否被暂用。
上面一一核实应该没有问题的

第二个问题
尝试下面解决方法
切换到dataDir配置的目录下，楼主的文件存在version-2，将他删除即可

详细操作：
找到dataDir=/var/lib/zookeeper

切换到路径/var/lib/zookeeper

[mw_shl_code=bash,true]cd /var/lib/zookeeper[/mw_shl_code]

查看目录下的文件：

ls

存在version-2

删除version-2

[mw_shl_code=bash,true]mv ./version-2 ./version-2.bak
[/mw_shl_code]


然后重启
[mw_shl_code=bash,true]zkServer.sh  start [/mw_shl_code]

问题1：
我尝试了下，现在jps看下 进程都起来了，有一个这个问题，不知道算不算？
[mw_shl_code=shell,true][hadoop@master sbin]$ start-hbase.sh
master: starting zookeeper, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-zookeeper-master.out
slave2: starting zookeeper, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-zookeeper-slave2.out
slave1: starting zookeeper, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-zookeeper-slave1.out
starting master, logging to /opt/hadoop/hbase-1.2.3//logs/hbase-hadoop-master-master.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
slave2: starting regionserver, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-regionserver-slave2.out
slave1: starting regionserver, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-regionserver-slave1.out
slave2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
slave2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
slave1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
slave1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0[/mw_shl_code]

问题2：
[mw_shl_code=shell,true][hadoop@master sbin]$ hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hadoop/hbase-1.2.3/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.7.0/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2016-12-20 15:28:38,918 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts
2016-12-20 15:28:38,920 WARN  [main] zookeeper.ZKUtil: hconnection-0x1a96d94c0x0, quorum=master:2181,slave1:2181,slave2:2181, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:419)
        at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
        at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:905)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:648)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)[/mw_shl_code]

问题1：
[mw_shl_code=shell,true][hadoop@master sbin]$ hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hadoop/hbase-1.2.3/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.7.0/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2016-12-20 15:39:54,480 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts
2016-12-20 15:39:54,481 WARN  [main] zookeeper.ZKUtil: hconnection-0x1a96d94c0x0, quorum=master:2181,slave1:2181,slave2:2181, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:419)
        at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
        at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:905)[/mw_shl_code]


问题2：
[mw_shl_code=shell,true][hadoop@master sbin]$ start-hbase.sh
slave2: starting zookeeper, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-zookeeper-slave2.out
master: starting zookeeper, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-zookeeper-master.out
slave1: starting zookeeper, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-zookeeper-slave1.out
starting master, logging to /opt/hadoop/hbase-1.2.3//logs/hbase-hadoop-master-master.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
slave2: starting regionserver, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-regionserver-slave2.out
slave1: starting regionserver, logging to /opt/hadoop/hbase-1.2.3/bin/../logs/hbase-hadoop-regionserver-slave1.out
slave2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
slave2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
slave1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
slave1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
[hadoop@master sbin]$ jps
22741 HMaster
22902 Jps
21961 NameNode
22201 SecondaryNameNode
22622 HQuorumPeer
[hadoop@master sbin]$ hbase shell[/mw_shl_code]

问题2 不知道算不算问题

Wyy_Ck 发表于 2016-12-20 15:41
问题1：
[mw_shl_code=shell,true][hadoop@master sbin]$ hbase shell
SLF4J: Class path contains multi ...

jdk应该是1.8的，不支持，不过应该不是什么大问题

进入hbase shell 报错，是防火墙没关闭

注意： centos关闭防火墙：

systemctl stop firewalld.service


后面出现一个小问题：
[mw_shl_code=shell,true] Can't get master address from ZooKeeper; znode data == null[/mw_shl_code]

最后参考 http://www.aboutyun.com/thread-8691-1-1.html  方法一搞定！


在此谢谢上面各位！ 谢谢！