问题：regionserver down掉了

查看了regionserver down掉了的那台机器的log日志，发现就一条错误信息
ERROR org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics: Inconsistent configuration. Previous configuration for using table name in metrics: true, new configuration: false
其他的都是info信息，是哪块的配置和以前不一样了，还是什么问题，希望大家给个解答。

你有查看zookeeper的日志吗？

lzw 发表于 2013-12-27 11:52
你有查看zookeeper的日志吗？

没有 我现在去查一下

fanbells 发表于 2013-12-27 12:06
没有 我现在去查一下

Sets the flag of whether to use table name in metric names. This flag is specified in configuration and is not expected to change at runtime, so we log an error message when it does change.
639
640  private static void More ...setUseTableName(final boolean useTableNameNew) {
641    if (useTableNameGlobally == null) {
642      // This configuration option has not yet been set.
643      useTableNameGlobally = useTableNameNew;
644    } else if (useTableNameGlobally != useTableNameNew
645        && !loggedConfInconsistency) {
646      // The configuration is inconsistent and we have not reported it
647      // previously. Once we report it, just keep ignoring the new setting.
648      LOG.error("Inconsistent configuration. Previous configuration "
649          + "for using table name in metrics: " + useTableNameGlobally + ", "
650          + "new configuration: " + useTableNameNew);
651      loggedConfInconsistency = true;
652    }
653  }
复制代码

从源代码和方面描述上看，这个错误是因为你对hbase做什么特殊配置了。我建议你把hbase的配置文件贴出来。

你是不是在配置文件做了这个hbase.metrics.showTableName参数设置

lzw 发表于 2013-12-27 13:01
你是不是在配置文件做了这个hbase.metrics.showTableName参数设置

没有配置这个参数，这个是我的hbase-site.xml文件，zookeeper使用的是单独安装的，不是hbase自带的。
<property>
    <name>hbase.rootdir</name>
    <value>hdfs://slave1:9000/hbase</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <property>
        <name>hbase.zookeeper.quorum</name>
        <value>slave1,slave2,slave3</value>
    </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/home/zk_data</value>
  </property>
  <property>
    <name>hbase.tmp.dir</name>
    <value>/home/hbase_tmp</value>
    </property>

fanbells 发表于 2013-12-27 13:47
没有配置这个参数，这个是我的hbase-site.xml文件，zookeeper使用的是单独安装的，不是hbase自带的。

...

现在我的master都启动不了了，日志报错如下：
WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2013-12-27 13:40:55,865 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server slave1/192.168.1.191:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration)
2013-12-27 13:40:55,865 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to  slave1/192.168.1.191:2181:2181, initiating session
2013-12-27 13:40:55,906 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2013-12-27 13:40:56,006 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = Connec
tionLoss for /hbase
2013-12-27 13:40:56,007 ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
2013-12-27 13:40:56,007 ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start master
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
        at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2106)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2120)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1069)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:199)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1109)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1099)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1083)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:162)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:155)
        at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:347)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2101)
        ... 5 more

fanbells 发表于 2013-12-27 13:49
现在我的master都启动不了了，日志报错如下：
WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for ...

我觉得问题就出在zookeeper那里，你确认你配置的密匙是没有问题的吗。

你试着禁用IPV6，将/etc/hosts文件里面的：：1 localhost那一行删掉重启

[grid@rac1 ~]$ cat /etc/hosts

127.0.0.1 localhost.localdomain localhost

# ::1 localhost6.localdomain6 localhosti6

192.168.2.101 rac1.localdomain rac1
192.168.2.102 rac2.localdomain rac2

192.168.2.100 gc.localdomain gc

lzw 发表于 2013-12-27 15:53
你试着禁用IPV6，将/etc/hosts文件里面的：：1 localhost那一行删掉重启

[grid@rac1 ~]$ cat /etc/hosts ...

问题解决了，是zookeeper的问题