Hadoop常见错误及解决办法汇总

1，错误一：java.io.IOException: Incompatible clusterIDs 时常出现在namenode重新格式化之后9 `7 k# I: L2 |9 U* @6 d

2014-04-29 14:32:53,877 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed forblock pool Block pool BP-1480406410-192.168.1.181-1398701121586 (storage idDS-167510828-192.168.1.191-50010-1398750515421) service tohadoop-master/192.168.1.181:9000" J' |7 h2q( T& @$ h" s' B
java.io.IOException: Incompatible clusterIDs in/data/dfs/data: namenode clusterID = CID-d1448b9e-da0f-499e-b1d4-78cb18ecdebb;datanode clusterID = CID-ff0faa40-2940-4838-b321-98272eb0dee3! U8 t) L- F( @0 ~' H0 N9 I
        atorg.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:391)5 ~" a4 j4 o6 M7 ~* r
        atorg.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:191)8 {* e. t; f7 ?# I8 I: \- v
        atorg.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:219)
        atorg.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:837)
        atorg.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:808)9 l  e( o1 o  u# D
        atorg.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:280)
        atorg.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:222)
        atorg.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:664)* j) }9 t/ x* ~
        atjava.lang.Thread.run(Thread.java:722)
2014-04-29 14:32:53,885 WARNorg.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for:Block pool BP-1480406410-192.168.1.181-1398701121586 (storage idDS-167510828-192.168.1.191-50010-1398750515421) service tohadoop-master/192.168.1.181:90002 V9 G- G3 f* L
2014-04-29 14:32:53,889 INFOorg.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool BP-1480406410-192.168.1.181-1398701121586(storage id DS-167510828-192.168.1.191-50010-1398750515421)
2014-04-29 14:32:55,897 WARNorg.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode

原因：每次namenode format会重新创建一个namenodeId,而data目录包含了上次format时的id,namenode format清空了namenode下的数据,但是没有清空datanode下的数据,导致启动时失败,所要做的就是每次fotmat前,清空data下的所有目录.
: d6 E2 t& M" g7 a* q3 l, H

解决办法：停掉集群，删除问题节点的data目录下的所有内容。即hdfs-site.xml文件中配置的dfs.data.dir目录。重新格式化namenode。

另一个更省事的办法：先停掉集群，然后将datanode节点目录/dfs/data/current/VERSION中的修改为与namenode一致即可。

Hadoop常见错误及解决办法汇总