使用hdfs fsck检查副本数不足，怎么恢复
今天将NO-HA转移到HA的时候出现了数据丢失，本来是全部丢失。后来将dfs备份的目录导入到了新生成的DN节点中dfs目录下了。找回来一部分，但是还有是39个文件丢失。还出现了一些文件副本数不足。
我是将曾经的一个DN节点设置成了HA的NN2节点，新的NN2节点上的dfs数据肯定丢失了。
丢失的DN节点中的文件副本不会自动同步到其他DN节点上吗？
请问下各位大牛怎么解决部分副本数不足的情况。
Connecting to namenode via http://s200:50070/fsck?ugi=centos&path=%2F
FSCK started by centos (auth:SIMPLE) from /192.168.31.200 for path / at Mon Aug 21 02:26:00 CST 2017
.................
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1501993091662_0003.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741862
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1501993091662_0003.summary: MISSING 1 blocks of total size 333 B.......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1501993091662_0005.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741885
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1501993091662_0005.summary: MISSING 1 blocks of total size 337 B..........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502345425700_0001.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741917
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502345425700_0001.summary: MISSING 1 blocks of total size 349 B........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502345425700_0003_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741942
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502345425700_0003_conf.xml: MISSING 1 blocks of total size 117707 B.....
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0002-1502607832989-centos-select+count%28*%29+from+t_sz01%28Stage%2D1%29-1502607859592-1-1-SUCCEEDED-default-1502607841215.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741967
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0002-1502607832989-centos-select+count%28*%29+from+t_sz01%28Stage%2D1%29-1502607859592-1-1-SUCCEEDED-default-1502607841215.jhist: MISSING 1 blocks of total size 35239 B.....
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0003.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741978
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0003.summary: MISSING 1 blocks of total size 389 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0003_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741980
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0003_conf.xml: MISSING 1 blocks of total size 237100 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0004.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741990
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0004.summary: MISSING 1 blocks of total size 375 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0004_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741992
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0004_conf.xml: MISSING 1 blocks of total size 237482 B.....
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0006-1502617027499-centos-select+count%28*%29+from+t_sz_...%28name%3D%27liuyan%27%29%28Stage-1502617052317-2-1-SUCCEEDED-default-1502617035277.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742020
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0006-1502617027499-centos-select+count%28*%29+from+t_sz_...%28name%3D%27liuyan%27%29%28Stage-1502617052317-2-1-SUCCEEDED-default-1502617035277.jhist: MISSING 1 blocks of total size 42466 B........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0008.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742051
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0008.summary: MISSING 1 blocks of total size 394 B.......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0010.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742075
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0010.summary: MISSING 1 blocks of total size 387 B.........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0013-1502636955830-centos-select+*+from+a+right+join+b+on+a.id+%3D+b.id%28Stage%2D-1502636971859-1-0-SUCCEEDED-default-1502636963452.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742114
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0013-1502636955830-centos-select+*+from+a+right+join+b+on+a.id+%3D+b.id%28Stage%2D-1502636971859-1-0-SUCCEEDED-default-1502636963452.jhist: MISSING 1 blocks of total size 22534 B......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0014_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742130
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0014_conf.xml: MISSING 1 blocks of total size 237515 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0015.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742140
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0015.summary: MISSING 1 blocks of total size 377 B..........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0018.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742182
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0018.summary: MISSING 1 blocks of total size 393 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0019-1502701052165-centos-create+table+t_sz_movie+as%0Aselec...t_sz_json%28Stage-1502701079118-1-0-SUCCEEDED-default-1502701060724.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742196
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0019-1502701052165-centos-create+table+t_sz_movie+as%0Aselec...t_sz_json%28Stage-1502701079118-1-0-SUCCEEDED-default-1502701060724.jhist: MISSING 1 blocks of total size 23187 B......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0020_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742211
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0020_conf.xml: MISSING 1 blocks of total size 238336 B..
..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0021_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742224
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0021_conf.xml: MISSING 1 blocks of total size 239124 B.......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0023_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742250
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0023_conf.xml: MISSING 1 blocks of total size 240062 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0024.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742260
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0024.summary: MISSING 1 blocks of total size 394 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0024_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742262
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0024_conf.xml: MISSING 1 blocks of total size 239326 B......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0026.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742284
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0026.summary: MISSING 1 blocks of total size 379 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0026_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742286
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0026_conf.xml: MISSING 1 blocks of total size 239956 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0027.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742296
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0027.summary: MISSING 1 blocks of total size 394 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0027_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742298
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0027_conf.xml: MISSING 1 blocks of total size 239418 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0028-1502730520218-centos-select+A.username%2CA.mon...A.username%2CA.month%28Stage-1502730544857-1-1-SUCCEEDED-default-1502730528561.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742309
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0028-1502730520218-centos-select+A.username%2CA.mon...A.username%2CA.month%28Stage-1502730544857-1-1-SUCCEEDED-default-1502730528561.jhist: MISSING 1 blocks of total size 35937 B.......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0030-1502962914754-centos-create+table+azres+as+select+*+from+aztest%28Stage%2D1-1502962930654-1-0-SUCCEEDED-default-1502962922487.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742332
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0030-1502962914754-centos-create+table+azres+as+select+*+from+aztest%28Stage%2D1-1502962930654-1-0-SUCCEEDED-default-1502962922487.jhist: MISSING 1 blocks of total size 22828 B............
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0033_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742372
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0033_conf.xml: MISSING 1 blocks of total size 237997 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0034.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742417
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0034.summary: MISSING 1 blocks of total size 331 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0034_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742419
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0034_conf.xml: MISSING 1 blocks of total size 130067 B........
/user/centos/yellow.jpg: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741831
/user/centos/yellow.jpg: MISSING 1 blocks of total size 214347 B....
/user/hive/warehouse/aztest/hiveinput.txt: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742348
/user/hive/warehouse/aztest/hiveinput.txt: MISSING 1 blocks of total size 36 B...
/user/hive/warehouse/demo.db/t_sz01/hivedemo.txt: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741944
/user/hive/warehouse/demo.db/t_sz01/hivedemo.txt: MISSING 1 blocks of total size 73 B......
/user/hive/warehouse/t_p/000000_0: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742163
/user/hive/warehouse/t_p/000000_0: MISSING 1 blocks of total size 10 B......
/user/hive/warehouse/t_sz_buck/000003_0: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742049
/user/hive/warehouse/t_sz_buck/000003_0: MISSING 1 blocks of total size 26 B..
/user/hive/warehouse/t_sz_json/rating.json: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742170
/user/hive/warehouse/t_sz_json/rating.json: MISSING 1 blocks of total size 65602698 B........
/wordcount/inverindex/input/b.txt: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741921
/wordcount/inverindex/input/b.txt: MISSING 1 blocks of total size 58 B................
/wordcount/output/out3/part-r-00004: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741875
/wordcount/output/out3/part-r-00004: MISSING 1 blocks of total size 597 B......Status: CORRUPT
Total size: 98136546 B
Total dirs: 56
Total files: 191
Total symlinks: 0
Total blocks (validated): 182 (avg. block size 539211 B)
********************************
UNDER MIN REPL'D BLOCKS: 39 (21.428572 %)
dfs.namenode.replication.min: 1
CORRUPT FILES: 39
MISSING BLOCKS: 39
MISSING SIZE: 68638958 B
CORRUPT BLOCKS: 39
********************************
Minimally replicated blocks: 143 (78.57143 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 2.3516483
Corrupt blocks: 39
Missing replicas: 0 (0.0 %)
Number of data-nodes: 3
Number of racks: 1
FSCK ended at Mon Aug 21 02:26:01 CST 2017 in 31 milliseconds
The filesystem under path '/' is CORRUPT
这个不用担心，hadoop会自动均衡的。
nextuser 发表于 2017-8-21 09:26
这个不用担心，hadoop会自动均衡的。
已经解决了，是我之前在hdfs-site.xml文件中设置了data.dir参数为两个目录，块文件分别保存在两个目录中了。
我一直以为设置这个参数是进行块备份用的。现在将这两个目录下的块都拷贝到新的dfs目录下就解决了。
下面是我hdfs-site.xml中的data.dir参数
<property>
<name>dfs.datanode.data.dir</name>
<value>file://${hadoop.tmp.dir}/dfs/data1,file://${hadoop.tmp.dir}/dfs/data2</value>
</property>
感谢分享
页:
[1]