使用hdfs fsck检查副本数不足，怎么恢复

今天将NO-HA转移到HA的时候出现了数据丢失，本来是全部丢失。
后来将dfs备份的目录导入到了新生成的DN节点中dfs目录下了。找回来一部分，但是还有是39个文件丢失。还出现了一些文件副本数不足。
我是将曾经的一个DN节点设置成了HA的NN2节点，新的NN2节点上的dfs数据肯定丢失了。
丢失的DN节点中的文件副本不会自动同步到其他DN节点上吗？
请问下各位大牛怎么解决部分副本数不足的情况。

[mw_shl_code=xml,true]Connecting to namenode via http://s200:50070/fsck?ugi=centos&path=%2F
FSCK started by centos (auth:SIMPLE) from /192.168.31.200 for path / at Mon Aug 21 02:26:00 CST 2017
.................
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1501993091662_0003.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741862

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1501993091662_0003.summary: MISSING 1 blocks of total size 333 B.......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1501993091662_0005.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741885

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1501993091662_0005.summary: MISSING 1 blocks of total size 337 B..........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502345425700_0001.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741917

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502345425700_0001.summary: MISSING 1 blocks of total size 349 B........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502345425700_0003_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741942

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502345425700_0003_conf.xml: MISSING 1 blocks of total size 117707 B.....
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0002-1502607832989-centos-select+count%28*%29+from+t_sz01%28Stage%2D1%29-1502607859592-1-1-SUCCEEDED-default-1502607841215.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741967

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0002-1502607832989-centos-select+count%28*%29+from+t_sz01%28Stage%2D1%29-1502607859592-1-1-SUCCEEDED-default-1502607841215.jhist: MISSING 1 blocks of total size 35239 B.....
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0003.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741978

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0003.summary: MISSING 1 blocks of total size 389 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0003_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741980

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0003_conf.xml: MISSING 1 blocks of total size 237100 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0004.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741990

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0004.summary: MISSING 1 blocks of total size 375 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0004_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741992

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0004_conf.xml: MISSING 1 blocks of total size 237482 B.....
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0006-1502617027499-centos-select+count%28*%29+from+t_sz_...%28name%3D%27liuyan%27%29%28Stage-1502617052317-2-1-SUCCEEDED-default-1502617035277.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742020

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0006-1502617027499-centos-select+count%28*%29+from+t_sz_...%28name%3D%27liuyan%27%29%28Stage-1502617052317-2-1-SUCCEEDED-default-1502617035277.jhist: MISSING 1 blocks of total size 42466 B........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0008.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742051

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0008.summary: MISSING 1 blocks of total size 394 B.......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0010.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742075

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0010.summary: MISSING 1 blocks of total size 387 B.........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0013-1502636955830-centos-select+*+from+a+right+join+b+on+a.id+%3D+b.id%28Stage%2D-1502636971859-1-0-SUCCEEDED-default-1502636963452.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742114

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0013-1502636955830-centos-select+*+from+a+right+join+b+on+a.id+%3D+b.id%28Stage%2D-1502636971859-1-0-SUCCEEDED-default-1502636963452.jhist: MISSING 1 blocks of total size 22534 B......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0014_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742130

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0014_conf.xml: MISSING 1 blocks of total size 237515 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0015.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742140

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0015.summary: MISSING 1 blocks of total size 377 B..........
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0018.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742182

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0018.summary: MISSING 1 blocks of total size 393 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0019-1502701052165-centos-create+table+t_sz_movie+as%0Aselec...t_sz_json%28Stage-1502701079118-1-0-SUCCEEDED-default-1502701060724.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742196

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0019-1502701052165-centos-create+table+t_sz_movie+as%0Aselec...t_sz_json%28Stage-1502701079118-1-0-SUCCEEDED-default-1502701060724.jhist: MISSING 1 blocks of total size 23187 B......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0020_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742211

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0020_conf.xml: MISSING 1 blocks of total size 238336 B..
..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0021_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742224

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0021_conf.xml: MISSING 1 blocks of total size 239124 B.......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0023_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742250

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0023_conf.xml: MISSING 1 blocks of total size 240062 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0024.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742260

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0024.summary: MISSING 1 blocks of total size 394 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0024_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742262

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0024_conf.xml: MISSING 1 blocks of total size 239326 B......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0026.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742284

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0026.summary: MISSING 1 blocks of total size 379 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0026_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742286

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0026_conf.xml: MISSING 1 blocks of total size 239956 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0027.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742296

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0027.summary: MISSING 1 blocks of total size 394 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0027_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742298

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0027_conf.xml: MISSING 1 blocks of total size 239418 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0028-1502730520218-centos-select+A.username%2CA.mon...A.username%2CA.month%28Stage-1502730544857-1-1-SUCCEEDED-default-1502730528561.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742309

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0028-1502730520218-centos-select+A.username%2CA.mon...A.username%2CA.month%28Stage-1502730544857-1-1-SUCCEEDED-default-1502730528561.jhist: MISSING 1 blocks of total size 35937 B.......
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0030-1502962914754-centos-create+table+azres+as+select+*+from+aztest%28Stage%2D1-1502962930654-1-0-SUCCEEDED-default-1502962922487.jhist: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742332

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0030-1502962914754-centos-create+table+azres+as+select+*+from+aztest%28Stage%2D1-1502962930654-1-0-SUCCEEDED-default-1502962922487.jhist: MISSING 1 blocks of total size 22828 B............
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0033_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742372

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0033_conf.xml: MISSING 1 blocks of total size 237997 B...
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0034.summary: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742417

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0034.summary: MISSING 1 blocks of total size 331 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0034_conf.xml: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742419

/tmp/hadoop-yarn/staging/history/done_intermediate/centos/job_1502602053942_0034_conf.xml: MISSING 1 blocks of total size 130067 B........
/user/centos/yellow.jpg: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741831

/user/centos/yellow.jpg: MISSING 1 blocks of total size 214347 B....
/user/hive/warehouse/aztest/hiveinput.txt: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742348

/user/hive/warehouse/aztest/hiveinput.txt: MISSING 1 blocks of total size 36 B...
/user/hive/warehouse/demo.db/t_sz01/hivedemo.txt: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741944

/user/hive/warehouse/demo.db/t_sz01/hivedemo.txt: MISSING 1 blocks of total size 73 B......
/user/hive/warehouse/t_p/000000_0: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742163

/user/hive/warehouse/t_p/000000_0: MISSING 1 blocks of total size 10 B......
/user/hive/warehouse/t_sz_buck/000003_0: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742049

/user/hive/warehouse/t_sz_buck/000003_0: MISSING 1 blocks of total size 26 B..
/user/hive/warehouse/t_sz_json/rating.json: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073742170

/user/hive/warehouse/t_sz_json/rating.json: MISSING 1 blocks of total size 65602698 B........
/wordcount/inverindex/input/b.txt: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741921

/wordcount/inverindex/input/b.txt: MISSING 1 blocks of total size 58 B................
/wordcount/output/out3/part-r-00004: CORRUPT blockpool BP-1318026961-192.168.211.200-1501490583248 block blk_1073741875

/wordcount/output/out3/part-r-00004: MISSING 1 blocks of total size 597 B......Status: CORRUPT
Total size:    98136546 B
Total dirs:    56
Total files:   191
Total symlinks:                0
Total blocks (validated):      182 (avg. block size 539211 B)
  ********************************
  UNDER MIN REPL'D BLOCKS:      39 (21.428572 %)
  dfs.namenode.replication.min: 1
  CORRUPT FILES:        39
  MISSING BLOCKS:       39
  MISSING SIZE:         68638958 B
  CORRUPT BLOCKS:       39
  ********************************
Minimally replicated blocks:   143 (78.57143 %)
Over-replicated blocks:        0 (0.0 %)
Under-replicated blocks:       0 (0.0 %)
Mis-replicated blocks:         0 (0.0 %)
Default replication factor:    3
Average block replication:     2.3516483
Corrupt blocks:                39
Missing replicas:              0 (0.0 %)
Number of data-nodes:          3
Number of racks:               1
FSCK ended at Mon Aug 21 02:26:01 CST 2017 in 31 milliseconds


The filesystem under path '/' is CORRUPT[/mw_shl_code]

这个不用担心，hadoop会自动均衡的。

nextuser 发表于 2017-8-21 09:26
这个不用担心，hadoop会自动均衡的。

已经解决了，是我之前在hdfs-site.xml文件中设置了data.dir参数为两个目录，块文件分别保存在两个目录中了。
我一直以为设置这个参数是进行块备份用的。现在将这两个目录下的块都拷贝到新的dfs目录下就解决了。

下面是我hdfs-site.xml中的data.dir参数
[mw_shl_code=xml,true]    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file://${hadoop.tmp.dir}/dfs/data1,file://${hadoop.tmp.dir}/dfs/data2</value>
    </property>[/mw_shl_code]

感谢分享