本人刚刚接触hadoop,属于嫩鸟,目前遇到一个问题:单个数据节点的2个挂载磁盘数据不均衡,怎么处理?数据如下:
/dev/vda1 412716208 373253312 18475112 96% /
devtmpfs 8126412 0 8126412 0% /dev
tmpfs 8134732 0 8134732 0% /dev/shm
tmpfs 8134732 123452 8011280 2% /run
tmpfs 8134732 0 8134732 0% /sys/fs/cgroup
/dev/vdb 515930552 61854320 427845448 13% /home/fulihui/hadoop-2.5.1/tmp/dfs/data1
我查看了相关文档,说是从一个目录下面转移到梁一个目录下面,内容如下:
3.12. On an individual data node, how do you balance the blocks on the disk?Hadoop currently does not have a method by which to do this automatically. To do this manually: Shutdown the DataNode involved Use the UNIX mv command to move the individual block replica and meta pairs from one directory to another on the selected host. On releases which have HDFS-6482 (Apache Hadoop 2.6.0+) you also need to ensure the subdir-named directory structure remains exactly the same when moving the blocks across the disks. For example, if the block replica and its meta pair were under/data/1/dfs/dn/current/BP-1788246909-172.23.1.202-1412278461680/current/finalized/subdir0/subdir1/, and you wanted to move it to /data/5/ disk, then it MUST be moved into the same subdirectory structure underneath that, i.e. /data/5/dfs/dn/current/BP-1788246909-172.23.1.202-1412278461680/current/finalized/subdir0/subdir1/. If this is not maintained, the DN will no longer be able to locate the replicas after the move. Restart the DataNode.
不知道有没有高手遇到过,指点一下,是不是也是这么操作的??
谢谢
|