新手求助，MapReduce程序运行出错，非计算机专业，恳请大神介绍得浅显点，感激不尽！

david@s0:~/Downloads$ hadoop jar HadoopDemo2.jar /user/it18zhang/ncdc_data/19*.gz /user/it18zhang/out
17/04/13 20:14:34 INFO client.RMProxy: Connecting to ResourceManager at s0/192.168.20.128:8032
17/04/13 20:14:34 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
17/04/13 20:14:35 INFO input.FileInputFormat: Total input paths to process : 2
17/04/13 20:14:35 INFO mapreduce.JobSubmitter: number of splits:2
17/04/13 20:14:36 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1492080262949_0002
17/04/13 20:14:36 INFO impl.YarnClientImpl: Submitted application application_1492080262949_0002
17/04/13 20:14:36 INFO mapreduce.Job: The url to track the job: http://s0:8088/proxy/application_1492080262949_0002/
17/04/13 20:14:36 INFO mapreduce.Job: Running job: job_1492080262949_0002
17/04/13 20:14:48 INFO mapreduce.Job: Job job_1492080262949_0002 running in uber mode : false
17/04/13 20:14:48 INFO mapreduce.Job:  map 0% reduce 0%
17/04/13 20:15:23 INFO mapreduce.Job: Task Id : attempt_1492080262949_0002_m_000000_0, Status : FAILED
Error: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1808748719-192.168.20.128-1488809333278:blk_1073741826_1002 file=/user/it18zhang/ncdc_data/1902.gz
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:983)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:642)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:882)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:934)
at java.io.DataInputStream.read(DataInputStream.java:149)
at org.apache.hadoop.io.compress.DecompressorStream.getCompressedData(DecompressorStream.java:159)
at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:143)
at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)
at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:144)
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:184)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
...........(略)
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

17/04/13 20:15:23 INFO mapreduce.Job: Task Id : attempt_1492080262949_0002_m_000001_0, Status : FAILED
Error: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1808748719-192.168.20.128-1488809333278:blk_1073741825_1001 file=/user/it18zhang/ncdc_data/1901.gz
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:983)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:642)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:882)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:934)
at java.io.DataInputStream.read(DataInputStream.java:149)
at org.apache.hadoop.io.compress.DecompressorStream.getCompressedData(DecompressorStream.java:159)
at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:143)
at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85)
at java.io.InputStream.read(InputStream.java:101)

.............（略）

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

17/04/13 20:15:50 INFO mapreduce.Job: Task Id : attempt_1492080262949_0002_m_000000_1, Status : FAILED
Error: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1808748719-192.168.20.128-1488809333278:blk_1073741826_1002 file=/user/it18zhang/ncdc_data/1902.gz
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:983)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:642)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:882)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:934)
at java.io.DataInputStream.read(DataInputStream.java:149)
at org.apache.hadoop.io.compress.DecompressorStream.getCompressedData(DecompressorStream.java:159)
at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:143)
at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85)
..........（略）
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

17/04/13 20:16:10 INFO mapreduce.Job:  map 50% reduce 0%
17/04/13 20:16:10 INFO mapreduce.Job: Task Id : attempt_1492080262949_0002_m_000000_2, Status : FAILED
Error: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1808748719-192.168.20.128-1488809333278:blk_1073741826_1002 file=/user/it18zhang/ncdc_data/1902.gz
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:983)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:642)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:882)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:934)
at java.io.DataInputStream.read(DataInputStream.java:149)
at org.apache.hadoop.io.compress.DecompressorStream.getCompressedData(DecompressorStream.java:159)
at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:143)
at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)
at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:144)
..............（略）
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

17/04/13 20:16:11 INFO mapreduce.Job:  map 0% reduce 0%
17/04/13 20:16:12 INFO mapreduce.Job: Task Id : attempt_1492080262949_0002_m_000001_1, Status : FAILED
Error: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1808748719-192.168.20.128-1488809333278:blk_1073741825_1001 file=/user/it18zhang/ncdc_data/1901.gz
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:983)
at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:642)
at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:882)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:934)
at java.io.DataInputStream.read(DataInputStream.java:149)
at org.apache.hadoop.io.compress.DecompressorStream.getCompressedData(DecompressorStream.java:159)
at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:143)
at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:85)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)
at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
............（略）

17/04/13 20:16:38 INFO mapreduce.Job:  map 100% reduce 100%
17/04/13 20:16:39 INFO mapreduce.Job: Job job_1492080262949_0002 failed with state FAILED due to: Task failed task_1492080262949_0002_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

17/04/13 20:16:40 INFO mapreduce.Job: Counters: 16
Job Counters
Failed map tasks=6
Killed map tasks=1
Killed reduce tasks=1
Launched map tasks=7
Other local map tasks=7
Total time spent by all maps in occupied slots (ms)=200749
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=200749
Total time spent by all reduce tasks (ms)=0
Total vcore-milliseconds taken by all map tasks=200749
Total vcore-milliseconds taken by all reduce tasks=0
Total megabyte-milliseconds taken by all map tasks=205566976
Total megabyte-milliseconds taken by all reduce tasks=0
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
false

你看看你集群中的DataNode节点是否正常哟。
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block

问一下：你的hadoop集群能跑基本的wordcount程序吗，先试一下跑一下wordcount程序

jps看下datanode可能有挂掉的。
最好重启下集群。
有的datanode在刚启动的时候是没有问题的，但是一会就挂掉了。

wqkenqing 发表于 2017-4-14 09:28
你看看你集群中的DataNode节点是否正常哟。
org.apache.hadoop.hdfs.BlockMissingException: Could not ob ...

通过jps命令查看过了，都活的好好的，我搭建的是一个NameNode节点和两个DataNode节点，但DataNode节点的日志文件出现了一个错误，都在图片里了

tanxl 发表于 2017-4-14 09:59
问一下：你的hadoop集群能跑基本的wordcount程序吗，先试一下跑一下wordcount程序

WordCount程序也一样出错，而且有时候还有Interruptedexception，哎，卡了好几天，要崩溃了

1601365 发表于 2017-4-14 15:58
通过jps命令查看过了，都活的好好的，我搭建的是一个NameNode节点和两个DataNode节点，但DataNode节点的 ...

一个个解决，你这里明显有问题。下面红字部分先解决掉。为什么还有gz文件。datanode根本不能保存。
attempt_1492080262949_0002_m_000000_0, Status : FAILED
Error: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1808748719-192.168.20.128-1488809333278:blk_1073741826_1002 file=/user/it18zhang/ncdc_data/1902.gz

einhep 发表于 2017-4-14 13:15
jps看下datanode可能有挂掉的。
最好重启下集群。
有的datanode在刚启动的时候是没有问题的，但是一会就 ...

jps过了，DataNode节点也都还在，目前发现在运行日志里有错误，还有异常，哎，好悲催

sstutu 发表于 2017-4-14 16:07
一个个解决，你这里明显有问题。下面红字部分先解决掉。为什么还有gz文件。datanode根本不能保存。
atte ...

我原先看教学视频上是gz文件，如果不是gz文件，我最好应该上传什么文件好啊。

1601365 发表于 2017-4-14 16:05
WordCount程序也一样出错，而且有时候还有Interruptedexception，哎，卡了好几天，要崩溃了

那最好是先把hadoop集群搭建正确了 ，先跑wordcount程序确保集群能够正常运行，再完成自己的mapreduce程序为好。