分享

计算圆周率出现如下错误

liyansong 发表于 2016-5-26 16:23:58 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 1 5584
[root@cluster-template-001 .ssh]# hadoop namenode -format
16/05/27 12:04:23 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = cluster-template-001.novalocal/172.16.100.149
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 1.2.1
STARTUP_MSG:   build = https://svn.apache.org/repos/asf ... branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:27:42 PDT 2013
STARTUP_MSG:   java = 1.7.0_51
************************************************************/
16/05/27 12:04:24 INFO util.GSet: Computing capacity for map BlocksMap
16/05/27 12:04:24 INFO util.GSet: VM type       = 64-bit
16/05/27 12:04:24 INFO util.GSet: 2.0% max memory = 129761280
16/05/27 12:04:24 INFO util.GSet: capacity      = 2^18 = 262144 entries
16/05/27 12:04:24 INFO util.GSet: recommended=262144, actual=262144
16/05/27 12:04:24 INFO namenode.FSNamesystem: fsOwner=root
16/05/27 12:04:24 INFO namenode.FSNamesystem: supergroup=supergroup
16/05/27 12:04:24 INFO namenode.FSNamesystem: isPermissionEnabled=false
16/05/27 12:04:24 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
16/05/27 12:04:24 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
16/05/27 12:04:24 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0
16/05/27 12:04:24 INFO namenode.NameNode: Caching file names occuring more than 10 times
16/05/27 12:04:25 INFO common.Storage: Image file file:/home/hadoop/tmp/dfs/name/current/fsimage of size 110 bytes saved in 0 seconds.
16/05/27 12:04:25 INFO namenode.FSEditLog: closing edit log: position=4, editlog=file:/home/hadoop/tmp/dfs/name/current/edits
16/05/27 12:04:25 INFO namenode.FSEditLog: close success: truncate to 4, editlog=file:/home/hadoop/tmp/dfs/name/current/edits
16/05/27 12:04:25 INFO common.Storage: Storage directory file:/home/hadoop/tmp/dfs/name has been successfully formatted.
16/05/27 12:04:25 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at cluster-template-001.novalocal/172.16.100.149
************************************************************/
[root@cluster-template-001 .ssh]# cd /usr/sbin/
[root@cluster-template-001 sbin]# ./start-all.sh
starting namenode, logging to /mnt/log/hadoop/root//hadoop-root-namenode-cluster-template-001.novalocal.out
localhost: Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
localhost: starting datanode, logging to /mnt/log/hadoop/root//hadoop-root-datanode-cluster-template-001.novalocal.out
localhost: starting secondarynamenode, logging to /mnt/log/hadoop/root//hadoop-root-secondarynamenode-cluster-template-001.novalocal.out
starting jobtracker, logging to /mnt/log/hadoop/root//hadoop-root-jobtracker-cluster-template-001.novalocal.out
localhost: starting tasktracker, logging to /mnt/log/hadoop/root//hadoop-root-tasktracker-cluster-template-001.novalocal.out
[root@cluster-template-001 sbin]# jps
-bash: jps: command not found
[root@cluster-template-001 sbin]# cd /usr/share/hadoop/
[root@cluster-template-001 hadoop]# hadoop jar hadoop-examples-1.2.1.jar pi 5 25
Number of Maps  = 5
Samples per Map = 25
16/05/27 12:05:13 INFO util.NativeCodeLoader: Loaded the native-hadoop library
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Starting Job
16/05/27 12:05:14 INFO mapred.FileInputFormat: Total input paths to process : 5
16/05/27 12:05:14 INFO mapred.JobClient: Running job: job_201605271204_0001
16/05/27 12:05:15 INFO mapred.JobClient:  map 0% reduce 0%
16/05/27 12:05:16 INFO mapred.JobClient: Task Id : attempt_201605271204_0001_m_000006_0, Status : FAILED
Error initializing attempt_201605271204_0001_m_000006_0:
java.io.FileNotFoundException: file:/home/hadoop/tmp/mapred/local/taskTracker/root/jobcache/job_201605271204_0001/jars/job.jar (No such file or directory)
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:215)
        at java.util.zip.ZipFile.<init>(ZipFile.java:145)
        at java.util.jar.JarFile.<init>(JarFile.java:153)
        at java.util.jar.JarFile.<init>(JarFile.java:117)
        at org.apache.hadoop.util.RunJar.unJar(RunJar.java:36)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
        at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:211)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1336)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1311)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
        at java.lang.Thread.run(Thread.java:744)

16/05/27 12:05:17 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 0&filter=stdout
16/05/27 12:05:17 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 0&filter=stderr
16/05/27 12:05:18 INFO mapred.JobClient: Task Id : attempt_201605271204_0001_m_000006_1, Status : FAILED
Error initializing attempt_201605271204_0001_m_000006_1:
java.io.FileNotFoundException: file:/home/hadoop/tmp/mapred/local/taskTracker/root/jobcache/job_201605271204_0001/jars/job.jar (No such file or directory)
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:215)
        at java.util.zip.ZipFile.<init>(ZipFile.java:145)
        at java.util.jar.JarFile.<init>(JarFile.java:153)
        at java.util.jar.JarFile.<init>(JarFile.java:117)
        at org.apache.hadoop.util.RunJar.unJar(RunJar.java:36)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
        at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:211)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1336)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1311)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
        at java.lang.Thread.run(Thread.java:744)

16/05/27 12:05:18 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 1&filter=stdout
16/05/27 12:05:18 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 1&filter=stderr
16/05/27 12:05:18 INFO mapred.JobClient: Task Id : attempt_201605271204_0001_m_000006_2, Status : FAILED
Error initializing attempt_201605271204_0001_m_000006_2:
java.io.FileNotFoundException: file:/home/hadoop/tmp/mapred/local/taskTracker/root/jobcache/job_201605271204_0001/jars/job.jar (No such file or directory)
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:215)
        at java.util.zip.ZipFile.<init>(ZipFile.java:145)
        at java.util.jar.JarFile.<init>(JarFile.java:153)
        at java.util.jar.JarFile.<init>(JarFile.java:117)
        at org.apache.hadoop.util.RunJar.unJar(RunJar.java:36)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
        at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:211)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1336)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1311)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
        at java.lang.Thread.run(Thread.java:744)

16/05/27 12:05:18 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 2&filter=stdout
16/05/27 12:05:18 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 2&filter=stderr
16/05/27 12:05:19 INFO mapred.JobClient: Task Id : attempt_201605271204_0001_m_000005_0, Status : FAILED
Error initializing attempt_201605271204_0001_m_000005_0:
java.io.FileNotFoundException: file:/home/hadoop/tmp/mapred/local/taskTracker/root/jobcache/job_201605271204_0001/jars/job.jar (No such file or directory)
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:215)
        at java.util.zip.ZipFile.<init>(ZipFile.java:145)
        at java.util.jar.JarFile.<init>(JarFile.java:153)
        at java.util.jar.JarFile.<init>(JarFile.java:117)
        at org.apache.hadoop.util.RunJar.unJar(RunJar.java:36)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
        at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:211)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1336)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1311)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
        at java.lang.Thread.run(Thread.java:744)

16/05/27 12:05:19 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 0&filter=stdout
16/05/27 12:05:19 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 0&filter=stderr
16/05/27 12:05:19 INFO mapred.JobClient: Task Id : attempt_201605271204_0001_m_000005_1, Status : FAILED
Error initializing attempt_201605271204_0001_m_000005_1:
java.io.FileNotFoundException: file:/home/hadoop/tmp/mapred/local/taskTracker/root/jobcache/job_201605271204_0001/jars/job.jar (No such file or directory)
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:215)
        at java.util.zip.ZipFile.<init>(ZipFile.java:145)
        at java.util.jar.JarFile.<init>(JarFile.java:153)
        at java.util.jar.JarFile.<init>(JarFile.java:117)
        at org.apache.hadoop.util.RunJar.unJar(RunJar.java:36)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
        at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:211)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1336)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1311)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
        at java.lang.Thread.run(Thread.java:744)

16/05/27 12:05:19 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 1&filter=stdout
16/05/27 12:05:19 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 1&filter=stderr
16/05/27 12:05:19 INFO mapred.JobClient: Task Id : attempt_201605271204_0001_m_000005_2, Status : FAILED
Error initializing attempt_201605271204_0001_m_000005_2:
java.io.FileNotFoundException: file:/home/hadoop/tmp/mapred/local/taskTracker/root/jobcache/job_201605271204_0001/jars/job.jar (No such file or directory)
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:215)
        at java.util.zip.ZipFile.<init>(ZipFile.java:145)
        at java.util.jar.JarFile.<init>(JarFile.java:153)
        at java.util.jar.JarFile.<init>(JarFile.java:117)
        at org.apache.hadoop.util.RunJar.unJar(RunJar.java:36)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
        at org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
        at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:211)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1336)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1311)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
        at java.lang.Thread.run(Thread.java:744)

16/05/27 12:05:19 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 2&filter=stdout
16/05/27 12:05:19 WARN mapred.JobClient: Error reading task outputhttp://cluster-template-001.nova ... 2&filter=stderr
16/05/27 12:05:19 INFO mapred.JobClient: Job complete: job_201605271204_0001
16/05/27 12:05:19 INFO mapred.JobClient: Counters: 4
16/05/27 12:05:19 INFO mapred.JobClient:   Job Counters
16/05/27 12:05:19 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=0
16/05/27 12:05:19 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
16/05/27 12:05:19 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
16/05/27 12:05:19 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
16/05/27 12:05:19 INFO mapred.JobClient: Job Failed: JobCleanup Task Failure, Task: task_201605271204_0001_m_000005
java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1357)
        at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:297)
        at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
[root@cluster-template-001 hadoop]# ^C
[root@cluster-template-001 hadoop]#
[root@cluster-template-001 hadoop]#

已有(1)人评论

跳转到指定楼层
einhep 发表于 2016-5-26 16:54:28
第一更改下临时目录
file:/home/hadoop/tmp/mapred/local/taskTracker/root/jobcache/job_201605271204_0001/jars/job.jar
这个系统重启后很容易删掉

第二
ssh似乎有问题,先确保ssh相互登录没有问题
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条