分享

win eclipse 提交job到hadoop集群报错

ltne 发表于 2015-12-28 15:04:08 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 2 10478
用eclipse插件在window编写mapreduce程序,设置
conf.set("fs.defaultFS.name", "192.168.215.142:8020");
conf.set("mapreduce.framework.name", "yarn");
conf.set("yarn.resourcemanager.address", "192.168.215.142:8032");

提交到hadoop,为什么如下错误
Exception in thread "main" java.io.IOException: The ownership on the staging directory /tmp/hadoop-yarn/staging/LT/.staging is not as expected. It is owned by Administrators. The directory must be owned by the submitter LT or by LT
        at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:120)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:437)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
        at org.lt.com.WordCount1.main(WordCount1.java:87)

window的用户为LT,hadoop上的用户为hadoop

已有(2)人评论

跳转到指定楼层
goldtimes 发表于 2015-12-28 18:04:43
本地hadoop版本与集群hadoop版本,还有配置文件是否一致
回复

使用道具 举报

goldtimes 发表于 2015-12-28 18:06:31

解决办法:
1、如果是测试环境,可以取消hadoop hdfs的用户权限检查。打开conf/hdfs-site.xml,找到dfs.permissions属性修改为false(默认为true)OK了。 ,

回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条