分享

spark提交application,Initial job has not accepted any resources

jason_Q 发表于 2016-5-19 09:17:06 [显示全部楼层] 只看大图 回帖奖励 阅读模式 关闭右栏 8 13931
spark-submit提交程序,一直处于waiting状态
日志打印:
16/05/19 09:10:29 INFO scheduler.TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
16/05/19 09:10:44 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
16/05/19 09:10:59 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
16/05/19 09:11:14 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
16/05/19 09:11:29 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
16/05/19 09:11:44 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
16/05/19 09:11:59 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
16/05/19 09:12:14 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
16/05/19 09:12:29 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

QQ图片20160519091407.png

求指点是什么问题导致的。是系统资源不够吗

已有(8)人评论

跳转到指定楼层
qcbb001 发表于 2016-5-19 09:58:46
是否内存过小


回复

使用道具 举报

jason_Q 发表于 2016-5-19 10:01:44

指的什么内存?是系统内存还是进程可用内存
回复

使用道具 举报

nextuser 发表于 2016-5-19 10:07:16
1、WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster uito ensure that workers are registered and have sufficient memory
当前的集群的可用资源不能满足应用程序所请求的资源。
资源分2类: cores 和 ram
Core代表对执行可用的executor slots
Ram代表每个Worker上被需要的空闲内存来运行你的Application。
解决方法:
应用不要请求多余空闲可用资源的
关闭掉已经执行结束的Application


来自
Spark 常见问题小结
http://www.aboutyun.com/forum.php?mod=viewthread&tid=9946


回复

使用道具 举报

NEOGX 发表于 2016-5-19 10:10:13
检测下防火墙 ,客户端是否开启对80端口的访问,是否开启了spark的端口等
回复

使用道具 举报

xw2016 发表于 2016-6-5 13:42:58
资源不够,内存太小了
回复

使用道具 举报

xw2016 发表于 2016-6-5 13:44:24
楼主可以用命令查一下机器的空闲内存
回复

使用道具 举报

wwq 发表于 2016-6-5 14:51:36
我遇到过  是你的内存不够 不是太小 我估计你是用虚拟机给整的,你关掉windows的那些没用的进程,然后启动sparkshell 的时候,记得将executor的大小设置一下 不要超过自己所给的内存
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条