18/04/02 15:48:31 INFO storage.BlockManagerMaster: Removal of executor 29 requested
18/04/02 15:48:31 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asked to remove non-existent executor 29
18/04/02 15:48:31 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 29 from BlockManagerMaster.
18/04/02 15:48:33 WARN cluster.YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
18/04/02 15:48:37 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Requesting driver to remove executor 30 for reason Container marked as failed: container_e49_1522654847559_0001_02_000034 on host: Iwanvi19. Exit status: -1000. Diagnostics: Not able to initialize app-cache directories in any of the configured local directories for user root
18/04/02 15:48:37 INFO storage.BlockManagerMaster: Removal of executor 30 requested
18/04/02 15:48:37 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Asked to remove non-existent executor 30
18/04/02 15:48:37 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 30 from BlockManagerMaster.
18/04/02 15:48:43 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Requesting driver to remove executor 31 for reason Container marked as failed: container_e49_1522654847559_0001_02_000035 on host: Iwanvi19. Exit status: -1000. Diagnostics: Not able to initialize app-cache directories in any of the configured local directories for user root
你的日志提示这样的错误:“18/04/02 15:48:33 WARN cluster.YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources”。提示你资源不够,你打开spark的web页面,再跑一次你这个脚本,看看资源消耗情况,如果脚本没有跑完,资源就耗光了,你再修改资源参数
你的日志提示这样的错误:“18/04/02 15:48:33 WARN cluster.YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources”。提示你资源不够,你打开spark的web页面,再跑一次你这个脚本,看看资源消耗情况,如果脚本没有跑完,资源就耗光了,你再修改资源参数