分享

运行完spark-submit后,master进程自动结束了

xw2016 发表于 2016-6-1 23:15:54 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 2 8879
在spark目录下,运行:bin/spark-submit --class org.apache.spark.examples.SparkPi --master spark://yun01-nn-01:7077  lib/spark-examples-1.5.2-hadoop2.6.0.jar 100


程序运行正常:
in 68 ms on 192.168.56.14 (93/100)
16/06/01 23:09:29 INFO TaskSetManager: Starting task 94.0 in stage 0.0 (TID 94, 192.168.56.14, PROCESS_LOCAL, 2163 bytes)
16/06/01 23:09:29 INFO TaskSetManager: Finished task 93.0 in stage 0.0 (TID 93) in 46 ms on 192.168.56.14 (94/100)
16/06/01 23:09:30 INFO TaskSetManager: Starting task 95.0 in stage 0.0 (TID 95, 192.168.56.14, PROCESS_LOCAL, 2163 bytes)
16/06/01 23:09:30 INFO TaskSetManager: Finished task 94.0 in stage 0.0 (TID 94) in 35 ms on 192.168.56.14 (95/100)
16/06/01 23:09:30 INFO TaskSetManager: Starting task 96.0 in stage 0.0 (TID 96, 192.168.56.14, PROCESS_LOCAL, 2163 bytes)
16/06/01 23:09:30 INFO TaskSetManager: Finished task 95.0 in stage 0.0 (TID 95) in 51 ms on 192.168.56.14 (96/100)
16/06/01 23:09:30 INFO TaskSetManager: Starting task 97.0 in stage 0.0 (TID 97, 192.168.56.14, PROCESS_LOCAL, 2163 bytes)
16/06/01 23:09:30 INFO TaskSetManager: Finished task 96.0 in stage 0.0 (TID 96) in 49 ms on 192.168.56.14 (97/100)
16/06/01 23:09:30 INFO TaskSetManager: Starting task 98.0 in stage 0.0 (TID 98, 192.168.56.14, PROCESS_LOCAL, 2163 bytes)
16/06/01 23:09:30 INFO TaskSetManager: Finished task 97.0 in stage 0.0 (TID 97) in 54 ms on 192.168.56.14 (98/100)
16/06/01 23:09:30 INFO TaskSetManager: Starting task 99.0 in stage 0.0 (TID 99, 192.168.56.14, PROCESS_LOCAL, 2163 bytes)
16/06/01 23:09:30 INFO TaskSetManager: Finished task 98.0 in stage 0.0 (TID 98) in 47 ms on 192.168.56.14 (99/100)
16/06/01 23:09:30 INFO TaskSetManager: Finished task 99.0 in stage 0.0 (TID 99) in 53 ms on 192.168.56.14 (100/100)
16/06/01 23:09:30 INFO DAGScheduler: ResultStage 0 (reduce at SparkPi.scala:36) finished in 17.674 s
16/06/01 23:09:30 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
16/06/01 23:09:30 INFO DAGScheduler: Job 0 finished: reduce at SparkPi.scala:36, took 18.562273 s
Pi is roughly 3.1427672
16/06/01 23:09:30 INFO SparkUI: Stopped Spark web UI at http://192.168.56.14:4040
16/06/01 23:09:30 INFO DAGScheduler: Stopping DAGScheduler
16/06/01 23:09:30 INFO SparkDeploySchedulerBackend: Shutting down all executors
16/06/01 23:09:30 INFO SparkDeploySchedulerBackend: Asking each executor to shut down
16/06/01 23:09:30 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/06/01 23:09:30 INFO MemoryStore: MemoryStore cleared
16/06/01 23:09:30 INFO BlockManager: BlockManager stopped
16/06/01 23:09:30 INFO BlockManagerMaster: BlockManagerMaster stopped
16/06/01 23:09:30 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/06/01 23:09:30 INFO SparkContext: Successfully stopped SparkContext
16/06/01 23:09:30 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/06/01 23:09:30 INFO ShutdownHookManager: Shutdown hook called
16/06/01 23:09:30 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/06/01 23:09:30 INFO ShutdownHookManager: Deleting directory /tmp/spark-8cf3ecbf-2d52-4401-8d6e-05311a03369c


运行结束后,jps查看,发现master进程没有了。

已有(2)人评论

跳转到指定楼层
tntzbzc 发表于 2016-6-2 06:46:07
如果以前正常,运行后关闭,日志应该有记录的。楼主可以定位下。
回复

使用道具 举报

xw2016 发表于 2016-6-2 08:54:44
tntzbzc 发表于 2016-6-2 06:46
如果以前正常,运行后关闭,日志应该有记录的。楼主可以定位下。

嗯,我再仔细看下。
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条