spark on yarn 运行命令后,一直循环出现:Application report for application_1468011021285_0005 (state: RUNNING),如下面日志所示:
[hadoop@yun01-dn-01 spark]$ bin/spark-submit --master yarn-cluster --executor-memory 1g --class sparkscala.k3.app.Query /application/hadoop/testjar/query.jar hdfs://ns1/data/log1.txt hdfs://ns1/sparkout/query4
16/07/09 06:11:12 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/07/09 06:11:14 INFO Client: Requesting a new application from cluster with 3 NodeManagers
16/07/09 06:11:14 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container)
16/07/09 06:11:14 INFO Client: Will allocate AM container, with 1408 MB memory including 384 MB overhead
16/07/09 06:11:14 INFO Client: Setting up container launch context for our AM
16/07/09 06:11:14 INFO Client: Setting up the launch environment for our AM container
16/07/09 06:11:14 WARN Client: SPARK_JAR detected in the system environment. This variable has been deprecated in favor of the spark.yarn.jar configuration variable.
16/07/09 06:11:14 INFO Client: Preparing resources for our AM container
16/07/09 06:11:15 WARN Client: SPARK_JAR detected in the system environment. This variable has been deprecated in favor of the spark.yarn.jar configuration variable.
16/07/09 06:11:15 INFO Client: Uploading resource file:/application/hadoop/spark/lib/spark-assembly-1.5.2-hadoop2.6.0.jar -> hdfs://ns1/user/hadoop/.sparkStaging/application_1468011021285_0005/spark-assembly-1.5.2-hadoop2.6.0.jar
16/07/09 06:11:26 INFO Client: Uploading resource file:/application/hadoop/testjar/query.jar -> hdfs://ns1/user/hadoop/.sparkStaging/application_1468011021285_0005/query.jar
16/07/09 06:11:27 INFO Client: Uploading resource file:/tmp/spark-47438339-ed4f-41f5-be4e-dfe5cce71101/__spark_conf__5752978701195266719.zip -> hdfs://ns1/user/hadoop/.sparkStaging/application_1468011021285_0005/__spark_conf__5752978701195266719.zip
16/07/09 06:11:27 INFO SecurityManager: Changing view acls to: hadoop
16/07/09 06:11:27 INFO SecurityManager: Changing modify acls to: hadoop
16/07/09 06:11:27 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); users with modify permissions: Set(hadoop)
16/07/09 06:11:28 INFO Client: Submitting application 5 to ResourceManager
16/07/09 06:11:28 INFO YarnClientImpl: Submitted application application_1468011021285_0005
16/07/09 06:11:29 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:29 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1468015895663
final status: UNDEFINED
tracking URL: http://yun01-nn-01:8088/proxy/application_1468011021285_0005/
user: hadoop
16/07/09 06:11:30 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:31 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:32 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:33 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:34 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:35 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:36 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:37 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:38 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:39 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:40 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:41 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:42 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:43 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:44 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:45 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:46 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:47 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:48 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:49 INFO Client: Application report for application_1468011021285_0005 (state: ACCEPTED)
16/07/09 06:11:50 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:50 INFO Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: 192.168.56.14
ApplicationMaster RPC port: 0
queue: default
start time: 1468015895663
final status: UNDEFINED
tracking URL: http://yun01-nn-01:8088/proxy/application_1468011021285_0005/
user: hadoop
16/07/09 06:11:51 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:52 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:53 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:54 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:55 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:56 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:57 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:58 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:11:59 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:12:00 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:12:01 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:12:03 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)
16/07/09 06:12:04 INFO Client: Application report for application_1468011021285_0005 (state: RUNNING)