hduser@master:~/hadoop2.6/spark1.6/sbin$ spark-submit --master spark://master:7070 --deploy-mode cluster --name "wordcount" --class WordCount /home/hduser/files/wordcount.jar hdfs://master:9000/wordcount/input hdfs://master:9000/wordcount/output
Running Spark using the REST application submission protocol.
17/09/30 11:46:25 INFO rest.RestSubmissionClient: Submitting a request to launch an application in spark://master:7070.
17/09/30 11:46:25 WARN rest.RestSubmissionClient: Unable to connect to server spark://master:7070.
Warning: Master endpoint spark://master:7070 was not a REST server. Falling back to legacy submission gateway instead.
17/09/30 11:46:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.io.IOException: Failed to connect to master/172.16.97.213:7070
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:167)
at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:200)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:187)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:183)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused: master/172.16.97.213:7070
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
... 1 more
Starting Nmap 7.01 ( https://nmap.org ) at 2017-09-30 11:42 EDT
Nmap scan report for master (172.16.97.213)
Host is up (0.00095s latency).
PORT STATE SERVICE
7077/tcp open unknown
Nmap done: 1 IP address (1 host up) scanned in 0.03 seconds
host配置如下:
ff02::1 ip6-allnodes
# The following lines are desirable for IPv6 capable hosts
ff02::2 ip6-allrouters
::1 localhost ip6-localhost ip6-loopback
172.16.97.213 master.cloud.mos master
172.16.97.216 node2
172.16.97.215 node1
#127.0.0.1 localhost