oozie 的配置真心繁琐,而且bug多多。
现在只剩下最后一个问题了。oozie 调用Hive ,报错如下:
2016-07-04 16:30:15,144 WARN ActionStartXCommand:523 - SERVER[node2] USER[root] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000004-160704143026776-oozie-root-W] ACTION[0000004-160704143026776-oozie-root-W@hive-node] Error starting action [hive-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: Unknown rpc kind in rpc headerRPC_WRITABLE]
org.apache.oozie.action.ActionExecutorException: JA009: Unknown rpc kind in rpc headerRPC_WRITABLE
at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:462)
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:436)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1008)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1162)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:64)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RpcServerException): Unknown rpc kind in rpc headerRPC_WRITABLE
at org.apache.hadoop.ipc.Client.call(Client.java:1468)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:244)
at org.apache.hadoop.mapred.$Proxy29.getDelegationToken(Unknown Source)
at org.apache.hadoop.mapred.JobClient.getDelegationToken(JobClient.java:2153)
at org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:375)
at org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1205)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:956)
... 8 more
2016-07-04 16:30:15,147 WARN ActionStartXCommand:523 - SERVER[node2] USER[root] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000004-160704143026776-oozie-root-W] ACTION[0000004-160704143026776-oozie-root-W@hive-node] Exceeded max retry count [3]. Suspending Job
2016-07-04 16:30:15,147 WARN ActionStartXCommand:523 - SERVER[node2] USER[root] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000004-160704143026776-oozie-root-W] ACTION[0000004-160704143026776-oozie-root-W@hive-node] Suspending Workflow Job id=0000004-160704143026776-oozie-root-W
网上找了一下,说是这个问题是由于oozie 和 hadoop分别 以 mapreduce 和 yarn 的不同方式运行引起的,需要设置 oozie 以yarn模式运行。
即 在 oozie-env.sh 中设置 OOZIE_BASE=/usr/local/modules/oozie/oozie-server
在job.properties 中设置 jobTracker=node1:8032 为yarn的端口 然后 是解压 oozie-sharelib-4.1.0-cdh5.4.3-yarn.tar.gz 并上传share到 hdfs。
这些我都已经完成了,但是还是报同样的错误。我还有遗漏的地方吗,或者根本不是这个问题引起的。请多多指教。多谢!
|