分享

hive 调hadoop job 挂掉,帮忙看下

ananan36 发表于 2016-6-15 11:43:38 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 4 10796
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 141, 1.247908E8]
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 141, 1.25288209E8]
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 141, 1.25288209E8]
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 4000 rows for join key [38400, 141, 1.25288209E8]
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 143, 5.4859921E7]
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 143, 5.4859921E7]
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 4000 rows for join key [38400, 143, 5.4859921E7]
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 145, 5.5051923E7]
16/06/15 00:38:28 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 145, 5.5051923E7]
16/06/15 00:38:29 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 15, 1.08341521E8]
16/06/15 00:38:29 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 15, 1.09171459E8]
16/06/15 00:38:29 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 15, 1.09171459E8]
16/06/15 00:38:29 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 15, 1.11117586E8]
16/06/15 00:38:30 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 150, 2.16763137E8]
16/06/15 00:38:30 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 156, 1.09598722E8]
16/06/15 00:38:30 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 156, 1.09598722E8]
16/06/15 00:38:30 GMT INFO exec.CommonJoinOperator: table 0 has 4000 rows for join key [38400, 156, 1.09598722E8]
16/06/15 00:38:30 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 156, 2.16644355E8]
16/06/15 00:38:30 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 156, 2.16644355E8]
16/06/15 00:38:31 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 158, 5.9470212E7]
16/06/15 00:38:31 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 158, 5.9470212E7]
16/06/15 00:38:31 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 158, 2.59105283E8]
16/06/15 00:38:31 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 158, 2.59105283E8]
16/06/15 00:38:31 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 161, 1.09495553E8]
16/06/15 00:38:31 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 161, 1.24027922E8]
16/06/15 00:38:31 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 161, 1.24027922E8]
16/06/15 00:38:31 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 163, 1.07559441E8]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 165, 1.07852841E8]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 165, 1.07852841E8]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 165, 1.24015888E8]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 165, 1.24884997E8]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 167, 1.23898625E8]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 167, 1.23898625E8]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 169, 5.7258884E7]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 17, 5.5305106E7]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 17, 5.7949059E7]
16/06/15 00:38:32 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 17, 2.60157713E8]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 170, 2.5912013E8]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 170, 2.5912013E8]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 172, 1.08639489E8]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 172, 1.08639489E8]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 172, 1.09729281E8]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 172, 2.60759056E8]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 174, 5.5437458E7]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 174, 5.5437458E7]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 4000 rows for join key [38400, 174, 5.5437458E7]
16/06/15 00:38:33 GMT INFO exec.CommonJoinOperator: table 0 has 8000 rows for join key [38400, 174, 5.5437458E7]
16/06/15 00:38:34 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 174, 5.6033923E7]
16/06/15 00:38:34 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 174, 5.6033923E7]
16/06/15 00:38:34 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 176, 2.52817424E8]
16/06/15 00:38:34 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 176, 2.52817424E8]
16/06/15 00:38:35 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 178, 2.59026436E8]
16/06/15 00:38:35 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 183, 1.11027973E8]
16/06/15 00:38:35 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 183, 1.24064017E8]
16/06/15 00:38:35 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 183, 1.24196113E8]
16/06/15 00:38:36 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 189, 2.59985154E8]
16/06/15 00:38:36 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 192, 1.0942669E8]
16/06/15 00:38:36 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 192, 1.09428739E8]
16/06/15 00:38:36 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 196, 1.24871697E8]
16/06/15 00:38:36 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 196, 1.24871697E8]
16/06/15 00:38:37 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 198, 1.24523026E8]
16/06/15 00:38:37 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 198, 1.24523026E8]
16/06/15 00:38:37 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 20, 5.901301E7]
16/06/15 00:38:37 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 20, 1.24126994E8]
16/06/15 00:38:38 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 200, 2.59802627E8]
16/06/15 00:38:38 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 200, 2.59802627E8]
16/06/15 00:38:38 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 202, 1.08354306E8]
16/06/15 00:38:38 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 202, 1.24201488E8]
16/06/15 00:38:38 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 202, 1.24201488E8]
16/06/15 00:38:38 GMT INFO exec.CommonJoinOperator: table 0 has 4000 rows for join key [38400, 202, 1.24201488E8]
16/06/15 00:38:38 GMT INFO exec.CommonJoinOperator: table 0 has 8000 rows for join key [38400, 202, 1.24201488E8]
16/06/15 00:38:38 GMT INFO exec.CommonJoinOperator: table 0 has 16000 rows for join key [38400, 202, 1.24201488E8]
16/06/15 00:38:39 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 204, 1.08534019E8]
16/06/15 00:38:39 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 204, 1.11076867E8]
16/06/15 00:38:39 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 204, 1.25242114E8]
16/06/15 00:38:39 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 204, 1.25242114E8]
16/06/15 00:38:39 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 206, 5.8720131E7]
16/06/15 00:38:39 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 206, 2.59056389E8]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 208, 5.5634561E7]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 208, 5.5634561E7]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 211, 5.5416723E7]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 211, 5.5416723E7]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 211, 5.6673668E7]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 211, 5.6673668E7]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 4000 rows for join key [38400, 211, 5.6673668E7]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 215, 5.549325E7]
16/06/15 00:38:40 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 215, 5.549325E7]
16/06/15 00:38:41 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 217, 5.7464466E7]
16/06/15 00:38:41 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 219, 1.08299524E8]
16/06/15 00:38:41 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 219, 1.08299524E8]
16/06/15 00:38:41 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 219, 1.09515025E8]
16/06/15 00:38:41 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 219, 1.09614097E8]
16/06/15 00:38:42 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 219, 1.09614097E8]
16/06/15 00:38:42 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 219, 1.10010385E8]
16/06/15 00:38:42 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 219, 1.24243729E8]
16/06/15 00:38:42 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 219, 1.24243729E8]
16/06/15 00:38:42 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 219, 1.25036305E8]
16/06/15 00:38:42 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 220, 5.9023761E7]
16/06/15 00:38:42 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 220, 1.10080272E8]
16/06/15 00:38:42 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 220, 1.25420072E8]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 222, 1.10918416E8]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 222, 1.10918416E8]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 222, 1.10920705E8]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 222, 1.10920705E8]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 226, 5.8695825E7]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 226, 5.8695825E7]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 228, 5.4823826E7]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 228, 1.09125385E8]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 228, 1.09144066E8]
16/06/15 00:38:43 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 228, 1.24283433E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 231, 1.24450321E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 231, 1.24450321E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 233, 1.08613634E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 233, 1.09443332E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 233, 1.09443332E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 4000 rows for join key [38400, 233, 1.09443332E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 8000 rows for join key [38400, 233, 1.09443332E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 233, 1.09709573E8]
16/06/15 00:38:44 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 233, 1.09709573E8]
16/06/15 00:38:45 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 235, 5.5440018E7]
16/06/15 00:38:45 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 235, 1.24547844E8]
16/06/15 00:38:45 GMT INFO exec.CommonJoinOperator: table 0 has 1000 rows for join key [38400, 239, 1.07444025E8]
16/06/15 00:38:45 GMT INFO exec.CommonJoinOperator: table 0 has 2000 rows for join key [38400, 239, 1.07444025E8]
16/06/15 00:38:45 GMT INFO exec.CommonJoinOperator: table 0 has 4000 rows for join key [38400, 239, 1.07444025E8]
16/06/15 00:38:45 GMT INFO exec.CommonJoinOperator: table 0 has 8000 rows for join key [38400, 239, 1.07444025E8]
16/06/15 00:38:45 GMT INFO exec.CommonJoinOperator: table 0 has 16000 rows for join key [38400, 239, 1.07444025E8]
16/06/15 00:38:45 GMT ERROR persistence.RowContainer: java.io.IOException: No such file or directory
java.io.IOException: No such file or directory
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.createNewFile(File.java:1006)
        at java.io.File.createTempFile(File.java:1989)
        at java.io.File.createTempFile(File.java:2040)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.setupWriter(RowContainer.java:520)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:294)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.addRow(RowContainer.java:165)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.addRow(RowContainer.java:74)
        at org.apache.hadoop.hive.ql.exec.JoinOperator.process(JoinOperator.java:125)
        at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:235)
        at com.platform.mapreduce.MRServiceJava.reducer.ReduceTask$InternalRunnerOld.runInternal(ReduceTask.java:546)
        at com.platform.mapreduce.MRServiceJava.reducer.ReduceTask.run(ReduceTask.java:240)
        at com.platform.mapreduce.MRServiceJava.MRService$2.run(MRService.java:590)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at com.platform.mapreduce.work.adaptor.CredentialsAdaptor.doAs(CredentialsAdaptor.java:296)
        at com.platform.mapreduce.MRServiceJava.MRService.onInvoke(MRService.java:584)
        at com.platform.symphony.soam.ServiceContainer.invoke(ServiceContainer.java:399)
        at com.platform.symphony.soam.SoamCallbackWorkerThread.nativeProcessNextCallbackEvent(Native Method)
        at com.platform.symphony.soam.SoamCallbackWorkerThread.run(SoamCallbackWorkerThread.java:77)
16/06/15 00:38:45 GMT ERROR persistence.RowContainer: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: No such file or directory
org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: No such file or directory
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.setupWriter(RowContainer.java:546)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:294)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.addRow(RowContainer.java:165)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.addRow(RowContainer.java:74)
        at org.apache.hadoop.hive.ql.exec.JoinOperator.process(JoinOperator.java:125)
        at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:235)
        at com.platform.mapreduce.MRServiceJava.reducer.ReduceTask$InternalRunnerOld.runInternal(ReduceTask.java:546)
        at com.platform.mapreduce.MRServiceJava.reducer.ReduceTask.run(ReduceTask.java:240)
        at com.platform.mapreduce.MRServiceJava.MRService$2.run(MRService.java:590)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at com.platform.mapreduce.work.adaptor.CredentialsAdaptor.doAs(CredentialsAdaptor.java:296)
        at com.platform.mapreduce.MRServiceJava.MRService.onInvoke(MRService.java:584)
        at com.platform.symphony.soam.ServiceContainer.invoke(ServiceContainer.java:399)
        at com.platform.symphony.soam.SoamCallbackWorkerThread.nativeProcessNextCallbackEvent(Native Method)
        at com.platform.symphony.soam.SoamCallbackWorkerThread.run(SoamCallbackWorkerThread.java:77)
Caused by: java.io.IOException: No such file or directory
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.createNewFile(File.java:1006)
        at java.io.File.createTempFile(File.java:1989)
        at java.io.File.createTempFile(File.java:2040)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.setupWriter(RowContainer.java:520)
        ... 16 more
16/06/15 00:38:45 GMT FATAL ExecReducer: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":"38400","reducesinkkey1":"239","reducesinkkey2":1.07444025E8},"value":{"_col0":4370146284496657,"_col1":"","_col2":"DATANG","_col3":"","_col4":"","_col5":"","_col6":"","_col7":"","_col8":"","_col9":419703,"_col10":"","_col11":107444025,"_col12":25183,"_col13":"451","_col14":"162","_col15":"2016-06-12 23:23:54.240","_col16":"","_col17":"","_col18":"","_col19":"","_col20":"26","_col21":"18","_col22":"0","_col23":"10.210.53.132","_col24":"39148","_col25":"31","_col26":"145","_col27":"4","_col28":"48","_col29":"","_col30":"","_col31":"","_col32":"","_col33":"","_col34":"","_col35":"","_col36":"","_col37":"","_col38":"","_col39":"","_col40":"","_col41":"","_col42":"","_col43":"","_col44":"","_col45":"","_col46":"","_col47":"4","_col48":"0","_col49":"8","_col50":"0","_col51":"0","_col52":"","_col53":"66","_col54":"27","_col55":"33","_col56":"0","_col57":"","_col58":"","_col59":""}}
        at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:244)
        at com.platform.mapreduce.MRServiceJava.reducer.ReduceTask$InternalRunnerOld.runInternal(ReduceTask.java:546)
        at com.platform.mapreduce.MRServiceJava.reducer.ReduceTask.run(ReduceTask.java:240)
        at com.platform.mapreduce.MRServiceJava.MRService$2.run(MRService.java:590)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at com.platform.mapreduce.work.adaptor.CredentialsAdaptor.doAs(CredentialsAdaptor.java:296)
        at com.platform.mapreduce.MRServiceJava.MRService.onInvoke(MRService.java:584)
        at com.platform.symphony.soam.ServiceContainer.invoke(ServiceContainer.java:399)
        at com.platform.symphony.soam.SoamCallbackWorkerThread.nativeProcessNextCallbackEvent(Native Method)
        at com.platform.symphony.soam.SoamCallbackWorkerThread.run(SoamCallbackWorkerThread.java:77)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: No such file or directory
        at org.apache.hadoop.hive.ql.exec.JoinOperator.process(JoinOperator.java:128)
        at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:235)
        ... 11 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: No such file or directory
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.setupWriter(RowContainer.java:546)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:294)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.addRow(RowContainer.java:165)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.addRow(RowContainer.java:74)
        at org.apache.hadoop.hive.ql.exec.JoinOperator.process(JoinOperator.java:125)
        ... 12 more
Caused by: java.io.IOException: No such file or directory
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.createNewFile(File.java:1006)
        at java.io.File.createTempFile(File.java:1989)
        at java.io.File.createTempFile(File.java:2040)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.setupWriter(RowContainer.java:520)
        ... 16 more

16/06/15 00:38:46 GMT INFO reducer.ReduceTask: deleted <1> in the working dir
16/06/15 00:38:46 GMT INFO MRServiceJava.TaskReporter: Start to stop reporting thread.
16/06/15 00:38:46 GMT INFO MRServiceJava.TaskReporter: Reporting thread has been stopped.
16/06/15 00:38:46 GMT INFO MRServiceJava.Task: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":"38400","reducesinkkey1":"239","reducesinkkey2":1.07444025E8},"value":{"_col0":4370146284496657,"_col1":"","_col2":"DATANG","_col3":"","_col4":"","_col5":"","_col6":"","_col7":"","_col8":"","_col9":419703,"_col10":"","_col11":107444025,"_col12":25183,"_col13":"451","_col14":"162","_col15":"2016-06-12 23:23:54.240","_col16":"","_col17":"","_col18":"","_col19":"","_col20":"26","_col21":"18","_col22":"0","_col23":"10.210.53.132","_col24":"39148","_col25":"31","_col26":"145","_col27":"4","_col28":"48","_col29":"","_col30":"","_col31":"","_col32":"","_col33":"","_col34":"","_col35":"","_col36":"","_col37":"","_col38":"","_col39":"","_col40":"","_col41":"","_col42":"","_col43":"","_col44":"","_col45":"","_col46":"","_col47":"4","_col48":"0","_col49":"8","_col50":"0","_col51":"0","_col52":"","_col53":"66","_col54":"27","_col55":"33","_col56":"0","_col57":"","_col58":"","_col59":""}}
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":"38400","reducesinkkey1":"239","reducesinkkey2":1.07444025E8},"value":{"_col0":4370146284496657,"_col1":"","_col2":"DATANG","_col3":"","_col4":"","_col5":"","_col6":"","_col7":"","_col8":"","_col9":419703,"_col10":"","_col11":107444025,"_col12":25183,"_col13":"451","_col14":"162","_col15":"2016-06-12 23:23:54.240","_col16":"","_col17":"","_col18":"","_col19":"","_col20":"26","_col21":"18","_col22":"0","_col23":"10.210.53.132","_col24":"39148","_col25":"31","_col26":"145","_col27":"4","_col28":"48","_col29":"","_col30":"","_col31":"","_col32":"","_col33":"","_col34":"","_col35":"","_col36":"","_col37":"","_col38":"","_col39":"","_col40":"","_col41":"","_col42":"","_col43":"","_col44":"","_col45":"","_col46":"","_col47":"4","_col48":"0","_col49":"8","_col50":"0","_col51":"0","_col52":"","_col53":"66","_col54":"27","_col55":"33","_col56":"0","_col57":"","_col58":"","_col59":""}}
        at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:256)
        at com.platform.mapreduce.MRServiceJava.reducer.ReduceTask$InternalRunnerOld.runInternal(ReduceTask.java:546)
        at com.platform.mapreduce.MRServiceJava.reducer.ReduceTask.run(ReduceTask.java:240)
        at com.platform.mapreduce.MRServiceJava.MRService$2.run(MRService.java:590)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at com.platform.mapreduce.work.adaptor.CredentialsAdaptor.doAs(CredentialsAdaptor.java:296)
        at com.platform.mapreduce.MRServiceJava.MRService.onInvoke(MRService.java:584)
        at com.platform.symphony.soam.ServiceContainer.invoke(ServiceContainer.java:399)
        at com.platform.symphony.soam.SoamCallbackWorkerThread.nativeProcessNextCallbackEvent(Native Method)
        at com.platform.symphony.soam.SoamCallbackWorkerThread.run(SoamCallbackWorkerThread.java:77)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":"38400","reducesinkkey1":"239","reducesinkkey2":1.07444025E8},"value":{"_col0":4370146284496657,"_col1":"","_col2":"DATANG","_col3":"","_col4":"","_col5":"","_col6":"","_col7":"","_col8":"","_col9":419703,"_col10":"","_col11":107444025,"_col12":25183,"_col13":"451","_col14":"162","_col15":"2016-06-12 23:23:54.240","_col16":"","_col17":"","_col18":"","_col19":"","_col20":"26","_col21":"18","_col22":"0","_col23":"10.210.53.132","_col24":"39148","_col25":"31","_col26":"145","_col27":"4","_col28":"48","_col29":"","_col30":"","_col31":"","_col32":"","_col33":"","_col34":"","_col35":"","_col36":"","_col37":"","_col38":"","_col39":"","_col40":"","_col41":"","_col42":"","_col43":"","_col44":"","_col45":"","_col46":"","_col47":"4","_col48":"0","_col49":"8","_col50":"0","_col51":"0","_col52":"","_col53":"66","_col54":"27","_col55":"33","_col56":"0","_col57":"","_col58":"","_col59":""}}
        at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:244)
        ... 11 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: No such file or directory
        at org.apache.hadoop.hive.ql.exec.JoinOperator.process(JoinOperator.java:128)
        at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:235)
        ... 11 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: No such file or directory
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.setupWriter(RowContainer.java:546)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:294)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.addRow(RowContainer.java:165)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.addRow(RowContainer.java:74)
        at org.apache.hadoop.hive.ql.exec.JoinOperator.process(JoinOperator.java:125)
        ... 12 more
Caused by: java.io.IOException: No such file or directory
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.createNewFile(File.java:1006)
        at java.io.File.createTempFile(File.java:1989)
        at java.io.File.createTempFile(File.java:2040)
        at org.apache.hadoop.hive.ql.exec.persistence.RowContainer.setupWriter(RowContainer.java:520)
        ... 16 more

已有(4)人评论

跳转到指定楼层
arsenduan 发表于 2016-6-15 16:22:06
楼主怎么调用的。是否尝试运行其它mapreduce或则hive job。
报错一直找不到文件以及hive元数据库错误。
首先确保hive是正常的,然后输入文件检测下,权限是否正常等因素
回复

使用道具 举报

ananan36 发表于 2016-6-15 18:11:10
arsenduan 发表于 2016-6-15 16:22
楼主怎么调用的。是否尝试运行其它mapreduce或则hive job。
报错一直找不到文件以及hive元数据库错误。
...

就是执行SQL报的错 看reduce日志报的错。HIVE没问题。
回复

使用道具 举报

langke93 发表于 2016-6-15 20:13:13
楼主报一下版本
回复

使用道具 举报

Joker 发表于 2016-6-16 09:35:46
最好把hql发出来看下
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条