我使用的cdh 版本是5.3.0 ,hive 的版本是0.13.1 .在执行hive任务的时候,时不时出现以下异常.重新跑,又不会,并不是 每次都会出现,但基本上每天都出现,我分析了,没找出具体原因,求大神指点.
-----------------------------------------------------------
Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: org.apache.hive.com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 109
Serialization trace:
parentOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:366)
at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:277)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:258)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:451)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:444)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:588)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 109
Serialization trace:
parentOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:656)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:767)
at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:131)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)
at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:943)
at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:851)
at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:865)
at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:335)
... 13 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-15: Map: 2 Reduce: 1 Cumulative CPU: 13.47 sec HDFS Read: 101706433 HDFS Write: 8804 SUCCESS
Stage-Stage-11: Map: 2 Reduce: 1 Cumulative CPU: 14.2 sec HDFS Read: 101706433 HDFS Write: 345 SUCCESS
Stage-Stage-1: Map: 2 Reduce: 1 Cumulative CPU: 17.88 sec HDFS Read: 101706433 HDFS Write: 40943 SUCCESS
Stage-Stage-16: Map: 1 Reduce: 1 Cumulative CPU: 2.51 sec HDFS Read: 9164 HDFS Write: 4883 SUCCESS
Stage-Stage-12: Map: 1 Reduce: 1 Cumulative CPU: 2.53 sec HDFS Read: 705 HDFS Write: 301 SUCCESS
Stage-Stage-2: Map: 1 Reduce: 1 Cumulative CPU: 2.96 sec HDFS Read: 41303 HDFS Write: 18015 SUCCESS
Stage-Stage-17: Map: 1 Reduce: 1 Cumulative CPU: 3.25 sec HDFS Read: 5243 HDFS Write: 4883 SUCCESS
Stage-Stage-13: Map: 1 Reduce: 1 Cumulative CPU: 2.62 sec HDFS Read: 661 HDFS Write: 301 SUCCESS
Stage-Stage-3: Map: 1 Reduce: 1 Cumulative CPU: 2.8 sec HDFS Read: 18375 HDFS Write: 18015 SUCCESS
Stage-Stage-18: Map: 1 Reduce: 1 Cumulative CPU: 2.52 sec HDFS Read: 5243 HDFS Write: 4883 SUCCESS
Stage-Stage-14: Map: 1 Reduce: 1 Cumulative CPU: 2.17 sec HDFS Read: 661 HDFS Write: 301 SUCCESS
Stage-Stage-4: Map: 1 Reduce: 1 Cumulative CPU: 2.35 sec HDFS Read: 18375 HDFS Write: 18015 SUCCESS
Stage-Stage-21: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 1 minutes 9 seconds 260 msec
|