立即注册 登录
About云-梭伦科技 返回首页

rsgg03的个人空间 https://aboutyun.com/?6 [收藏] [复制] [分享] [RSS]

日志

java.io.IOException: com.esotericsoftware.kryo.KryoException: Encountered unregi

已有 3264 次阅读2015-4-7 07:11

15/03/09 20:13:38 WARN scheduler.TaskSetManager: Lost task 202.0 in stage 1.0 (TID 217, slave19.cluster03): java.io.IOException: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 13994
Serialization trace:
ord (org.apache.spark.util.BoundedPriorityQueue)
    at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1003)
    at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:164)
    at org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:64)
    at org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:64)
    at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:87)
    at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:61)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
    at org.apache.spark.scheduler.Task.run(Task.scala:56)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 13994
Serialization trace:
ord (org.apache.spark.util.BoundedPriorityQueue)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
    at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:610)
    at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.read(FieldSerializer.java:599)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:221)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:732)
    at org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:142)
    at org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:216)
    at org.apache.spark.broadcast.TorrentBroadcast$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:177)
    at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1000)
    ... 12 more

在spark-default.conf中我将spark.serializer配置成org.apache.spark.serializer.KryoSerializer,但并没有正确的配置,改回org.apache.spark.serializer.JavaSerializer即可,关于如何正确配置成org.apache.spark.serializer.KryoSerializer日后会介绍~

路过

雷人

握手

鲜花

鸡蛋

评论 (0 个评论)

facelist doodle 涂鸦板

您需要登录后才可以评论 登录 | 立即注册

关闭

推荐上一条 /2 下一条