reduce时出现java.io.IOException: Filesystem closed
今天在hadoop0.20上跑reduce的时候遇到Filesystem closed异常。reduce主要做的工作是，从iterator中读取数据，并收集到一个arrayList对象中，当数据条数达到较大值时（一个阀值），将arrayList对象以文件方式存到hdfs。在reduce完成对所有iterator条目的遍历之后，从hdfs中再读取数据，存入变量中。只在存入的文件比较大，文件数目比较多（400MB，大概50个文件）的时候出现，当文件规模比较小（100MB，10个）时，运行正常。哪位高人能告诉下小弟，这个Filesystem closed异常一般在什么情况下发生吗？不胜感激。以下是异常发生时的调用栈：
java.io.IOException: Filesystem closed at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:232)
at org.apache.hadoop.hdfs.DFSClient.access$600(DFSClient.java:70)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1920)
at java.io.DataInputStream.read(DataInputStream.java:132)
at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2266)
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2279)
at java.io.ObjectInputStream$BlockDataInputStream.readInt(ObjectInputStream.java:2774)
at java.io.ObjectInputStream.readHandle(ObjectInputStream.java:1431)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1490)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
... 同样的，求哪个高人解答一下吗？吗？吗？吗？
页:
[1]