小编JK *_*K R的帖子

我们的Hadoop集群中存在一个奇怪的问题.我们注意到我们的一些作业因文件未找到异常而失败[见下文].基本上,当任务仍在主机上运行时,"attempt_*"目录中的文件和目录本身将被删除.查看一些hadoop文档,我发现当它获得KillJobAction时,作业目录会被删除但是我不确定为什么它会在作业仍在运行时被消除.

我的问题是在作业运行时可能会删除它？任何关于如何调试它的想法或指示都会有所帮助.

谢谢!

java.io.FileNotFoundException: <dir>/hadoop/mapred/local_data/taskTracker/<user>/jobcache/job_201211030344_15383/attempt_201211030344_15383_m_000169_0/output/spill29.out (Permission denied)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.java:120)
    at org.apache.hadoop.fs.RawLocalFileSystem$TrackingFileInputStream.<init>(RawLocalFileSystem.java:71)
    at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.<init>(RawLocalFileSystem.java:107)
    at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:177)
    at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
    at org.apache.hadoop.mapred.Merger$Segment.init(Merger.java:205)
    at org.apache.hadoop.mapred.Merger$Segment.access$100(Merger.java:165)
    at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:418)
    at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:381)
    at org.apache.hadoop.mapred.Merger.merge(Merger.java:77)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1692)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1322)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:698)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:765)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
    at org.apache.hadoop.mapred.Child.main(Child.java:253)