访问正在写入的文件

Question

You use the hadoop fs –put command to write a 300 MB file using and HDFS block size of 64 MB. Just after this command has finished writing 200 MB of this file, what would another user see when trying to access this file?

a.) They would see Hadoop throw an ConcurrentFileAccessException when they try to access this file.
b.) They would see the current state of the file, up to the last bit written by the command.
c.) They would see the current of the file through the last completed block.
d.) They would see no content until the whole file written and closed.

根据我对hadoop fs -put命令的理解,答案是D,但有人说它是C.

任何人都可以为这两个选项提供建设性的解释吗？

谢谢xx

Answer 1

创建文件后,文件系统命名空间中就会显示该文件.但是,不保证写入文件的任何内容都是可见的:

一旦编写了超过块的数据,第一个块将对新读者可见.对于后续块也是如此:它始终是正在写入的当前块,其他读取器不可见.(来自Hadoop权威指南,Coherency模型).

那么,我会选择C.

另外,看看这个相关的问题.

Answer 2

在写入和关闭整个文件之前无法访问该文件的原因(选项D)是因为,为了访问文件,请求首先被发送到NameNode,以获取与不同块有关的元数据撰写文件.只有在收到确认文件的所有块都已成功写入的确认后,才会由NameNode写入此元数据.

因此,即使块可用,用户也无法在更新元数据之前看到文件,这是在写入所有块之后完成的.