为什么分叉我的进程会导致文件被无限读取

Question

我把我的整个程序归结为一个复制问题的简短主题,所以原谅我没有任何意义.

input.txt是一个文本文件,其中包含几行文本.这个简化的程序应该打印这些行.但是,如果调用fork,程序将进入无限循环,在此循环中反复打印文件的内容.

据我所知,我在这个片段中使用它的方式本质上是一个无操作.它分叉,父母在继续之前等待孩子,孩子立即被杀死.

#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

enum { MAX = 100 };

int main(){
    freopen("input.txt", "r", stdin);
    char s[MAX];

    int i = 0;
    char* ret = fgets(s, MAX, stdin);
    while (ret != NULL) {
        //Commenting out this region fixes the issue
        int status;
        pid_t pid = fork();
        if (pid == 0) {
            exit(0);
        } else {
            waitpid(pid, &status, 0);
        }
        //End region
        printf("%s", s);
        ret = fgets(s, MAX, stdin);
    }
}

编辑:进一步的调查只会使我的问题变得更加奇怪.如果文件包含<4个空白行或<3行文本,则不会中断.但是,如果不止于此,它会无限循环.

Edit2:如果文件包含3行数字,它将无限循环,但如果它包含3行单词则不会.

Answer 1

I am surprised that there is a problem, but it does seem to be a problem on Linux (I tested on Ubuntu 16.04 LTS running in a VMWare Fusion VM on my Mac) — but it was not a problem on my Mac running macOS 10.13.4 (High Sierra), and I wouldn't expect it to be a problem on other variants of Unix either.

As I noted in a comment:

There's an open file description and an open file descriptor behind each stream. When the process forks, the child has its own set of open file descriptors (and file streams), but each file descriptor in the child shares the open file description with the parent. IF (and that's a big 'if') the child process closing the file descriptors first did the equivalent of lseek(fd, 0, SEEK_SET), then that would also position the file descriptor for the parent process, and that could lead to an infinite loop. However, I've never heard of a library that does that seek; there's no reason to do it.

See POSIX open() and fork() for more information about open file descriptors and open file descriptions.

打开的文件描述符对进程是私有的; 打开的文件描述由初始"打开文件"操作创建的文件描述符的所有副本共享.打开文件描述的关键属性之一是当前搜索位置.这意味着子进程可以更改父进程的当前搜索位置 - 因为它位于共享打开文件描述中.

neof97.c

我使用了以下代码 - 原始版本的温和版本,可以使用严格的编译选项进行干净编译:

#include "posixver.h"
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

enum { MAX = 100 };

int main(void)
{
    if (freopen("input.txt", "r", stdin) == 0)
        return 1;
    char s[MAX];
    for (int i = 0; i < 30 && fgets(s, MAX, stdin) != NULL; i++)
    {
        // Commenting out this region fixes the issue
        int status;
        pid_t pid = fork();
        if (pid == 0)
        {
            exit(0);
        }
        else
        {
            waitpid(pid, &status, 0);
        }
        // End region
        printf("%s", s);
    }
    return 0;
}

其中一个修改将周期数(子项)限制为30个.我使用了4行20个随机字母和一个换行符(总共84个字节)的数据文件:

ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe

我strace在Ubuntu上运行命令:

$ strace -ff -o st-out -- neof97
ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe
…
uGnxGhSluywhlAEBIXNP
plRXLszVvPgZhAdTLlYe
ywYaGKiRtAwzaBbuzvNb
eRsjPoBaIdxZZtJWfSty
$

有31个文件的名称,st-out.808##其中哈希是2位数字.主进程文件非常大; 其他的很小,其中一个尺寸为66,110,111或137:

$ cat st-out.80833
lseek(0, -63, SEEK_CUR)                 = 21
exit_group(0)                           = ?
+++ exited with 0 +++
$ cat st-out.80834
lseek(0, -42, SEEK_CUR)                 = -1 EINVAL (Invalid argument)
exit_group(0)                           = ?
+++ exited with 0 +++
$ cat st-out.80835
lseek(0, -21, SEEK_CUR)                 = 0
exit_group(0)                           = ?
+++ exited with 0 +++
$ cat st-out.80836
exit_group(0)                           = ?
+++ exited with 0 +++
$

事实恰恰相反,前4个孩子各自表现出四种行为中的一种 - 每个进一步的4个孩子表现出相同的模式.

这表明,四个孩子中有三个确实lseek()在退出前做了标准输入.显然,我现在已经看到一个库做了.我不知道为什么它被认为是一个好主意,但从经验上来说,这就是正在发生的事情.

neof67.c

这个版本的代码,使用单独的文件流(和文件描述符),fopen()而不是freopen()遇到问题.

#include "posixver.h"
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

enum { MAX = 100 };

int main(void)
{
    FILE *fp = fopen("input.txt", "r");
    if (fp == 0)
        return 1;
    char s[MAX];
    for (int i = 0; i < 30 && fgets(s, MAX, fp) != NULL; i++)
    {
        // Commenting out this region fixes the issue
        int status;
        pid_t pid = fork();
        if (pid == 0)
        {
            exit(0);
        }
        else
        {
            waitpid(pid, &status, 0);
        }
        // End region
        printf("%s", s);
    }
    return 0;
}

这也表现出相同的行为,除了发生搜索的文件描述符3而不是0.因此,我的两个假设被证明是错误的 - 它与freopen()和stdin; 第二个测试代码都显示错误.

初步诊断

IMO,这是一个错误.你不应该遇到这个问题.它很可能是Linux(GNU C)库中的错误而不是内核.它是由lseek()子进程中引起的.目前尚不清楚(因为我没有去看源代码)库正在做什么或为什么.

GLIBC Bug 23151

GLIBC 错误23151 - 具有未关闭文件的分叉进程在退出之前执行lseek并且可能导致父I/O中的无限循环.

该错误创建于2019-05-08美国/太平洋地区,并于2018-05-09关闭为无效.给出的理由是:

请阅读 http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01,尤其是以下段落:

请注意,在a之后fork(),存在两个句柄,其中之前存在一个句柄.[...]

POSIX

POSIX的完整部分(除了注意到C标准未涵盖的措辞之外)是:

2.5.1文件描述符和标准I/O流的交互

An open file description may be accessed through a file descriptor, which is created using functions such as open() or pipe(), or through a stream, which is created using functions such as fopen() or popen(). Either a file descriptor or a stream is called a "handle" on the open file description to which it refers; an open file description may have several handles.

Handles can be created or destroyed by explicit user action, without affecting the underlying open file description. Some of the ways to create them include fcntl(), dup(), fdopen(), fileno(), and fork(). They can be destroyed by at least fclose(), close(), and the exec functions.

A file descriptor that is never used in an operation that could affect the file offset (for example, read(), write(), or lseek()) is not considered a handle for this discussion, but could give rise to one (for example, as a consequence of fdopen(), dup(), or fork()). This exception does not include the file descriptor underlying a stream, whether created with fopen() or fdopen(), so long as it is not used directly by the application to affect the file offset. The read() and write() functions implicitly affect the file offset; lseek() explicitly affects it.

The result of function calls involving any one handle (the "active handle") is defined elsewhere in this volume of POSIX.1-2017, but if two or more handles are used, and any one of them is a stream, the application shall ensure that their actions are coordinated as described below. If this is not done, the result is undefined.

A handle which is a stream is considered to be closed when either an fclose(), or freopen() with non-full⁽¹⁾ filename, is executed on it (for freopen() with a null filename, it is implementation-defined whether a new handle is created or the existing one reused), or when the process owning that stream terminates with exit(), abort(), or due to a signal. A file descriptor is closed by close(), _exit(), or the exec() functions when FD_CLOEXEC is set on that file descriptor.

⁽¹⁾ [sic] Using 'non-full' is probably a typo for 'non-null'.

For a handle to become the active handle, the application shall ensure that the actions below are performed between the last use of the handle (the current active handle) and the first use of the second handle (the future active handle). The second handle then becomes the active handle. All activity by the application affecting the file offset on the first handle shall be suspended until it again becomes the active file handle. (If a stream function has as an underlying function one that affects the file offset, the stream function shall be considered to affect the file offset.)

The handles need not be in the same process for these rules to apply.

Note that after a fork(), two handles exist where one existed before. The application shall ensure that, if both handles can ever be accessed, they are both in a state where the other could become the active handle first. The application shall prepare for a fork() exactly as if it were a change of active handle. (If the only action performed by one of the processes is one of the exec() functions or _exit() (not exit()), the handle is never accessed in that process.)

For the first handle, the first applicable condition below applies. After the actions required below are taken, if the handle is still open, the application can close it.

• If it is a file descriptor, no action is required.

• If the only further action to be performed on any handle to this open file descriptor is to close it, no action need be taken.

• If it is a stream which is unbuffered, no action need be taken.

• If it is a stream which is line buffered, and the last byte written to the stream was a <newline> (that is, as if a:

  putc('\n')
  

  was the most recent operation on that stream), no action need be taken.

• If it is a stream which is open for writing or appending (but not also open for reading), the application shall either perform an fflush(), or the stream shall be closed.

• If the stream is open for reading and it is at the end of the file (feof() is true), no action need be taken.

• If the stream is open with a mode that allows reading and the underlying open file description refers to a device that is capable of seeking, the application shall either perform an fflush(), or the stream shall be closed.

For the second handle:

• If any previous active handle has been used by a function that explicitly changed the file offset, except as required above for the first handle, the application shall perform an lseek() or fseek() (as appropriate to the type of handle) to an appropriate location.

If the active handle ceases to be accessible before the requirements on the first handle, above, have been met, the state of the open file description becomes undefined. This might occur during functions such as a fork() or _exit().

The exec() functions make inaccessible all streams that are open at the time they are called, independent of which streams or file descriptors may be available to the new process image.

When these rules are followed, regardless of the sequence of handles used, implementations shall ensure that an application, even one consisting of several processes, shall yield correct results: no data shall be lost or duplicated when writing, and all data shall be written in order, except as requested by seeks. It is implementation-defined whether, and under what conditions, all input is seen exactly once.

Each function that operates on a stream is said to have zero or more "underlying functions". This means that the stream function shares certain traits with the underlying functions, but does not require that there be any relation between the implementations of the stream function and its underlying functions.

Exegesis

That is hard reading! If you're not clear on the distinction between open file descriptor and open file description, read the specification of open() and fork() (and dup() or dup2()). The definitions for file descriptor and open file description are also relevant, if terse.

In the context of the code in this question (and also for Unwanted child processes being created while file reading), we have a file stream handle open for reading only which has not yet encountered EOF (so feof() would not return true, even though the read position is at the end of the file).

One of the crucial parts of the specification is: The application shall prepare for a fork() exactly as if it were a change of active handle.

This means that the steps outlined for 'first file handle' are relevant, and stepping through them, the first applicable condition is the last:

• If the stream is open with a mode that allows reading and the underlying open file description refers to a device that is capable of seeking, the application shall either perform an fflush(), or the stream shall be closed.

If you look at the definition for fflush(), you find:

If stream points to an output stream or an update stream in which the most recent operation was not input, fflush() shall cause any unwritten data for that stream to be written to the file, [CX] ? and the last data modification and last file status change timestamps of the underlying file shall be marked for update.

For a stream open for reading with an underlying file description, if the file is not already at EOF, and the file is one capable of seeking, the file offset of the underlying open file description shall be set to the file position of the stream, and any characters pushed back onto the stream by ungetc() or ungetwc() that have not subsequently been read from the stream shall be discarded (without further changing the file offset). ?

It isn't exactly clear what happens if you apply fflush() to an input stream associated with a non-seekable file, but that isn't our immediate concern. However, if you're writing generic library code, then you might need to know whether the underlying file descriptor is seekable before doing a fflush() on the stream. Alternatively, use fflush(NULL) to have the system do whatever is necessary for all I/O streams, noting that this will lose any pushed-back characters (via ungetc() etc).

The lseek() operations shown in the strace output seem to be implementing the fflush() semantics associating the file offset of the open file description with the file position of the stream.

So, for the code in this question, it seems that fflush(stdin) is necessary before the fork() to ensure consistency. Not doing that leads to undefined behaviour ('if this is not done, the result is undefined') — such as looping indefinitely.