以 root 身份手动运行时 logrotate 成功，但由 logrotate.service 运行时失败并显示“只读文件系统”

Question

我有一个（以前有效的）OpenResty logrotate 设置，它是我从之前的 Ubuntu 18.04 安装中获取的。然而 logrotate.service 现在失败并出现此错误......

\n

error: error renaming /usr/local/openresty/nginx/logs/access.log.60.zst\nto /usr/local/openresty/nginx/logs/access.log.61.zst: Read-only file system\n

\n

...我很难理解为什么。新机器正在运行 Ubuntu 20.04，但我不明白为什么这会在这种情况下产生影响。

\n

首先，这是配置：

\n

$ cat /etc/logrotate.d/custom-openresty\n\n/usr/local/openresty/nginx/logs/access.log\n/usr/local/openresty/nginx/logs/error.log\n{\n  daily\n  rotate 60\n  maxsize 1G\n  missingok\n  notifempty\n  compress\n  compresscmd /usr/bin/zstd\n  uncompresscmd /usr/bin/unzstd\n  compressoptions -9 --long -T1\n  compressext .zst\n  delaycompress\n  sharedscripts\n  postrotate\n    test ! -f /usr/local/openresty/nginx/logs/nginx.pid || kill -USR1 `cat /usr/local/openresty/nginx/logs/nginx.pid`\n endscript\n}\n

\n

/etc/logrotate.conf没有改变，看起来像这样：

\n

\n# see "man logrotate" for details\n# rotate log files weekly\nweekly\n\n# use the adm group by default, since this is the owning group\n# of /var/log/syslog.\nsu root adm\n\n# keep 4 weeks worth of backlogs\nrotate 4\n\n# create new (empty) log files after rotating old ones\ncreate\n\n# use date as a suffix of the rotated file\n#dateext\n\n# uncomment this if you want your log files compressed\n#compress\n\n# packages drop log rotation information into this directory\ninclude /etc/logrotate.d\n\n# system-specific logs may be also be configured here.\n

\n

这是我的文件的状态（我希望access.log.1在 logrotate 运行后得到）：

\n

$ ls -alhg /usr/local/openresty/nginx/logs/\ntotal 10G\ndrwxr-xr-x  2 root 4.0K Sep 16 20:04 .\ndrwxr-xr-x 18 root 4.0K Sep 16 19:42 ..\n-rw-r--r--  1 root  10G Sep 16 19:56 access.log\n-rw-r--r--  1 root 5.5K Sep 16 19:56 error.log\n

\n

但是，logrotate.service 失败并出现以下错误：

\n

~$ systemctl status logrotate.service \n\xe2\x97\x8f logrotate.service - Rotate log files\n     Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset: enabled)\n     Active: failed (Result: exit-code) since Wed 2020-09-16 20:11:32 UTC; 4min 32s ago\nTriggeredBy: \xe2\x97\x8f logrotate.timer\n       Docs: man:logrotate(8)\n             man:logrotate.conf(5)\n    Process: 27403 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)\n   Main PID: 27403 (code=exited, status=1/FAILURE)\n\n$ sudo journalctl --unit logrotate.service\n\nSep 16 20:11:32 fetcher-scheduler systemd[1]: Starting Rotate log files...\nSep 16 20:11:32 fetcher-scheduler logrotate[27403]: error: error renaming /usr/local/openresty/nginx/logs/access.log.60.zst to /usr/local/openresty/nginx/logs/access.log.61.zst: Read-only file system\nSep 16 20:11:32 fetcher-scheduler systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE\nSep 16 20:11:32 fetcher-scheduler systemd[1]: logrotate.service: Failed with result 'exit-code'.\nSep 16 20:11:32 fetcher-scheduler systemd[1]: Failed to start Rotate log files.\n

\n

当我以 root 身份在调试模式下运行它（没有 systemd）时，我得到以下输出：

\n

# logrotate -v -d /etc/logrotate.d/custom-openresty\n\nreading config file /etc/logrotate.d/custom-openresty\ncompress_prog is now /usr/bin/zstd\nuncompress_prog is now /usr/bin/unzstd\ncompress_options is now  -9 --long -T1\ncompress_ext is now .zst\nReading state from file: /var/lib/logrotate/status\nAllocating hash table for state file, size 64 entries\nCreating new state\nCreating new state\n...\nCreating new state\n\nHandling 1 logs\n\nrotating pattern: /usr/local/openresty/nginx/logs/access.log\n/usr/local/openresty/nginx/logs/error.log\n after 1 days (60 rotations)\nempty log files are not rotated, log files >= 1073741824 are rotated earlier, old logs are removed\nconsidering log /usr/local/openresty/nginx/logs/access.log\n  Now: 2020-09-16 20:24\n  Last rotated at 2020-09-16 20:11\n  log needs rotating\nconsidering log /usr/local/openresty/nginx/logs/error.log\n  Now: 2020-09-16 20:24\n  Last rotated at 2020-09-16 19:00\n  log does not need rotating (log has been rotated at 2020-9-16 19:0, that is not day ago yet)\nrotating log /usr/local/openresty/nginx/logs/access.log, log->rotateCount is 60\ndateext suffix '-20200916'\nglob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'\nprevious log /usr/local/openresty/nginx/logs/access.log.1 does not exist\nrenaming /usr/local/openresty/nginx/logs/access.log.60.zst to /usr/local/openresty/nginx/logs/access.log.61.zst (rotatecount 60, logstart 1, i 60), \nrenaming /usr/local/openresty/nginx/logs/access.log.59.zst to /usr/local/openresty/nginx/logs/access.log.60.zst (rotatecount 60, logstart 1, i 59), \n...\n/logs/access.log.3.zst (rotatecount 60, logstart 1, i 2), \nrenaming /usr/local/openresty/nginx/logs/access.log.1.zst to /usr/local/openresty/nginx/logs/access.log.2.zst (rotatecount 60, logstart 1, i 1), \nrenaming /usr/local/openresty/nginx/logs/access.log.0.zst to /usr/local/openresty/nginx/logs/access.log.1.zst (rotatecount 60, logstart 1, i 0), \nlog /usr/local/openresty/nginx/logs/access.log.61.zst doesn't exist -- won't try to dispose of it\nrenaming /usr/local/openresty/nginx/logs/access.log to /usr/local/openresty/nginx/logs/access.log.1\nrunning postrotate script\nrunning script with arg /usr/local/openresty/nginx/logs/access.log\n/usr/local/openresty/nginx/logs/error.log\n: "\n    test ! -f /usr/local/openresty/nginx/logs/nginx.pid || kill -USR1 `cat /usr/local/openresty/nginx/logs/nginx.pid`\n"\n

\n

对我来说，一切看起来都很正常。如果我运行它，它也会起作用：

\n

# logrotate -v /etc/logrotate.d/custom-openresty\nreading config file /etc/logrotate.d/custom-openresty\ncompress_prog is now /usr/bin/zstd\nuncompress_prog is now /usr/bin/unzstd\ncompress_options is now  -9 --long -T1\ncompress_ext is now .zst\nReading state from file: /var/lib/logrotate/status\nAllocating hash table for state file, size 64 entries\nCreating new state\n...\nCreating new state\n\nHandling 1 logs\n\nrotating pattern: /usr/local/openresty/nginx/logs/access.log\n/usr/local/openresty/nginx/logs/error.log\n after 1 days (60 rotations)\nempty log files are not rotated, log files >= 1073741824 are rotated earlier, old logs are removed\nconsidering log /usr/local/openresty/nginx/logs/access.log\n  Now: 2020-09-16 20:26\n  Last rotated at 2020-09-16 20:11\n  log needs rotating\nconsidering log /usr/local/openresty/nginx/logs/error.log\n  Now: 2020-09-16 20:26\n  Last rotated at 2020-09-16 19:00\n  log does not need rotating (log has been rotated at 2020-9-16 19:0, that is not day ago yet)\nrotating log /usr/local/openresty/nginx/logs/access.log, log->rotateCount is 60\ndateext suffix '-20200916'\nglob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'\nprevious log /usr/local/openresty/nginx/logs/access.log.1 does not exist\nrenaming /usr/local/openresty/nginx/logs/access.log.60.zst to /usr/local/openresty/nginx/logs/access.log.61.zst (rotatecount 60, logstart 1, i 60), \nold log /usr/local/openresty/nginx/logs/access.log.60.zst does not exist\n...\nold log /usr/local/openresty/nginx/logs/access.log.2.zst does not exist\nrenaming /usr/local/openresty/nginx/logs/access.log.1.zst to /usr/local/openresty/nginx/logs/access.log.2.zst (rotatecount 60, logstart 1, i 1), \nold log /usr/local/openresty/nginx/logs/access.log.1.zst does not exist\nrenaming /usr/local/openresty/nginx/logs/access.log.0.zst to /usr/local/openresty/nginx/logs/access.log.1.zst (rotatecount 60, logstart 1, i 0), \nold log /usr/local/openresty/nginx/logs/access.log.0.zst does not exist\nlog /usr/local/openresty/nginx/logs/access.log.61.zst doesn't exist -- won't try to dispose of it\nrenaming /usr/local/openresty/nginx/logs/access.log to /usr/local/openresty/nginx/logs/access.log.1\nrunning postrotate script\n

\n

没有错误，最终access.log.1按预期创建：

\n

# ls -algh /usr/local/openresty/nginx/logs/\ntotal 10G\ndrwxr-xr-x  2 root 4.0K Sep 16 20:26 .\ndrwxr-xr-x 18 root 4.0K Sep 16 19:42 ..\n-rw-r--r--  1 root  10G Sep 16 19:56 access.log.1\n-rw-r--r--  1 root 5.5K Sep 16 19:56 error.log\n

\n

请注意，它说...

\n

log /usr/local/openresty/nginx/logs/access.log.61.zst\ndoesn't exist -- won't try to dispose of it\n

\n

...而不是error renaming ... Read-only file system（就像logrotate.service那样）

\n

为什么以 root 身份手动运行时可以工作，但通过 logrotate.service 执行时却失败？

\n

我没有对 进行任何更改logrotate.service。为了完整起见，这里是单元文件：

\n

$ systemctl cat logrotate.service\n# /lib/systemd/system/logrotate.service\n[Unit]\nDescription=Rotate log files\nDocumentation=man:logrotate(8) man:logrotate.conf(5)\nConditionACPower=true\n\n[Service]\nType=oneshot\nExecStart=/usr/sbin/logrotate /etc/logrotate.conf\n\n# performance options\nNice=19\nIOSchedulingClass=best-effort\nIOSchedulingPriority=7\n\n# hardening options\n#  details: https://www.freedesktop.org/software/systemd/man/systemd.exec.html\n#  no ProtectHome for userdir logs\n#  no PrivateNetwork for mail deliviery\n#  no ProtectKernelTunables for working SELinux with systemd older than 235\n#  no MemoryDenyWriteExecute for gzip on i686\nPrivateDevices=true\nPrivateTmp=true\nProtectControlGroups=true\nProtectKernelModules=true\nProtectSystem=full\nRestrictRealtime=true\n

\n

现在我已经没有选择了。非常感谢任何解决问题的帮助。

\n

Answer 1

我想，我找到了。logrotate 文件不是问题。

相反，它是由 systemd 单元文件中的强化功能引起的。在我禁用ProtectSystem=full强化选项后，它起作用了。原因是 logrotate 必须对/usr我的目录进行操作，如果启用该选项，该目录是只读的。

从文档上ProtectSystem=

采用布尔参数或特殊值“full”或“strict”。如果为 true，则将此单元调用的进程以只读方式安装 /usr 和引导加载程序目录（/boot 和 /efi）。如果设置为“full”，/etc 目录也会以只读方式挂载。如果设置为“strict”，则整个文件系统层次结构将以只读方式安装，API 文件系统子树 /dev、/proc 和 /sys 除外（使用 PrivateDevices=、ProtectKernelTunables=、ProtectControlGroups= 保护这些目录）。此设置可确保服务禁止对供应商提供的操作系统（以及可选的配置和本地安装）进行任何修改。建议为所有长时间运行的服务启用此设置，除非它们涉及系统更新或需要以其他方式修改操作系统。如果使用此选项，则可以使用 ReadWritePaths= 将特定目录排除为只读。如果设置了 DynamicUser=，则隐含此设置。此设置无法确保在所有情况下提供保护。一般来说，它具有与 ReadOnlyPaths= 相同的限制，请参见下文。默认为关闭。

为了正确修复它，我现在添加了以下行/lib/systemd/system/logrotate.service：

ReadWritePaths=/usr/local/openresty/nginx/logs

然后ProtectSystem=full将除日志所在目录之外的所有内容安装为只读。现在，只读错误消失了：

# systemctl daemon-reload && systemctl start logrotate
(completed successfully without output)

我想，我从来没有在 Ubuntu 18.04 上遇到过这个问题，因为强化选项还不是默认值。Ubuntu 20.04 现在有了它，但我不知道它们是在哪个 Ubuntu 版本中首次引入的。