"警告conda.gateways.disk:exp_backoff_fn(47):在"conda安装"期间使用errno 41"未捕获退避"

Question

从今天开始,我得到了很多

警告conda.gateways.disk:exp_backoff_fn(47):使用errno 41进行未检出的退避

我尝试使用conda install或更新或安装软件包时发出警告conda update.例如:

(...) C:\Users\...> conda install numba
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment C:\...:

The following packages will be DOWNGRADED due to dependency conflicts:

    numba: 0.30.0-np111py35_0 --> 0.30.1-np111py35_0

Proceed ([y]/n)? y

numba-0.30.0-np111p35_0 100% |###############################| Time: 0:00:00   2.50 MB/s
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41
WARNING conda.gateways.disk:exp_backoff_fn(47): Uncaught backoff with errno 41

之后安装了软件包,但所有这些警告似乎表明某些内容无法正常运行.

OS: Windows 10 64 bit
conda: 4.3.4

你能说出我需要做些什么来修复这些警告吗？或者我可以忽略它们吗？

Answer 1

根据当前的conda源树,您看到的这些警告是"正常的".要理解上述警告的起源,让我们看看问题中的源代码和conda存储库中的最近提交(https://github.com/conda/conda).打印您看到的警告的相关源代码如下:

https://github.com/conda/conda/blob/4.3.4/conda/gateways/disk/ 初始化的.py

# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function, unicode_literals

import sys
from errno import EACCES, ENOENT, EPERM, EPROTOTYPE
from logging import getLogger
from os.path import basename
from time import sleep

from ...common.compat import on_win

log = getLogger(__name__)

MAX_TRIES = 7


def exp_backoff_fn(fn, *args, **kwargs):
    """Mostly for retrying file operations that fail on Windows due to virus scanners"""
    max_tries = kwargs.pop('max_tries', MAX_TRIES)
    if not on_win:
        return fn(*args, **kwargs)

    import random
    # with max_tries = 6, max total time ~= 3.2 sec
    # with max_tries = 7, max total time ~= 6.5 sec
    for n in range(max_tries):
        try:
            result = fn(*args, **kwargs)
        except (OSError, IOError) as e:
            log.trace(repr(e))
            if e.errno in (EPERM, EACCES):
                if n == max_tries-1:
                    raise
                sleep_time = ((2 ** n) + random.random()) * 0.1
                caller_frame = sys._getframe(1)
                log.trace("retrying %s/%s %s() in %g sec",
                          basename(caller_frame.f_code.co_filename),
                          caller_frame.f_lineno,
                          fn.__name__,
                          sleep_time)
                sleep(sleep_time)
            elif e.errno in (ENOENT, EPROTOTYPE):
                # errno.ENOENT File not found error / No such file or directory
                # errno.EPROTOTYPE OSError(41, 'The directory is not empty')
                raise
            else:
                log.warn("Uncaught backoff with errno %d", e.errno)
                raise
        else:
            return result

从上面的源代码看,它似乎是Windows上可能出现的警告

由于病毒扫描程序,重试在Windows上失败的文件操作

使用https://msdn.microsoft.com/en-us/library/t3ayayh1.aspx进入细节,看来errno 41对应于

ENOTEMPTY:目录不为空

表示指定的目录不为空.这是一个未被捕获的错误,因为他们没有处理这种错误的分支(ENOTEMPTY),而例如他们在这种情况下它是另一个错误,如EPERM或EACCES.
在提交https://github.com/conda/conda/commit/fb2a783d9b9371559b5ea82aaf8ae631c2ce0450#diff-3757ed9862260ae3b54768b3e482e3fe
他们明确地删除报告EPROTOTYPE为OSError(41, 'The directory is not empty'),所以现在你看到的报道作为一个警告,错误号

log.warn("Uncaught backoff with errno %d", e.errno)

他们修改的另一部分是在函数中的https://github.com/conda/conda/blob/fb2a783d9b9371559b5ea82aaf8ae631c2ce0450/conda/gateways/disk/delete.py,delete_trash()所以现在,如果要启用信息日志,最有可能你会看到像这样的一条线

"无法完全清除垃圾邮件目录%s \n剩余%d个文件."

启用

log.info("Unable to fully clean trash directory %s\nThere are %d remaining file(s).",
                 trash_dir, len(files_remaining))  

现在,delete_trash()您引用(安装,更新)的命令调用:
https://github.com/conda/conda/blob/f4b386476307e3979679957292d4f6e4c581df03/conda/cli/main_install.py
https://github.com/conda/康达/ BLOB/a26b1eff17dcaf12f03aea5bbe8dee1e01308de7 /畅达/ CLI/main_update.py

可以看出,delete_trash()分别在前面提到的"安装"和"更新"文件中由以下代码片段触发:

from ..gateways.disk.delete import delete_trash

# some other code ...

def execute(args, parser):
    install(args, parser, 'install')
    delete_trash()

和

from ..gateways.disk.delete import delete_trash

# some other code ...

def execute(args, parser):
    install(args, parser, 'update')
    delete_trash()  

delete_trash()然后将触发该代码路径,backoff_rmdir()或者backoff_unlink()最终将导致您从exp_backoff_fn()前面看到的警告.
总而言之,主要的呼叫链是

update or install --> delete_trash() --> backoff_rmdir() or backoff_unlink() --> exp_backoff_fn() --> your warning message

根据在存储库中完成的源代码修改,开发人员认为这些警告可以安全地忽略,因为这些警告发生在更新或安装命令的"清理"阶段,即在更新或安装操作成功之后执行.
我不能说100%你可以安全地忽略这些警告.如果几次尝试后删除垃圾目录的命令成功,那么没有问题.但如果它没有成功,那么你将遇到这个目录由于不删除它而变得越来越大的问题.在回购中打开的问题很少,我不知道修补程序是否涵盖了你所遇到的代码路径.我的印象可能不是.要获得进一步的见解,您可以激活信息日志级别.

更新:这个问题https://github.com/conda/conda/issues/4164确切地提到了你报告的警告,因为由于所有的重试,人们得到了很长的更新和安装时间.正如我所提到的那样,在所有重试(指数退避)之后,删除操作可能成功或失败,那个人在他的报告中也提到了这个方面.
正如你在这里看到的那样
https://github.com/conda/conda/issues/3664
,由于重试,人们用来解决漫长等待时间的问题很少,并且还会使你的下一次命令运行conda install X或conda update X没有警告.这些建议是:

1. (以加快重试/删除的时间)MAX_TRIES = 1在conda/gateways/disk/init .py的副本中设置
2. 在运行下一个conda install X或之前删除.trash目录conda update X.有关某些人使用简单脚本自动删除该目录的解决方法,请参阅https://github.com/conda/conda/issues/3664.通常这应该是安全的.

因此,您的问题的答案将是:
1)您可以使用https://github.com/conda/conda/issues/3664中提到的解决方法,它使用以下powershell脚本(和另一个脚本):

$cir = conda info --root
$trash_dir = "$($cir)\pkgs\.trash"
if (Test-Path $trash_dir){
    Remove-Item -Recurse -Force $trash_dir
}
conda --debug update --all --yes --quiet 

基本上清理.trash目录;

2)您可以安全地忽略警告,因为它们不会影响功能; 问题是越多的.trash被填充,删除项目的时间和重试次数就越多,这样你就会遇到性能问题; 正如你所提到的,它有一种"泄漏",但它不会影响功能.该目录应该被清空并删除,因为它包含不再需要的垃圾.系统将尝试删除它,但可能无法执行此操作.所以使用1).

更新2:正如我的一条评论中提到的,其中一个关键变化是在文件中conda/gateways/disk/__init__.py,这是一个"修复"(https://github.com/conda/conda/commit/6cb3be39aec1c738678ae27b3a264941d08c859a),使其达到4.3. 6版本的conda(conda 4.3.6发布信息)解决了有问题的警告.
没有看到警告的关键是通过一个明确捕获并处理之前解释的错误的分支.现在,当ENOTEMPTY类型的错误发生时(这是在这种情况下触发警告打印的错误),这将被捕获并且不会转到打印问题检查的警告的分支.为了理解主要的差异,在版本4.3.4中它是

elif e.errno in (ENOENT, EPROTOTYPE):                   
     raise
else:
     log.warn("Uncaught backoff with errno %d", e.errno)
     raise

而在版本4.3.6中它变成:

elif e.errno in (ENOENT, ENOTEMPTY):    
    raise
else:
    log.warn("Uncaught backoff with errno %s %d", errorcode[e.errno], e.errno)
    raise

并且你清楚地看到,现在该错误不会进入else分支,因此在这种情况下你不会看到该消息.