pip install AttributeError:_DistInfoDistribution__dep_map

Question

我正在尝试在运行Conda的Ubuntu 16.04中使用一些python库(支持GPU的tensorflow,opencv和gdal)及其各种依赖项来启动一个nvidia-docker(2.0)容器.

问题的一般解释

我需要在该环境中使用pip安装一些库(例如tensorflow-gpu 1.10.0和其他一些自定义库),但是无论何时我尝试使用pip在我的Dockerfile中安装包,或者之后,我得到以下错误:

Exception:
Traceback (most recent call last):
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
    return self.__dep_map
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
    raise exc
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
    requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
    status = self.run(options, args)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
    self._warn_about_conflicts(to_install)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
    package_set, _dep_info = check_install_conflicts(to_install)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
    package_set = create_package_set_from_installed()
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
    dm = self._dep_map
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
    reqs.extend(parse_requirements(req))
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
    yield Requirement(line)
  File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
    raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"

如果我尝试使用pip安装其他软件包而不是conda install(其他我尝试过的是setuptools,h5py,以及其他几个我不记得了),也会出现同样的错误

Dockerfile

这是用于生成此的Dockerfile,直到抛出错误的位置(安装tensorflow时):

FROM nvidia/cuda:9.0-devel-ubuntu16.04
LABEL maintainer "[deleted]"

# get correct version of CUDNN for my system's CUDA
ENV CUDNN_VERSION 7.3.0.29
LABEL com.nvidia.cudnn.version="${CUDNN_VERSION}"

RUN apt-get update && apt-get install -y --no-install-recommends \
            libcudnn7=$CUDNN_VERSION-1+cuda9.0 \
            libcudnn7-dev=$CUDNN_VERSION-1+cuda9.0 && \
    apt-mark hold libcudnn7 && \
    rm -rf /var/lib/apt/lists/*

# install underlying requirements
RUN apt-get update \
  && apt-get install -y --no-install-recommends \
    bc \
    bzip2 \
    ca-certificates \
    curl \
    git \
    libgl1 \
    jq \
    nfs-common \
    parallel \
    python-pip \
    python-wheel \
    python-setuptools \
    unzip \
    wget \
    build-essential \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

# install anaconda
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh -O ~/miniconda.sh && \
    /bin/bash ~/miniconda.sh -b -p /opt/conda && \
    rm ~/miniconda.sh && \
    /opt/conda/bin/conda clean -tipsy && \
    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
    echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
    echo "conda activate base" >> ~/.bashrc

ENV TINI_VERSION v0.16.1
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini
RUN chmod +x /usr/bin/tini

# add conda to $PATH and create a conda environment
ENV PATH /opt/conda/bin:$PATH
RUN conda update conda && \
    conda config --remove channels defaults && \
    conda config --add channels conda-forge && \
    conda create -n tf_keras python=3.6 \
                    && echo "source activate tf_keras" > ~/.bashrc
ENV PATH /opt/conda/envs/tf_keras/bin:$PATH

SHELL ["/bin/bash", "-c"]

# install required libraries (and some dependencies)
RUN conda install -n tf_keras \
              osmnx=0.7.3 \
              affine \
              pyproj \
              pyhamcrest=1.9.0 \
              cython \
              fiona \
              h5py \
              ncurses \
              jupyter \
              jupyterlab \
              ipykernel \
              libgdal \
              matplotlib \
              numpy \
              opencv \
              pandas \
              pillow \
              pip \
              scipy \
              scikit-image \
              scikit-learn \
              shapely \
              gdal \
              rtree \
              tqdm \
              pandas \
              geopandas \
              rasterio

# get tensorflow
ARG TENSORFLOW_VERSION=1.10.0
ARG TENSORFLOW_DEVICE=gpu
ARG TENSORFLOW_APPEND=_gpu
RUN source activate tf_keras && \
    pip --no-cache-dir install https://storage.googleapis.com/tensorflow/linux/${TENSORFLOW_DEVICE}/tensorflow${TENSORFLOW_APPEND}-${TENSORFLOW_VERSION}-cp36-cp36m-linux_x86_64.whl

最后一个RUN命令抛出错误.

我试过的事情:

• 不同版本的点子(10.0.1和18.0)
• pip upgrade pip或conda upgrade pip(尝试以这种方式更新pip会抛出相同的错误)
• 更新setuptools(如果我尝试用pip执行它会抛出相同的错误)
• 我已经检查过以确保echo $(which pip) &&在我的pip install命令之前预先添加正确的pip - 它返回虚拟环境的pip.

我只是使用conda安装所有内容并忽略它,但我的组内部有几个库不适用于conda.

Answer 1

你应该降级你的版本testpath:

conda install 'testpath<0.4'

请参阅此处的问题:https://github.com/conda-forge/testpath-feedstock/issues/7