在 Windows 上使用 MKL 编译 NumPy - DLL 加载失败

Question

我一直在尝试使用 MSVC 编译器和英特尔 MKL 从 Windows 10 上的源代码编译 NumPy。我正在使用 Microsoft Visual Studio 2019 (16.8.4) 和 Intel MKL 2017.8.275 运行 Windows 10.0.18363。

我设法使用最新的 Python 和 NumPy 以最少的设置重现了这个问题。

1. 下载最新的 Python (3.9.1) 和最新的 NumPy (1.20.1) 源。

2. 打开 VS 命令提示符，解压 Python 源代码，构建 PCbuild\build.bat

3. 运行mklvars.bat intel64以获得正确的环境变量集。

4. 将英特尔编译器（ifort 所需）添加到 PATH：

   set PATH=C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64;%PATH%
   

5. 创建一个虚拟环境，从 Python 构建中复制一些文件并激活虚拟环境：

   copy Python\PCbuild\amd64\python39.dll venv\Scripts
   copy Python\PC\pyconfig.h venv\Include
   

6. 从源代码构建 NumPy 并安装： pip install . -v

7. 尝试导入 NumPy： python -c "import numpy"

错误信息

Traceback (most recent call last):
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 22, in <module>
    from . import multiarray
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\multiarray.py", line 12, in <module>
    from . import overrides
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\__init__.py", line 145, in <module>
    from . import core
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "C:\cygwin\home\user\numpy_clean_env\venv\Scripts\python.exe"
  * The NumPy version is: "1.20.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.

其他信息

MKL 库在编译期间被选取，因为它返回：

FOUND:
        libraries = ['mkl_rt']
        library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib\\intel64']
        define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
        include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\include', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib']

我尝试使用 Dependencies（Dependency Walker 的较新版本）分析 DLL 解析，但似乎 MKL DLL 加载正常。有一些 DLL 显示为未正确加载，但据我所知，这是由 Windows API 集的检查软件限制引起的，而不是由该 DLL 的实际问题引起的，所以我认为系统设置正确。以下是NOT_FOUND在分析时导致依赖关系的唯一 DLL 列表_multiarray_umath.pyd：

EMCLIENT.dll (NOT_FOUND) :
HvsiFileTrust.dll (NOT_FOUND) :
UpdateAPI.dll (NOT_FOUND) :
api-ms-win-core-comm-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-ansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-stringansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-winrt-string-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-0.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-1.dll (NOT_FOUND) :
api-ms-win-coreui-secruntime-l1-1-0.dll (NOT_FOUND) :
ext-ms-mf-pal-l2-1-0.dll (NOT_FOUND) :
ext-ms-onecore-appmodel-emclient-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-defaultdiscovery-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-orientation-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-shellchromeapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-deployment-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-usercontext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-viewscalefactor-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-audiocore-pal-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-com-suspendresiliency-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-core-winrt-remote-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dwmapi-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dxcore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-networking-wlanstorage-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-2.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-3.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-4.dll (NOT_FOUND) :
ext-ms-win-ras-tapi32-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ro-typeresolution-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-input-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-private-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-capauthz-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-chambers-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-knownfolderext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-shell32-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-shell-shlwapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-tabbedtitlebar-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellcom-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellfolders-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ui-viewmanagement-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-uiacore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-wer-xbox-l1-1-0.dll (NOT_FOUND) :
ext-ms-win32-subsystem-query-l1-1-0.dll (NOT_FOUND) :
ext-ms-windowscore-deviceinfo-l1-1-0.dll (NOT_FOUND) :

我也尝试遵循本指南，我可以确认os.path.exists('_multiarray_umath.pyd')返回True，但ctypes.WinDLL('_multiarray_umath.pyd')无法加载 DLL（或其依赖项之一）。

工作示例

与上述步骤相同但不执行mklvars.bat，因此不链接到英特尔 MKL。

过去，此设置用于使用 Python 3.6、VS2015 和类似版本的英特尔 MKL。我可能遗漏了一些东西，任何建议都非常受欢迎。

Answer 1

通过遵循 Numpy 邮件列表（永久链接）上的建议解决。

Python >= 3.8 将不再使用 PATH 来解决扩展模块的依赖关系。在导入 numpy 之前在所有脚本中使用 os.add_dll_directory(mkl_bin_path) https://docs.python.org/3/library/os.html#os.add_dll_directory或添加对 numpy 包目录中的 _distributor_init.py 文件的调用。