numpy.ufunc的大小错误,请尝试重新编译.即使是最新的熊猫和numpy版本

Question

我在容器中使用pandas,我收到以下错误:

Traceback (most recent call last):
  File "/volumes/dependencies/site-packages/celery/app/trace.py", line 374, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/volumes/dependencies/site-packages/celery/app/trace.py", line 629, in __protected_call__
    return self.run(*args, **kwargs)
  File "/volumes/code/autoai/celery/data_template/api.py", line 16, in run_data_template_task
    data_template.run(data_bundle, columns=columns)
  File "/volumes/code/autoai/models/data_template.py", line 504, in run
    self.to_parquet(data_bundle, columns=columns)
  File "/volumes/code/autoai/models/data_template.py", line 162, in to_parquet
    }, parquet_path=data_file.path, directory="", dataset=self)
  File "/volumes/code/autoai/core/datasets/parquet_converter.py", line 46, in convert
    file_system.write_dataframe(parquet_path, chunk, directory, append=append)
  File "/volumes/code/autoai/core/file_systems.py", line 76, in write_dataframe
    append=append)
  File "/volumes/dependencies/site-packages/pandas/core/frame.py", line 1945, in to_parquet
    compression=compression, **kwargs)
  File "/volumes/dependencies/site-packages/pandas/io/parquet.py", line 256, in to_parquet
    impl = get_engine(engine)
  File "/volumes/dependencies/site-packages/pandas/io/parquet.py", line 40, in get_engine
    return FastParquetImpl()
  File "/volumes/dependencies/site-packages/pandas/io/parquet.py", line 180, in __init__
    import fastparquet
  File "/volumes/dependencies/site-packages/fastparquet/__init__.py", line 8, in <module>
    from .core import read_thrift
  File "/volumes/dependencies/site-packages/fastparquet/core.py", line 13, in <module>
    from . import encoding
  File "/volumes/dependencies/site-packages/fastparquet/encoding.py", line 11, in <module>
    from .speedups import unpack_byte_array
  File "__init__.pxd", line 861, in init fastparquet.speedups
ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216

我在其他答案中读到,当pandas针对比你安装的版本更新的numpy版本进行编译时,会显示此消息.但更新大熊猫和numpy并不适合我.我试图找出我是否有几个版本的numpy,但pip show numpy似乎显示最新版本.

此外,以一种奇怪的方式,这只发生在我本地部署而不是在服务器上.

任何想法如何解决这个问题？或至少如何调试我的numpy和pandas版本(如果有多个版本我该如何检查)

我尝试过:升级两个软件包并删除并重新安装它们.没有帮助.

Answer 1

TLDR：如果docker添加：

RUN pip install numpy

在安装pandas 之前（可能只是pip install -r requirements.txt），它将再次起作用。

我在高山上的docker build pandas上执行此操作，并遇到了同样的问题，它只是弹出（2018年12月27日），用于以前可以正常运行的构建。

Answer 2

答案是因为某种原因，fastparquet（熊猫使用的软件包）正在使用numpy较旧的二进制文件。

更新该软件包很有帮助。我想如果有人解决此问题，尝试更新所有相关的软件包（使用numpy）将是正确的方法

Answer 3

好吧，实际上我的问题以某种方式解决了

 pip uninstall numpy
 pip install numpy

真正的过程是

?  ~ pip3 uninstall numpy -y
Uninstalling numpy-1.14.5:
  Successfully uninstalled numpy-1.14.5
?  ~ pip3 install numpy     
Requirement already satisfied: numpy in /usr/lib/python3/dist-packages (1.16.1)
?  ~ pip3 uninstall numpy   
Uninstalling numpy-1.16.1:
  Would remove:
    /usr/bin/f2py3
    /usr/bin/f2py3.7
    /usr/lib/python3/dist-packages/numpy
    /usr/lib/python3/dist-packages/numpy-1.16.1.egg-info
Proceed (y/n)? y
  Successfully uninstalled numpy-1.16.1
?  ~ pip3 install numpy   
Collecting numpy...

这意味着问题可能是版本冲突？