在 Glue Python Shell 中使用 geopandas 将 Shapefile 写入 AWS S3

Question

我已通过 geopandas 成功从我的 S3 存储桶中读取了 zip 格式的 shapefile，但在尝试将相同的地理数据帧作为 shapefile 输出到同一 S3 存储桶时出现错误。

下面的代码是我读取 zip 文件的方式，效果很好：

## session for connecting to S3
session = boto3.session.Session(aws_access_key_id='MY-KEY-ID',
                                aws_secret_access_key='MY-KEY')
s3 = session.resource('s3') 
bucket = s3.Bucket('my_bucket')

## read shapefile
TPG = bucket.Object(key='/shapefiles/grid.zip') 
TPGrid = geopandas.read_file(TPG.get()['Body'])

但是当我尝试输出相同的地理数据框时，如下所示：

TPGrid.to_file(filename='s3://my_bucket/output/TPGrid.zip', driver='ESRI Shapefile')

我会得到错误代码：

ERROR:fiona._env:Only read-only mode is supported for /vsicurl
ERROR:fiona._env:Only read-only mode is supported for /vsicurl
ERROR:fiona._env:Only read-only mode is supported for /vsicurl
ERROR:fiona._env:Unable to open /vsis3/my_bucket/output/TPGrid.zip/TPGrid.shp or /vsis3/my_bucket/output/TPGrid.zip/TPGrid.SHP.
Traceback (most recent call last):
  File "fiona/ogrext.pyx", line 1133, in fiona.ogrext.WritingSession.start
  File "fiona/_err.pyx", line 291, in fiona._err.exc_wrap_pointer
fiona._err.CPLE_AppDefinedError: Unable to open /vsis3/my_bucket/output/TPGrid.zip/TPGrid.shp or /vsis3/my_bucket/output/TPGrid.zip/TPGrid.SHP.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/runscript.py", line 211, in <module>
    runpy.run_path(temp_file_path, run_name='__main__')
  File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/glue-python-scripts-c8krhm5u/test_to_file_geo.py", line 40, in <module>
  File "/glue/lib/installation/geopandas/geodataframe.py", line 1086, in to_file
    _to_file(self, filename, driver, schema, index, **kwargs)
  File "/glue/lib/installation/geopandas/io/file.py", line 328, in _to_file
    filename, mode=mode, driver=driver, crs_wkt=crs_wkt, schema=schema, **kwargs
  File "/glue/lib/installation/fiona/env.py", line 408, in wrapper
    return f(*args, **kwargs)
  File "/glue/lib/installation/fiona/__init__.py", line 274, in open
    **kwargs)
  File "/glue/lib/installation/fiona/collection.py", line 165, in __init__
    self.session.start(self, **kwargs)
  File "fiona/ogrext.pyx", line 1141, in fiona.ogrext.WritingSession.start
fiona.errors.DriverIOError: Unable to open /vsis3/my_bucket/output/TPGrid.zip/TPGrid.shp or /vsis3/my_bucket/output/TPGrid.zip/TPGrid.SHP.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/runscript.py", line 230, in <module>
    raise e_type(e_value).with_traceback(new_stack)
  File "/tmp/glue-python-scripts-c8krhm5u/test_to_file_geo.py", line 40, in <module>
  File "/glue/lib/installation/geopandas/geodataframe.py", line 1086, in to_file
    _to_file(self, filename, driver, schema, index, **kwargs)
  File "/glue/lib/installation/geopandas/io/file.py", line 328, in _to_file
    filename, mode=mode, driver=driver, crs_wkt=crs_wkt, schema=schema, **kwargs
  File "/glue/lib/installation/fiona/env.py", line 408, in wrapper
    return f(*args, **kwargs)
  File "/glue/lib/installation/fiona/__init__.py", line 274, in open
    **kwargs)
  File "/glue/lib/installation/fiona/collection.py", line 165, in __init__
    self.session.start(self, **kwargs)
  File "fiona/ogrext.pyx", line 1141, in fiona.ogrext.WritingSession.start
fiona.errors.DriverIOError: Unable to open /vsis3/my_bucket/output/TPGrid.zip/TPGrid.shp or /vsis3/my_bucket/output/TPGrid.zip/TPGrid.SHP.

我尝试了多种方法，例如使用“.csv”或“.shp”，但没有一种有效。我正在使用 python 3.6 和下面的软件包，希望这些信息能有所帮助：

• geopandas-0.9.0
• 匀称-1.7.1
• 菲奥娜-1.8.20
• GDAL-3.2.3

我一整天都在与这个问题作斗争......任何帮助将不胜感激。