如何在 sagemaker 笔记本中打开存储在 S3 存储桶中的模型 tarfile？

Question

我知道从 S3 存储桶将 .csv 文件加载到 sagemaker 笔记本中非常简单，但我想加载存储在 S3 存储桶中的 model.tar.gz 文件。我尝试执行以下操作

import botocore 
import sagemaker
from sagemaker import get_execution_role
from sagemaker.predictor import csv_serializer
import boto3

sm_client = boto3.client(service_name='sagemaker')
runtime_sm_client = boto3.client(service_name='sagemaker-runtime')

s3 = boto3.resource('s3')
s3_client = boto3.client('s3')

sagemaker_session = sagemaker.Session()
role = get_execution_role()

ACCOUNT_ID  = boto3.client('sts').get_caller_identity()['Account']
REGION      = boto3.Session().region_name
BUCKET      = 'sagemaker.prismade.net'
data_key    = 'DEMO_MME_ANN/multi_model_artifacts/axel.tar.gz'
loc = 's3://{}/{}'.format(BUCKET, data_key)
print(loc)
with tarfile.open(loc) as tar:
    tar.extractall(path='.')

我收到以下错误：

--------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-215-bfdddac71b95> in <module>()
     20 loc = 's3://{}/{}'.format(BUCKET, data_key)
     21 print(loc)
---> 22 with tarfile.open(loc) as tar:
     23     tar.extractall(path='.')

~/anaconda3/envs/python3/lib/python3.6/tarfile.py in open(cls, name, mode, fileobj, bufsize, **kwargs)
   1567                     saved_pos = fileobj.tell()
   1568                 try:
-> 1569                     return func(name, "r", fileobj, **kwargs)
   1570                 except (ReadError, CompressionError):
   1571                     if fileobj is not None:

~/anaconda3/envs/python3/lib/python3.6/tarfile.py in gzopen(cls, name, mode, fileobj, compresslevel, **kwargs)
   1632 
   1633         try:
-> 1634             fileobj = gzip.GzipFile(name, mode + "b", compresslevel, fileobj)
   1635         except OSError:
   1636             if fileobj is not None and mode == 'r':

~/anaconda3/envs/python3/lib/python3.6/gzip.py in __init__(self, filename, mode, compresslevel, fileobj, mtime)
    161             mode += 'b'
    162         if fileobj is None:
--> 163             fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
    164         if filename is None:
    165             filename = getattr(fileobj, 'name', '')

FileNotFoundError: [Errno 2] No such file or directory: 's3://sagemaker.prismade.net/DEMO_MME_ANN/multi_model_artifacts/axel.tar.gz'

这里有什么错误，我该如何实现这个目标？

Answer 1

并非每个设计用于文件系统（本例中为 tarfile.open）的 Python 库都知道如何从 S3 中将对象作为文件读取。

解决这个问题的简单方法是首先将对象作为文件复制到本地文件系统中。

import boto3

s3 = boto3.client('s3')
s3.download_file('BUCKET_NAME', 'OBJECT_NAME', 'FILE_NAME')