如何在 FastAPI 中上传 CSV 文件并将其转换为 JSON？

Question

我正在尝试将.csv文件上传到 FastAPI 服务器，然后将其转换为 JSON 并将其返回给客户端。但是，当我尝试直接处理它（而不将其存储在某处）时，我收到此错误：

Error : FileNotFoundError: [Error 2] No such file or directory : "testdata.csv"

这是我的 FastAPI 代码：

async def upload(file: UploadFile = File(...)):
    data = {}    
    with open(file.filename,encoding='utf-8') as csvf:
        csvReader = csv.DictReader(csvf)
        for rows in csvReader:             
            key = rows['No']
            data[key] = rows    
    return {data}```

Answer 1

下面给出了如何将上传的.csv文件转换为 JSON 的各种选项。.csv下面的示例中使用了以下示例文件。

\n

数据.csv

\n

Id,name,age,height,weight\n1,Alice,20,62,120.6\n2,Freddie,21,74,190.6\n3,Bob,17,68,120.0\n

\n

选项1

\n

该方法也csv.DictReader()可以接受文件对象file作为参数。FastAPI使用 Python ，一个类似文件的对象（有关更多信息，请查看此答案）。您可以通过对象的属性来访问它。但是，由于 FastAPI/Starlette 以模式打开文件，如果直接将其传递给方法，则会收到错误，即. 因此，您可以使用（如本答案中所建议的）使用增量解码器来迭代解码迭代器提供的输入（在本例中为 from to ）。例子：UploadFileSpooledTemporaryFile.fileUploadFilebytescsv.DictReader()_csv.Error: iterator should return strings, not bytescodecs.iterdecode()bytesstr

\n

Id,name,age,height,weight\n1,Alice,20,62,120.6\n2,Freddie,21,74,190.6\n3,Bob,17,68,120.0\n

\n

输出

\n

from fastapi import FastAPI, File, UploadFile\nimport csv\nimport codecs\n\napp = FastAPI()\n    \n@app.post("/upload")\ndef upload(file: UploadFile = File(...)):\n    csvReader = csv.DictReader(codecs.iterdecode(file.file, \'utf-8\'))\n    data = {}\n    for rows in csvReader:             \n        key = rows[\'Id\']  # Assuming a column named \'Id\' to be the primary key\n        data[key] = rows  \n    \n    file.file.close()\n    return data\n

\n

如果您想返回list字典，您可以使用以下内容。由于下面的代码需要在返回结果时file执行，因此会阻止服务器在完成后正确关闭文件（通过调用），因此可以使用（在返回响应后运行）来关闭文件：openfile.file.close()BackgroundTasks

\n

{\n  "1": {\n    "Id": "1",\n    "name": "Alice",\n    "age": "20",\n    "height": "62",\n    "weight": "120.6"\n  },\n  "2": {\n    "Id": "2",\n    "name": "Freddie",\n    "age": "21",\n    "height": "74",\n    "weight": "190.6"\n  },\n  "3": {\n    "Id": "3",\n    "name": "Bob",\n    "age": "17",\n    "height": "68",\n    "weight": "120.0"\n  }\n}\n

\n

输出

\n

from fastapi import FastAPI, File, UploadFile, BackgroundTasks\nimport csv\nimport codecs\n\napp = FastAPI()\n\n@app.post("/upload")\ndef upload(background_tasks: BackgroundTasks, file: UploadFile = File(...)):\n    csvReader = csv.DictReader(codecs.iterdecode(file.file, \'utf-8\'))\n    background_tasks.add_task(file.file.close)\n    return list(csvReader)\n

\n

选项2

\n

另一个解决方案是使用contents = file.file.read()（async读/写请参阅此答案）\xe2\x80\x94 读取上传文件\xe2\x80\x94 的字节数据，然后将字节转换为字符串，最后将它们加载到内存中文本缓冲区（即StringIO），如此处所述，可以传递给csv.DictReader(). 下面的例子：

\n

[\n  {\n    "Id": "1",\n    "name": "Alice",\n    "age": "20",\n    "height": "62",\n    "weight": "120.6"\n  },\n  {\n    "Id": "2",\n    "name": "Freddie",\n    "age": "21",\n    "height": "74",\n    "weight": "190.6"\n  },\n  {\n    "Id": "3",\n    "name": "Bob",\n    "age": "17",\n    "height": "68",\n    "weight": "120.0"\n  }\n]\n

\n

选项3

\n

要以您的方式解决问题\xe2\x80\x94，即使用文件路径读取csv文件，而不是直接使用文件内容或类似文件的对象，如早期\xe2\x80\x94所述，您可以将文件内容复制到 a 中，与提供的NamedTemporaryFile不同，“在文件系统中有一个可见的名称”，“可用于打开文件”（再次，请检查此答案以获取更多信息）。下面是一个工作示例：SpooledTemporaryFileUploadFile

\n

from fastapi import FastAPI, File, UploadFile\nimport csv\nfrom io import StringIO\n\napp = FastAPI()\n    \n@app.post("/upload")\ndef upload(file: UploadFile = File(...)):\n    data = {}\n    contents = file.file.read()\n    buffer = StringIO(contents.decode(\'utf-8\'))\n    csvReader = csv.DictReader(buffer)\n    for row in csvReader:  \n        key = row[\'Id\']  # Assuming a column named \'Id\' to be the primary key\n        data[key] = row  \n    \n    buffer.close()\n    file.file.close()\n    return data\n

\n

选项4

\n

您还可以将上传文件中的字节写入BytesIO流，然后将其转换为 Pandas DataFrame。接下来，使用该to_dict()方法（如本答案中所述），您可以将数据帧转换为字典并返回它\xe2\x80\x94，FastAPI在幕后将使用 ，jsonable_encoder最后转换为JSON兼容的数据，序列化数据并返回 a JSONResponse（有关更多详细信息，请参阅此答案）。作为更快的替代方案，您可以使用该to_json()方法并直接返回自定义，如本答案的选项 1（更新 2）Response中所述。

\n

from fastapi import FastAPI, File, UploadFile\nfrom tempfile import NamedTemporaryFile\nimport os\nimport csv\n\napp = FastAPI()\n    \n@app.post("/upload")\ndef upload(file: UploadFile = File(...)):\n    data = {}\n    temp = NamedTemporaryFile(delete=False)\n    try:\n        try:\n            contents = file.file.read()\n            with temp as f:\n                f.write(contents);\n        except Exception:\n            return {"message": "There was an error uploading the file"}\n        finally:\n            file.file.close()\n        \n        with open(temp.name,\'r\', encoding=\'utf-8\') as csvf:\n            csvReader = csv.DictReader(csvf)\n            for rows in csvReader:             \n                key = rows[\'Id\']  # Assuming a column named \'Id\' to be the primary key\n                data[key] = rows\n    except Exception:\n        return {"message": "There was an error processing the file"}\n    finally:\n        #temp.close()  # the `with` statement above takes care of closing the file\n        os.remove(temp.name)  # Delete the file\n    \n    return data\n

\n

注意：如果文件太大并且占用了所有内存和/或花费太多时间来处理和/或返回结果，请查看此答案以及此答案和此答案。

\n

Answer 2

您收到的原因Error : FileNotFoundError: [Error 2] No such file or directory : "testdata.csv"是因为您正在尝试读取未存储在本地的文件。

如果您想以这种方式读取文件，您应该在继续之前保存上传的文件：

async def upload(uploaded_file: UploadFile = File(...)):
    # save csv to local dir
    csv_name = uploaded_file.filename
    csv_path = 'path_to/csv_dir/'
    file_path = os.path.join(csv_path, csv_name)
    with open(file_path, mode='wb+') as f:
        f.write(uploaded_file.file.read())

    # read csv and convert to json
    data = {}
    with open(file_path, mode='r', encoding='utf-8') as csvf:
        csvReader = csv.DictReader(csvf)
        for rows in csvReader:             
            key = rows['No']
            data[key] = rows    
    return {data}