如何读取从URL -Gzip压缩的CSV文件

Question

我正在请求一个gzip压缩的csv文件.

如何解压缩该文件并将其转换为csv对象？

csv_gz_file = get("example.com/filename.csv.gz", headers=csv_headers, timeout=30, stream=True)

reader = csv.reader(csv_gz_file)
for row in reader:
   print row

它抛出这个因为它没有解压缩

_csv.Error: line contains NULL byte

Answer 1

import gzip
import io
import requests

web_response = requests.get("example.com/filename.csv.gz", headers=csv_headers,
                            timeout=30, stream=True)
csv_gz_file = web_response.content # Content in bytes from requests.get
                                   # See comments below why this is used.

f = io.BytesIO(csv_gz_file)
with gzip.GzipFile(fileobj=f) as fh:
    # Passing a binary file to csv.reader works in PY2
    reader = csv.reader(fh)
    for row in reader:
        print(row)

通过将gz数据保存在内存中,使用gzip模块提取它,然后将明文数据读入另一个内存容器,最后用读取器打开该容器.

我对csv.reader文件处理或list数据的期望有点不确定,但我认为这样可行.如果不是简单的话:

reader = csv.reader(csv_content.splitlines())

这应该可以解决问题.