在Python中使用csvreader对gzip文件

Question

我有一堆gzip压缩文件,我想用Python的内置CSV阅读器打开检查.我想在没有先将它们手动解压缩到磁盘的情况下执行此操作.我想我想以某种方式获得未压缩数据的流,并将其传递给CSV阅读器.这在Python中可行吗？

Answer 1

使用gzip模块:

with gzip.open(filename, mode='rt') as f:
    reader = csv.reader(f)
    #...

Answer 2

我已经尝试过上面的版本进行编写和阅读,由于"字节"错误,它在Python 3.3中无效.但是,经过一些试验和错误,我可以得到以下工作.也许它也有助于其他人:

import csv
import gzip
import io


with gzip.open("test.gz", "w") as file:
    writer = csv.writer(io.TextIOWrapper(file, newline="", write_through=True))
    writer.writerow([1, 2, 3])
    writer.writerow([4, 5, 6])

with gzip.open("test.gz", "r") as file:
    reader = csv.reader(io.TextIOWrapper(file, newline=""))
    print(list(reader))

正如amohr建议的那样,以下方面也有效:

import gzip, csv

with gzip.open("test.gz", "wt", newline="") as file:
    writer = csv.writer(file)
    writer.writerow([1, 2, 3])
    writer.writerow([4, 5, 6])

with gzip.open("test.gz", "rt", newline="") as file:
    reader = csv.reader(file)
    print(list(reader))

Answer 3

更完整的解决方案:

import csv, gzip
class GZipCSVReader:
    def __init__(self, filename):
        self.gzfile = gzip.open(filename)
        self.reader = csv.DictReader(self.gzfile)
    def next(self):
        return self.reader.next()
    def close(self):
        self.gzfile.close()
    def __iter__(self):
        return self.reader.__iter__()

现在你可以像这样使用它:

r = GZipCSVReader('my.csv')
for map in r:
    for k,v in map:
        print k,v
r.close()

编辑:按照以下评论,如何更简单的方法:

def gzipped_csv(filename):
    with gzip.open(filename) as f:
        r = csv.DictReader(f)
        for row in r:
            yield row

哪个让你呢

for row in gzipped_csv(filename):
    for k, v in row:
        print(k, v)