在python中的字符串上为"for each"循环更改分隔符

Question

我需要通过逐行流式传输读取python中的输入文本文件.这意味着逐行加载文本文件而不是一次加载到内存中.但我的行分隔符不是空格,它们是任意字符.

这是Stack Overflow上用于逐行加载文件的方法:

with open("log.txt") as infile:
    for line in infile:
        do_something_with(line)

以上是完美的,但我需要将分隔符从空格更改为不同的字符.

如何才能做到这一点？谢谢.

Answer 1

import re
def open_delimited(filename, delimiter, chunksize=1024, *args, **kwargs):
    with open(filename, *args, **kwargs) as infile:
        remainder = ''
        for chunk in iter(lambda: infile.read(chunksize), ''):
            pieces = re.split(delimiter, remainder+chunk)
            for piece in pieces[:-1]:
                yield piece
            remainder = pieces[-1]
        if remainder:
            yield remainder

for line in open_delimited("log.txt", delimiter='/'):
    print(repr(line))