Python代码运行缓慢

Question

Python代码运行速度极慢.快速启动,然后变成爬行.我可以做些什么来加快速度？我正在提取一个文本文件,读取文件的内容,过滤文本文件的内容,然后将其写入csv以供其他人以后用于json.

我刚开始这样做,我相信你可以告诉我.任何帮助将不胜感激.

import glob
from pathlib import Path
import datetime
import re
import csv


get_this = []
thislist = []

def timeteller():
    now = datetime.datetime.now()
    month = str('{:02d}'.format(now.month))
    day1 = now.day -1
    day =  str('{:02d}'.format(day1))
    year =str(now.year)
    time =year+month+day
    return time

def these_files(x, y):

    configfiles = Path('O:/Unit Management/Reports/G4S/').glob('{}*/{}*Transaction.txt'.format(x, y))

    for files in configfiles:
        thislist.append(files)

    return thislist

def hasNumbers(inputString):
    numberfinal = []
    numberfinal = re.findall("\d+", inputString)
    if numberfinal == []:
        numberfinal = '1'
    return numberfinal

def get_odometers(thesepath):

    for thispath in thesepath:

        with open(thispath,"r") as f:
            searchlines = f.readlines()
        for i, line in enumerate(searchlines):
            if "Odometers" in line:
                get_this.append(line)
            elif "Lifetime" in line:
                get_this.append(line)


    return get_this

def make_pretty(checkthis):
    the_numbers = {}
    the_numbers['Serial'] = banumber
    for i, line in enumerate(checkthis):
        the_numbers['Serial'] = banumber
        if '(BNR) Odometers  Accept' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['BNR'] = numberschecked[0]
        elif '(BNR 2) Odometers  Accept' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['BNR 2'] = numberschecked[0]
        elif '(BCR) Odometers  Accept' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['BCR'] = numberschecked[0]
        elif '(BCR) Odometers  Hopper1' in line:
            for l in checkthis[i:i+2]:
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['Hopper1'] = numberschecked[0]
        elif '(BCR) Odometers  Hopper2' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['Hopper2'] = numberschecked[0]
        elif '(BCR) Odometers  Hopper3' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['Hopper3'] = numberschecked[0]
        elif '(BCR) Odometers  Hopper4' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['Hopper4'] = numberschecked[0]
        elif '(BCR) Odometers  Hopper5' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['Hopper5'] = numberschecked[0]
        elif '(BCR) Odometers  Hopper6' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['Hopper6'] = numberschecked[0]
        elif '(BCR) Odometers  Hopper7' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['Hopper7'] = numberschecked[0]
        elif '(BCR) Odometers  Hopper8' in line:
            for l in checkthis[i:i+2]: 
                numbers = l[0:20]
                numberschecked = hasNumbers(numbers)
                the_numbers['Hopper8'] = numberschecked[0]


    return the_numbers

intnow = int(timeteller())
intnow -= 1
now = str(intnow)
thelist = []
thispath = open('banumberlist1.txt')
finallist = []
for files in thispath:
    getem = files
    banumber = getem[0:8]
    print(banumber)
    combined = '{}_{}'.format(banumber,now)
    thepaths = these_files(banumber, combined)
    needtomakepretty = get_odometers(thepaths)
    goeslast = make_pretty(needtomakepretty)
    finallist.append(goeslast)


f = open ('odom01.txt', 'w')
for ba in finallist:
    sba = str(ba)
    f.write(sba)    

Answer 1

如果您想知道代码的哪一部分很慢,则需要使用分析器.

您可以使用cProfile哪个是标准库的一部分:

python -m cProfile -o profile.txt -s cumtime myscript.py

这会将分析结果写入profile.txt,按累计时间对配置文件进行排序.这将为您提供程序花费时间的概述.

有关更详细的视图,您可以使用line_profiler模块.这甚至可以显示每行代码使用的时间.

安装后line_profiler,将以下内容添加到脚本的开头:

import line_profiler
import atexit
profile = line_profiler.LineProfiler()
atexit.register(profile.print_stats)

现在将@profile装饰器添加到您的所有函数中,如下所示:

@profile
def timeteller():
    now = datetime.datetime.now()
    month = str('{:02d}'.format(now.month))
    day1 = now.day -1
    day =  str('{:02d}'.format(day1))
    year =str(now.year)
    time =year+month+day
    return time

你应该把脚本末尾的东西放到一个函数中(例如main)并添加@profile它.

现在运行您的脚本,您将看到一个配置文件.