LtP*_*ack 6 python logging exception
我发现自己将python用于许多文件管理脚本,如下所示.在网上查找示例时,我很惊讶示例中的日志记录和异常处理有多少.每次我写一个新的脚本时,我的意图都不会像下面那样结束,但如果它处理文件,那么无论我的偏执狂接管了什么,最终结果都不像我在网上看到的例子.因为我是新手,我想知道这是否正常.如果没有,那么你如何处理未知和删除有价值信息的恐惧？
def flatten_dir(dirname):
'''Flattens a given root directory by moving all files from its sub-directories and nested
sub-directories into the root directory and then deletes all sub-directories and nested
sub-directories. Creates a backup directory preserving the original structure of the root
directory and restores this in case of errors.
'''
RESTORE_BACKUP = False
log.info('processing directory "%s"' % dirname)
backup_dirname = str(uuid.uuid4())
try:
shutil.copytree(dirname, backup_dirname)
log.debug('directory "%s" backed up as directory "%s"' % (dirname,backup_dirname))
except shutil.Error:
log.error('shutil.Error: Error while trying to back up the directory')
sys.stderr.write('the program is terminating with an error\n')
sys.stderr.write('press consult the log file\n')
sys.stderr.flush()
time.sleep(0.25)
print 'Press any key to quit this program.'
msvcrt.getch()
sys.exit()
for root, dirs, files in os.walk(dirname, topdown=False):
log.debug('os.walk passing: (%s, %s, %s)' % (root, dirs, files))
if root != dirname:
for file in files:
full_filename = os.path.join(root, file)
try:
shutil.move(full_filename, dirname)
log.debug('"%s" copied to directory "%s"' % (file,dirname))
except shutil.Error:
RESTORE_BACKUP = True
log.error('file "%s" could not be copied to directory "%s"' % (file,dirname))
log.error('flagging directory "%s" for reset' % dirname)
if not RESTORE_BACKUP:
try:
shutil.rmtree(root)
log.debug('directory "%s" deleted' % root)
except shutil.Error:
RESTORE_BACKUP = True
log.error('directory "%s" could not be deleted' % root)
log.error('flagging directory "%s" for reset' % dirname)
if RESTORE_BACKUP:
break
if RESTORE_BACKUP:
RESTORE_FAIL = False
try:
shutil.rmtree(dirname)
except shutil.Error:
log.error('modified directory "%s" could not be deleted' % dirname)
log.error('manual restoration from backup directory "%s" necessary' % backup_dirname)
RESTORE_FAIL = True
if not RESTORE_FAIL:
try:
os.renames(backup_dirname, dirname)
log.debug('back up of directory "%s" restored' % dirname)
print '>'
print '>******WARNING******'
print '>There was an error while trying to flatten directory "%s"' % dirname
print '>back up of directory "%s" restored' % dirname
print '>******WARNING******'
print '>'
except WindowsError:
log.error('backup directory "%s" could not be renamed to original directory name' % backup_dirname)
log.error('manual renaming of backup directory "%s" to original directory name "%s" necessary' % (backup_dirname,dirname))
print '>'
print '>******WARNING******'
print '>There was an error while trying to flatten directory "%s"' % dirname
print '>back up of directory "%s" was NOT restored successfully' % dirname
print '>no information is lost'
print '>check the log file for information on manually restoring the directory'
print '>******WARNING******'
print '>'
else:
try:
shutil.rmtree(backup_dirname)
log.debug('back up of directory "%s" deleted' % dirname)
log.info('directory "%s" successfully processed' % dirname)
print '>directory "%s" successfully processed' % dirname
except shutil.Error:
log.error('backup directory "%s" could not be deleted' % backup_dirname)
log.error('manual deletion of backup directory "%s" necessary' % backup_dirname)
print '>'
print '>******WARNING******'
print '>directory "%s" successfully processed' % dirname
print '>cleanup of backup directory "%s" failed' % backup_dirname
print '>manual cleanup necessary'
print '>******WARNING******'
print '>'
学会放手(或者我学会了如何与炸弹一起生活)......
问问自己:你究竟害怕什么,如果它发生了,你将如何处理？在您提供的示例中,您希望避免数据丢失.您处理它的方式是查找您认为是错误的每种条件组合,并对其进行大量日志记录.事情仍然会出错,并且不清楚拥有大量日志记录将是处理它的好方法.勾勒出你想要实现的目标:
for each file in a tree
if file is below the root
move it into the root
if nothing went wrong
delete empty subtrees
那么在这个过程中会出现什么样的问题呢？好吧,由于底层文件系统,移动文件操作可以通过多种方式进行barf.我们可以列出所有这些并提供处理它们的好方法吗？不......但总的来说,你将以同样的方式处理它们.有时错误只是一个错误,无论它是什么.
因此,在这种情况下,如果发生任何错误,那么您要中止并撤消任何更改.您决定这样做的方法是创建备份副本并在出现问题时进行恢复.但你最可能的错误是文件系统已满,在这种情况下这些步骤可能会失败......好吧,这是一个常见的问题 - 如果你担心任何一点未知的错误,你如何阻止你的恢复错误的路径？
一般的答案是确保你先做任何中间工作,然后采取一个麻烦(希望是原子)步骤.在你的情况下,你需要翻转你的恢复.而不是将副本构建为备份,而是构建结果的副本.如果一切都成功,则可以将新结果替换为旧的原始树.或者,如果你真的是偏执狂,你可以为人类留下这一步.这里的优点是,如果出现问题,您可以中止并丢弃您构建的部分状态.
然后你的结构变成:
make empty result directory
for every file in the tree
copy file into new result
on failure abort otherwise
move result over old source directory
顺便说一句,当前脚本中存在一个错误,即此伪代码更明显:如果在不同分支中具有相同名称的文件,则它们将在新的展平版本中相互覆盖.
关于这个伪代码的第二点是所有的错误处理都在同一个地方(即将make new目录和递归副本包装在一个try块中并捕获它之后的所有错误),这解决了你的原始问题记录/错误检查与实际工作代码的比例很大.
backup_dirname = str(uuid.uuid4())
try:
shutil.mkdir(backup_dirname)
for root, dirs, files in os.walk(dirname, topdown=False):
for file in files:
full_filename = os.path.join(root, file)
target_filename = os.path.join(backup_dirname,file)
shutil.copy(full_filename, target_filename)
catch Exception, e:
print >>sys.stderr, "Something went wrong %s" % e
exit(-1)
shutil.move(back_dirname,root) # I would do this bit by hand really