Python将日志滚动到变量

Question

我有一个使用多线程的应用程序,并在后台运行在服务器上.为了在不必登录服务器的情况下监视应用程序,我决定包含Bottle以响应一些HTTP端点并报告状态,执行远程关闭等.

我还想添加一种方法来查阅日志文件.我可以使用FileHandler并在请求URL时发送目标文件(例如/log).

但是,我想知道是否可以实现像a这样的东西RotatingFileHandler,而不是记录到文件,记录到变量(例如BytesIO).这样,我可以将日志限制为最新信息,同时能够将其作为文本返回到浏览器,而不是单独的文件下载.

将RotatingFileHandler需要的文件名,所以它不是一个选项,它传递一个BytesIO流.记录到变量本身是完全可行的(例如,在变量中捕获Python日志输出),但我对如何进行滚动部分感到有点困惑.

任何想法,提示,建议将不胜感激.

Answer 1

使用捕获变量中的Python日志输出中描述的技术,但将其捕获到抛弃旧数据的自定义流中.

像这样:

# Adapted from http://alanwsmith.com/capturing-python-log-output-in-a-variable

import logging
import io
import collections

class FIFOIO(io.TextIOBase):
    def __init__(self, size, *args):
        self.maxsize = size
        io.TextIOBase.__init__(self, *args)
        self.deque = collections.deque()
    def getvalue(self):
        return ''.join(self.deque)
    def write(self, x):
        self.deque.append(x)
        self.shrink()
    def shrink(self):
        if self.maxsize is None:
            return
        size = sum(len(x) for x in self.deque)
        while size > self.maxsize:
            x = self.deque.popleft()
            size -= len(x)

### Create the logger
logger = logging.getLogger('basic_logger')
logger.setLevel(logging.DEBUG)

### Setup the console handler with a FIFOIO object
log_capture_string = FIFOIO(256)
ch = logging.StreamHandler(log_capture_string)
ch.setLevel(logging.DEBUG)

### Optionally add a formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)

### Add the console handler to the logger
logger.addHandler(ch)


### Send log messages. 
logger.debug('debug message')
logger.info('info message')
logger.warn('warn message')
logger.error('error message')
logger.critical('critical message')


### Pull the contents back into a string and close the stream
log_contents = log_capture_string.getvalue()
log_capture_string.close()

### Output as lower case to prove it worked. 
print(log_contents.lower())

Answer 2

根据 Andrew Guy 的建议，我进一步分类logging.Handler并实现了一个处理程序，该处理程序使用collections.deque固定长度来保存日志消息的记录。

import logging
import collections


class TailLogHandler(logging.Handler):

    def __init__(self, log_queue):
        logging.Handler.__init__(self)
        self.log_queue = log_queue

    def emit(self, record):
        self.log_queue.append(self.format(record))


class TailLogger(object):

    def __init__(self, maxlen):
        self._log_queue = collections.deque(maxlen=maxlen)
        self._log_handler = TailLogHandler(self._log_queue)

    def contents(self):
        return '\n'.join(self._log_queue)

    @property
    def log_handler(self):
        return self._log_handler

用法示例：

import random

logger = logging.getLogger(__name__)

tail = TailLogger(10)

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

log_handler = tail.log_handler
log_handler.setFormatter(formatter)
logger.addHandler(log_handler)

levels = [logging.INFO, logging.ERROR, logging.WARN, logging.DEBUG, logging.CRITICAL]
logger.setLevel(logging.ERROR)

for i in range(500):
    logger.log(random.choice(levels), 'Message {}'.format(i))

print(tail.contents())

输出：

2016-06-22 13:58:25,975 - __main__ - CRITICAL - Message 471
2016-06-22 13:58:25,975 - __main__ - ERROR - Message 472
2016-06-22 13:58:25,975 - __main__ - ERROR - Message 473
2016-06-22 13:58:25,975 - __main__ - ERROR - Message 474
2016-06-22 13:58:25,975 - __main__ - ERROR - Message 477
2016-06-22 13:58:25,975 - __main__ - CRITICAL - Message 481
2016-06-22 13:58:25,975 - __main__ - CRITICAL - Message 483
2016-06-22 13:58:25,975 - __main__ - ERROR - Message 484
2016-06-22 13:58:25,975 - __main__ - CRITICAL - Message 485
2016-06-22 13:58:25,976 - __main__ - CRITICAL - Message 490