从日志文件中提取Java堆栈跟踪的工具

Question

是否有任何工具可以提取出现在日志文件中的堆栈跟踪列表,并且可能会计算唯一的堆栈跟踪列表？

编辑:我会做一些不基于GUI的东西,并在后台运行并提供某种报告.我从很多环境收集了很多日志,只是想快速浏览一下.

Answer 1

这是一个快速而又脏的grep表达式...如果你使用的是log4j这样的记录器,那么异常的第一行通常会包含WARNor ERROR,下一行将包含Exception名称和可选的消息,然后后续堆栈跟踪将从以下之一开始:

1. "\tat" (tab + at)
2. "Caused by: "
3. "\t... <some number> more" (这些是指示堆栈中未显示在"由...引起"异常中的帧数的行)
4. 堆栈前的异常名称(可能是消息)

我们想得到以上所有的行,所以grep表达式是:

grep -P "(WARN|ERROR|^\tat |Exception|^Caused by: |\t... \d+ more)"

它假定一个Exception类总是包含Exception可能或可能不是真的单词,但毕竟这是快速而又脏的.

根据您的具体情况进行调整.

Answer 2

你可以很容易地自己写这个.这是模式:

1. 打开文件
2. 搜索字符串"\n\tat "(即新行,制表符at,空白)这是堆栈跟踪之外的非常罕见的字符串.

现在您需要做的就是找到第一行不能\t找到堆栈跟踪结束的行.您可能希望在此之后跳过1-3行来捕获链式异常.

另外,在堆栈跟踪的第一行之前添加几行(比如10或50)以获得一些上下文.

Answer 3

我用Python编写了一个工具.它设法分割两个堆栈跟踪,即使它们在日志中正好相继.

#!/usr/bin/env python
#
# Extracts exceptions from log files.
#

import sys
import re
from collections import defaultdict

REGEX = re.compile("(^\tat |^Caused by: |^\t... \\d+ more)")
# Usually, all inner lines of a stack trace will be "at" or "Caused by" lines.
# With one exception: the line following a "nested exception is" line does not
# follow that convention. Due to that, this line is handled separately.
CONT = re.compile("; nested exception is: *$")

exceptions = defaultdict(int)

def registerException(exc):
  exceptions[exc] += 1

def processFile(fileName):
  with open(fileName, "r") as fh:
    currentMatch = None
    lastLine = None
    addNextLine = False
    for line in fh.readlines():
      if addNextLine and currentMatch != None:
        addNextLine = False
        currentMatch += line
        continue
      match = REGEX.search(line) != None
      if match and currentMatch != None:
        currentMatch += line
      elif match:
        currentMatch = lastLine + line
      else:
        if currentMatch != None:
          registerException(currentMatch)
        currentMatch = None
      lastLine = line
      addNextLine = CONT.search(line) != None
    # If last line in file was a stack trace
    if currentMatch != None:
      registerException(currentMatch)

for f in sys.argv[1:]:
  processFile(f)

for item in sorted(exceptions.items(), key=lambda e: e[1], reverse=True):
  print item[1], ":", item[0]

Answer 4

我想出了以下 Groovy 脚本。当然，它非常适合我的需要，但我希望它对某人有所帮助。

def traceMap = [:]

// Number of lines to keep in buffer
def BUFFER_SIZE = 100

// Pattern for stack trace line
def TRACE_LINE_PATTERN = '^[\\s\\t]+at .*$'

// Log line pattern between which we try to capture full trace
def LOG_LINE_PATTERN = '^([<#][^/]|\\d\\d).*$'

// List of patterns to replace in final captured stack trace line 
// (e.g. replace date and transaction information that may make similar traces to look as different)
def REPLACE_PATTERNS = [
  '^\\d+-\\d+\\@.*?tksId: [^\\]]+\\]',
  '^<\\w+ \\d+, \\d+ [^>]*?> <[^>]*?> <[^>]*?> <[^>]*?> <',
  '^####<[^>]+?> <[^>]*?> <[^>]*?> <[^>]*?> <[^>]*?> <[^>]*?> <[^>]*?> <[^>]*?> <[^>]*?> <[^>]*?> <[^>]*?> <',
  '<([\\w:]+)?TransaktionsID>[^<]+?</([\\w:]+)?TransaktionsID>',
  '<([\\w:]+)?TransaktionsTid>[^<]+?</([\\w:]+)?TransaktionsTid>'
]

new File('.').eachFile { File file ->
  if (file.name.contains('.log') || file.name.contains('.out')) {
    def bufferLines = []
    file.withReader { Reader reader ->
      while (reader.ready()) {      
        def String line = reader.readLine()
        if (line.matches(TRACE_LINE_PATTERN)) {
          def trace = []
          for(def i = bufferLines.size() - 1; i >= 0; i--) {
            if (!bufferLines[i].matches(LOG_LINE_PATTERN)) {
              trace.add(0, bufferLines[i])
            } else {
              trace.add(0, bufferLines[i])
              break
            }
          }
          trace.add(line)
          if (reader.ready()) {
            line = reader.readLine()
            while (!line.matches(LOG_LINE_PATTERN)) {
              trace.add(line)
              if (reader.ready()) {
                line = reader.readLine()
              } else {
                break;
              }
            }
          }
          def traceString = trace.join("\n")
          REPLACE_PATTERNS.each { pattern ->
            traceString = traceString.replaceAll(pattern, '')
          }
          if (traceMap.containsKey(traceString)) {
            traceMap.put(traceString, traceMap.get(traceString) + 1)
          } else {
            traceMap.put(traceString, 1)
          }
        }
        // Keep the buffer of last lines.
        bufferLines.add(line)
        if (bufferLines.size() > BUFFER_SIZE) {
          bufferLines.remove(0)
        }
      }
    }
  }
}

traceMap = traceMap.sort { it.value }

traceMap.reverseEach { trace, number ->
  println "-- Occured $number times -----------------------------------------"
  println trace
}