标签: logql

我正在使用 Grafana Loki，我需要计算特定时间间隔内特定日志消息的总数。例如，我需要12:00:00到14:00:00期间日志消息“some-text”的总数。我刚刚找到了以下方法来计算最后一分钟的出现次数，例如：count_over_time({container="some-containter"} |= "some-text")[1m]，但我没有找到任何方法来查询特定的时间间隔。

如果这是可能的并且有人可以提供帮助，我将非常高兴。

我正在向 loki 发送 json 日志并在 grafana 中进行可视化。最初，我的日志如下所示。

  {   
     "log": "{\"additionalDetails\":{\"body\":{},\"ip\":\"::ffff:1.1.1.1\",\"params\":{},\"query\":{},\"responseTime\":0,\"userAgent\":\"ELB-HealthChecker/2.0\"},\"context\":\"http\",\"endpoint\":\"/healthz\",\"level\":\"info\",\"message\":\"[::ffff:1.1.1.1] HTTP/1.1 GET 200 /healthz 0ms\",\"requestId\":\"9fde4910-86cd-11ec-a1c5-cd8277a61e4a\",\"statusCode\":200}\n",   
     "stream": "stdout",   
     "time": "2022-02-05T21:49:58.178290044Z" 
  }

为了使其更可用，我使用以下查询。

{app="awesome-loki-logs-with-grafana"} | json | line_format "{{.log}}" 

结果真的很好。它会自动检测以下文件。

如何通过 grafana 已检测到的 statusCode 进行过滤？

对于同一个应用程序，我们有不同类型的日志。有些来自我们的应用程序，以 JSON 格式记录，有些则来自不同类型的日志消息。

例如这 3 行日志：

"{\"written_at\": \"2022-03-30T07:51:04.934Z\", \"written_ts\": 1648626664934052000, \"msg\": \"Step 'X' started at 2022-03-30 07:51:04\", \"type\": \"log\", \"logger\": \"my-logger\", \"thread\": \"MainThread\", \"level\": \"DEBUG\", \"module\": \"my.module\", \"line_no\": 48}\n"
"                    ERROR    Data processing error: Did not work       \n"
"FileNotFoundError: [Errno 2] No such file or directory: '/local.json'\n"

为了解析应用程序 JSON 日志，我们执行以下 LogQL 查询：

| json log="log" 
| line_format "{{.log}}"
| json | line_format "{{.msg}}"
| __error__ != "JSONParserErr"

正如我们的查询已经指出的那样，我们无法解析其他行日志，因为它们不是 JSON 格式。

我们能否根据条件定义不同的解析和格式？或者作为 JSONParserErr 发生时的后备？

我正在使用 Grafana 和 Loki 来分析应用程序中的日志，并且已经在一些地方使用了 Bar Gauge。

这是我的查询：

sum(count_over_time({namespace=~"$namespace", job=~"$namespace-logs"} 
|= "KPIExecuted" [$__interval])) by (namespace)

我得到：

我想知道是否可以根据指标结果对结果进行排序？或者我是否可以使用其他替代方案来实现类似的结果？

谢谢。

我想将 Kibana 查询转换为 LogQL：

host:("test1-myservice-*") AND level:ERROR 
AND NOT logger_name:"com.example.ExampleClass" 
AND _exists_:stack_trace 
AND NOT stack_trace:(
    "interrupted"
    OR "Read timed out"
    OR "java.lang.InterruptedException"
)

我在 Grafana Explore 中尝试了以下操作，但它没有返回 JSON 日志消息的任何记录：

{host=~"test1-myservice-.*"} | json 
| logger_name != "com.example.ExampleClass" 
| stack_trace !="" 
| stack_trace =~ ".*InterruptedException.*"

当使用!=代替它时=~，它会返回所有记录：

{host=~"test1-myservice-.*"} | json 
| logger_name != "com.example.ExampleClass" 
| stack_trace !="" 
| stack_trace !~ ".*InterruptedException.*"

如果我是对的，以下内容适用于文档中的stack_traceJSON 日志行字段：

字符串类型的工作方式与日志流选择器中使用的 Prometheus 标签匹配器完全相同。这意味着您可以使用相同的操作（=，！=，=〜，！〜）。

来源：标签过滤表达式

以下似乎有效，但似乎很尴尬：

{host=~"test1-myservice-.*"} | json 
| logger_name != "com.example.ExampleClass" 
| …

我正在 Grafana 上开发基于 Loki 的仪表板。我有一个用于在 Loki 跟踪日志中搜索文本的面板，当前查询如下：

{job="abc-service"}
|~ "searchTrace"
|json
|line_format "{if .trace_message}} Message: \t{{.trace_message}} {{end}}"

其中searchTrace是“文本框”类型的变量，供用户输入搜索文本。

我想包含另一个变量skipTestLog来跳过某些测试 cron 任务创建的日志。skipTestLog是两个选项的自定义变量：Yes,No.

假设测试 cron 任务创建的日志包含解析器后面CronTest字段中的文本，是否有任何方法可以根据所选的值将其过滤掉？trace_messagejsonskipTestLog

我有一个带有 UserID= 标签的日志流。我正在尝试计算一小时内唯一用户的数量。

这是我的日志流的示例：

ts=2022-09-16T10:52:54.21344541Z level=INFO UserID=65166 elapsed=2.364015ms
ts=2022-09-16T10:52:51.580617785Z level=INFO UserID=24413 elapsed=2.324235ms
ts=2022-09-16T10:52:48.947248244Z level=INFO UserID=65166 elapsed=2.818146ms
ts=2022-09-16T10:52:41.51854716Z level=INFO UserID=24413 elapsed=2.633352ms
ts=2022-09-16T10:51:14.584272582Z level=INFO UserID=24413 elapsed=2.04884ms
ts=2022-09-16T10:51:14.45564065Z level=INFO UserID=65166 elapsed=4.889566ms

我设法实现的最接近的事情是计算每个用户的请求数量，但我只需要知道给定时间范围内唯一用户的数量。这是我所拥有的：

count(count_over_time({app="app"} | logfmt [1h])) by (UserID)

我有来自存储库的日志到grafana（loki）。我正在尝试获取给定持续时间（例如 6 小时）内的存储库数量（具有 msg="Repository finish"）

我使用了这些查询，但它们没有提供任何结果

sum by (repository) (sum_over_time ({job=~"$cronjob", job_name=~"$job"} | json  | durationMs != "" | unwrap durationMs | __error__="" [6h] ))

对于此查询也是如此：

sum by (repository) (sum_over_time ({job=~"$cronjob", job_name=~"$job"} | json  | durationMs != "" | unwrap time| __error__="" [6h] ))

{job=~"$cronjob", job_name=~"$job"} | json | msg="Repository finished"我的日志在执行后看起来像这样

{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3216,"msg":"Repository finished","time":"2022-08-09T12:00:25.580Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3558,"msg":"Repository finished","time":"2022-08-09T11:00:12.767Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3036,"msg":"Repository finished","time":"2022-08-09T10:01:30.224Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":8,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3275,"msg":"Repository finished","time":"2022-08-09T09:00:31.077Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":8,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3233,"msg":"Repository finished","time":"2022-08-09T08:00:18.020Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3581,"msg":"Repository finished","time":"2022-08-09T07:00:28.657Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":7,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3592,"msg":"Repository finished","time":"2022-08-09T06:00:19.073Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":10,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":8509,"msg":"Repository finished","time":"2022-08-09T05:00:34.047Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":8,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3730,"msg":"Repository finished","time":"2022-08-09T04:00:22.514Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3876,"msg":"Repository finished","time":"2022-08-09T03:00:42.023Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":10,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3754,"msg":"Repository finished","time":"2022-08-09T02:00:29.661Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":10,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3360,"msg":"Repository finished","time":"2022-08-09T01:00:17.274Z","v":0}