Grafana Loki 解析和聚合标签 - LogQL

Question

我有来自存储库的日志到grafana（loki）。我正在尝试获取给定持续时间（例如 6 小时）内的存储库数量（具有 msg="Repository finish"）

我使用了这些查询，但它们没有提供任何结果

sum by (repository) (sum_over_time ({job=~"$cronjob", job_name=~"$job"} | json  | durationMs != "" | unwrap durationMs | __error__="" [6h] ))

对于此查询也是如此：

sum by (repository) (sum_over_time ({job=~"$cronjob", job_name=~"$job"} | json  | durationMs != "" | unwrap time| __error__="" [6h] ))

{job=~"$cronjob", job_name=~"$job"} | json | msg="Repository finished"我的日志在执行后看起来像这样

{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3216,"msg":"Repository finished","time":"2022-08-09T12:00:25.580Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3558,"msg":"Repository finished","time":"2022-08-09T11:00:12.767Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3036,"msg":"Repository finished","time":"2022-08-09T10:01:30.224Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":8,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3275,"msg":"Repository finished","time":"2022-08-09T09:00:31.077Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":8,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3233,"msg":"Repository finished","time":"2022-08-09T08:00:18.020Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3581,"msg":"Repository finished","time":"2022-08-09T07:00:28.657Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":7,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3592,"msg":"Repository finished","time":"2022-08-09T06:00:19.073Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":10,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":8509,"msg":"Repository finished","time":"2022-08-09T05:00:34.047Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":8,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3730,"msg":"Repository finished","time":"2022-08-09T04:00:22.514Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":9,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3876,"msg":"Repository finished","time":"2022-08-09T03:00:42.023Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":10,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3754,"msg":"Repository finished","time":"2022-08-09T02:00:29.661Z","v":0}
{"name":"reno","hostname":"reno-01234","pid":10,"level":30,"logContext":"abc","repository":"abc/bmw/gmc","durationMs":3360,"msg":"Repository finished","time":"2022-08-09T01:00:17.274Z","v":0}

Answer 1

虽然我没有直接回答你的问题，但我确实有类似的东西可以帮助任何正在寻找答案的人。

\n

我决定提供答案，因为事实证明，在 99% 的搜索案例中，这只是hitting a wall- 没有诸如此处所示的问题的示例或答案。

\n\n

我的目标是count number of entries with given string, grouped by days or months并且我已经设法得到我想要的东西。

\n

看：

\n

\n

首先是日志示例：

\n

[2022-03-12T13:42:56.782023+00:00] NotifierProxyLogger.INFO: Now calling api:  {"calledMethod":"insertDiscordMessage","baseUrl":"http://...:8902","requestUri":"/api/external/discord/insert-message","dataBag":{"webhookName":"NewsFeed","messageContent":"\\n**Title: ** Katarzyna Pe\xc5\x82czy\xc5\x84ska-Na\xc5\x82\xc4\x99cz go\xc5\x9bcini\xc4\x85 Jacka G\xc4\x85dka w Porannej Rozmowie Gazeta.pl [NA \xc5\xbbYWO],\\n**Description: ** W poniedzia\xc5\x82ek w Porannej Rozmowie Gazeta.pl go\xc5\x9bcimy Katarzyn\xc4\x99 P\xc5\x82eczy\xc5\x84sk\xc4\x85-Na\xc5\x82\xc4\x99cz, by\xc5\x82\xc4\x85 ambasadork\xc4\x99 Polski w Moskwie i cz\xc5\x82onkini\xc4\x99 Instytutu Strategie 2050. Rozmow\xc4\x99 prowadzi Jacek G\xc4\x85dek.\\n**Link: ** https://wiadomosci.gazeta.pl/wiadomosci/7,114884,28135035,katarzyna-pelczynska-nalecz-goscinia-jacka-gadka-w-porannej.html\\n        ","messageTitle":"**[NEWS]** Keywords: (wezwanie armia) OR (pob\xc3\xb3r wojskowy) OR (atak na polsk\xc4\x99)","source":"NewsFeed"}} []\n

\n

[2022-03-19T12:00:03.372023+00:00] NotifierProxyLogger.INFO: Now calling api:  {"calledMethod":"insertDiscordMessage","baseUrl":"http://...","requestUri":"/api/external/discord/insert-message","dataBag":{"webhookName":"NewsFeed","messageContent":"\\n**Title: ** Polska mocarstwem \xc5\x9bwiatowym,\\n**Description: ** Wojna toczy si\xc4\x99 ju\xc5\xbc 20 dzie\xc5\x84, cho\xc4\x87 nie wiadomo dlaczego.\\n**Link: ** https://wolnemedia.net/polska-mocarstwem-swiatowym/\\n        ","messageTitle":"**[NEWS]** Keywords: (wezwanie armia) OR (pob\xc3\xb3r wojskowy) OR (atak na polsk\xc4\x99) OR (atak nato) OR (wojna polska)","source":"NewsFeed"}} []\n

\n\n

每天

\n

\n
• 询问：count_over_time({filename="/var/log/external/NAME/news-feed/notifier-proxy-logger-bridge.log"} |= `insertDiscordMessage` | regexp `(?P<timestamp>[0-9]{4}-[0-9]{1,2}-[0-9]{1,2})` [$__interval])
\n
• 查询选项:\n
  \n
  • 传奇：{{timestamp}}
  \n
  \n
\n
• 可视化：Bar gauge
\n
• 可视化配置：\n
  \n
  • 方向：Horizontal
  \n
  • 计算：Total
  \n
  \n
\n

\n\n

2023 年每天

\n

\n
• 询问：count_over_time({filename="/var/log/external/NAME/news-feed/notifier-proxy-logger-bridge.log"} |= `insertDiscordMessage` |= `2023` | regexp `(\\[)(?P<timestamp>2023-[0-9]{1,2}-[0-9]{2})(T)` | timestamp != `` [$__interval])
\n
• 查询选项:\n
  \n
  • 传奇：{{timestamp}}
  \n
  \n
\n
• 可视化：Bar gauge
\n
• 可视化配置：\n
  \n
  • 方向：Vertical
  \n
  • 计算：Total
  \n
  \n
\n

\n\n

2023 年每天 + 总金额

\n

\n
• 查询1：count_over_time({filename="/var/log/external/NAME/news-feed/notifier-proxy-logger-bridge.log"} |= `insertDiscordMessage` |= `2023` | regexp `(\\[)(?P<timestamp>2023-[0-9]{1,2}-[0-9]{2})(T)` | timestamp != `` [$__interval])
\n
• 查询选项:\n
  \n
  • 传奇：{{timestamp}}
  \n
  \n
\n
• 查询2：sum(count_over_time({filename="/var/log/external/NAME/news-feed/notifier-proxy-logger-bridge.log"} |= `insertDiscordMessage` |= `2023` | regexp `(\\[)(?P<timestamp>2023-[0-9]{1,2}-[0-9]{2})(T)` | timestamp != `` [$__interval]))
\n
• 查询选项:\n
  \n
  • 传奇：Total
  \n
  \n
\n
• 可视化：Bar gauge
\n
• 可视化配置：\n
  \n
  • 方向：Vertical
  \n
  • 计算：Total
  \n
  \n
\n

\n