我有一个问题可以用promtail解析json日志,请有人帮我。我尝试了许多配置,但是不解析时间戳或其他标签。
日志条目:
{timestamp=2019-10-25T15:25:41.041-03, level=WARN, thread=http-nio-0.0.0.0-8080-exec-2, mdc={handler=MediaController, ctxCli=127.0.0.1, ctxId=FdD3FVqBAb0}, logger=br.com.brainyit.cdn.vbox.
controller.MediaController, message=[http://localhost:8080/media/sdf],c[500],t[4],l[null], context=default}
promtail-config.yml
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://localhost:3100/loki/api/v1/push
scrape_configs:
- job_name: vbox-main
static_configs:
- targets:
- localhost
labels:
job: vbox
appender: main
__path__: /var/log/vbox/main.log
pipeline_stages:
- json:
expressions:
timestamp: timestamp
message: message
context: context
level: level
timestamp:
source: timestamp
format: RFC3339Nano
labels:
context:
level:
output:
source: message
我已经尝试使用Java SpringBoot应用程序(通过Logstash logback编码器将日志生成为json格式的文件生成日志),并且可以使用。
示例日志行由应用程序生成:
{"timestamp":"2020-06-06T01:00:30.840+02:00","version":1,"message":"Started ApiApplication in 1.431 seconds (JVM running for 6.824)","logger_name":"com.github.pnowy.spring.api.ApiApplication","thread_name":"main","level":"INFO","level_value":20000}
prometail配置:
# Promtail Server Config
server:
http_listen_port: 9080
grpc_listen_port: 0
# Positions
positions:
filename: /tmp/positions.yaml
clients:
- url: http://localhost:3100/loki/api/v1/push
scrape_configs:
- job_name: springboot
pipeline_stages:
- json:
expressions:
level: level
message: message
timestamp: timestamp
logger_name: logger_name
stack_trace: stack_trace
thread_name: thread_name
- labels:
level:
- template:
source: new_key
template: 'logger={{ .logger_name }} threadName={{ .thread_name }} | {{ or .message .stack_trace }}'
- output:
source: new_key
static_configs:
- targets:
- localhost
labels:
job: applogs
__path__: /Users/przemek/tools/promtail/*.log
请注意,首先通过Go模板将输出(日志文本)配置为new_key
,然后再将其设置为输出源。 logger={{ .logger_name }}
有助于识别在Loki视图上解析的字段(但是,如何为应用程序配置字段由您个人决定)。
在这里您会找到有关整个过程的非常不错的文档:https://github.com/grafana/loki/blob/master/docs/clients/promtail/pipelines.md
该示例在Loki和Promtail的v1.5.0发行版上运行。
关于时间戳的部分在这里:https://github.com/grafana/loki/blob/master/docs/clients/promtail/stages/timestamp.md带有示例-我已经测试了它,也没有发现任何问题。希望能有所帮助。
当然是json配置部分:https://github.com/grafana/loki/blob/master/docs/clients/promtail/stages/json.md
loki上的结果: