如何检查过滤器插件中Logstash管道中的空值？

Question

我有一个通过 Logstash http 插件将数据发送到我的 Logstash 管道的源。

发送的数据模型如下：

{
  "myArray": [
    {
      "myGrocery": {
        "myId": "aString",
        "apple": "aString",
        "banana": aNumber,
        "vegetable": {
          "garlic": "aString",
          "onion": "aString",
          "mushroom": "aString",    
        }
      }
    }
  ]
}

Logstash 中我的过滤器插件：

filter {

    split {
        field => "myArray"
        add_field => {
            "my_id" => "%{[myArray][myGrocery][myId]}"
            "apple" => "%{[myArray][myGrocery][apple]}"
            "banana" => "%{[myArray][myGrocery][banana]}"           
            "vegetable" => "%{[myArray][myGrocery][vegetable]}"
        }
        remove_field => "myArray"
    }

    json {
        source => "[vegetable]"
        target => "[vegetable]"
    }

    mutate {
        rename => { "[vegetable][garlic]" => "[vegetable][myNewGarlic]" }
        rename => { "[vegetable][onion]" => "[vegetable][myNewOnion]" }
        rename => { "[vegetable][mushroom]" => "[vegetable][myNewMushroom]" }
    }

    mutate {
        remove_field => [ "host", "headers", "@version", "url", "event", "user_agent", "http" ]
    }

}

一切正常，但如果香蕉作为 null 发送，我想显示 “null” 并将其保存为日志消息 “banana is null” 但我不想删除整个信息。结果应该是这样的：

"myId": "aString",
"apple": "aString",
"banana": "null",
"myNewGarlic" : "aString",
"myNewOnion": "aString",
"myNewMushroom": "aString",

可以这样做吗？又如何？

Answer 1

您可以使用 logstash if condition 来检查

banana

字段是否具有某些值。如果没有，您可以使用

mutate

过滤器添加/编辑/更新字段值。

input {}
filter {
  if ! [banana]  {
    mutate { update => { "banana" => "null" } }
    mutate { add_tag => "banana is null" }
    }
  }
output {}

注意：您还可以在 Elasticsearch 中检查null value处理器。