logstash 输出配置文件

Question

logstash kafka配置文件中，如果输出中有连接失败，就会阻塞。如何保证一个连接失败，其余连接正常运行

input{
stdin{}
}
output {
kafka {
bootstrap_servers =\> ":9091"
security_protocol =\> ""
ssl_key_password =\> ""
ssl_keystore_password =\> ""
ssl_truststore_password =\> ""
ssl_keystore_location =\> "/tmp"
ssl_truststore_location =\> "/tmp"
topic_id =\> "test"
ssl_endpoint_identification_algorithm =\> ""
}
stdout {}
}

我想使用多个KafKa连接成功，而不受单个连接失败的影响

Answer 1

首先，确保Kafka Brokers服务端口已启动并运行。 正如您在请求中提到的，您的情况为 9091。 远程登录 kafka_server1 9091 远程登录 kafka_server2 9091 远程登录 kafka_server3 9091 如果没有则重启kafka服务

然后按照下一个格式在 Logstash 输出中设置所有代理 格式为 host1:port1,host2:port2，列表可以是代理的子集或指向代理子集的 VIP。

input {
  # Your input configuration, e.g., file, beats, etc.
}

filter {
  # Your filter configuration, if needed
}

output {
  kafka {
    #### List of Kafka bootstrap servers
    bootstrap_servers => "kafka_server1:9091,kafka_server2:9091,kafka_server3:9091"

    ####  Kafka topic to which Logstash will send messages
    topic_id => "your_kafka_topic"

    ####  Optional settings (you may need to adjust based on your Kafka setup)
    codec => "json"
    acks => "all"
  }
}