将 kafdrop 与安全的 AWS MSK 代理连接

Question

我可以使用以下步骤将 kafka cli 生产者连接到使用 IAM 身份验证和授权保护的 AWS MSK 代理：

• 本地设置 AWS CLI
• 将 aws-msk-iam-auth-2.1.1-all.jar 放入 kafka 安装的 libs 文件夹中
• 运行以下命令

./kafka-console-producer.sh --broker-list "b-1.amazonaws.com:9098,b-2.amazonaws.com:9098" --producer.config /tmp/kafka.iam.client.properties --topic some-topic --property "parse.key=true" --property "key.separator=,"

使用 jar 文件或使用 docker 文件运行 kafdrop 时如何做同样的事情？

我尝试了以下命令，但无法启动 kafdrop 容器

docker run -d --platform linux/amd64 -p 9042:9000 \
    -v /tmp/kafka.iam.client.properties:/tmp/kafka.iam.client.properties:ro \
    -v /tmp/kafka.client.truststore.jks:/tmp/kafka.client.truststore.jks:ro \
    -v ~/.aws/credentials:/root/.aws/credentials:ro \
    -v /Users/ME/apps/kafka/extra-kafdrop-classes/aws-msk-iam-auth-2.1.0-all.jar:/kafdrop-4.0.1/lib/aws-msk-iam-auth-2.1.0-all.jar:ro \
    -e KAFKA_BROKERCONNECT="b-1.amazonaws.com:9098,b-2.amazonaws.com:9098" \
    -e KAFKA_PROPERTIES_FILE="/tmp/kafka.iam.client.properties" \
    obsidiandynamics/kafdrop

这是错误消息

2024-06-26 17:47:28 org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'messageInspector' defined in URL [jar:file:/kafdrop-4.0.1/kafdrop-4.0.1.jar!/BOOT-INF/classes!/kafdrop/service/MessageInspector.class]: Unsatisfied dependency expressed through constructor parameter 0: Error creating bean with name 'kafkaMonitorImpl' defined in URL [jar:file:/kafdrop-4.0.1/kafdrop-4.0.1.jar!/BOOT-INF/classes!/kafdrop/service/KafkaMonitorImpl.class]: Unsatisfied dependency expressed through constructor parameter 0: Error creating bean with name 'kafkaHighLevelConsumer': Invocation of init method failed

...

2024-06-26 17:47:28     at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
2024-06-26 17:47:28     at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
2024-06-26 17:47:28     at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
2024-06-26 17:47:28     at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
2024-06-26 17:47:28 Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'kafkaMonitorImpl' defined in URL [jar:file:/kafdrop-4.0.1/kafdrop-4.0.1.jar!/BOOT-INF/classes!/kafdrop/service/KafkaMonitorImpl.class]: Unsatisfied dependency expressed through constructor parameter 0: Error creating bean with name 'kafkaHighLevelConsumer': Invocation of init method failed

...
 
2024-06-26 17:47:28 Caused by: org.apache.kafka.common.config.ConfigException: Invalid value software.amazon.msk.auth.iam.IAMClientCallbackHandler for configuration sasl.client.callback.handler.class: Class software.amazon.msk.auth.iam.IAMClientCallbackHandler could not be found.

Answer 1

如果有人有解决方法，将会很有帮助。