apache-kafka-connect 相关问题

the抛出了一个未被发现且无法恢复的例外。任务被杀死，直到手动重新启动才能恢复

2025-03-07 07:59:34,574 ERROR MongoDB|dummy|task0 Error while attempting to Checking change stream: Timed out while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=REPLICA_SET, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}] [io.debezium.connector.mongodb.connection.MongoDbConnections] com.mongodb.MongoTimeoutException: Timed out while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=REPLICA_SET, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}] at com.mongodb.internal.connection.BaseCluster.logAndThrowTimeoutException(BaseCluster.java:427) at com.mongodb.internal.connection.BaseCluster.lambda$selectServer$0(BaseCluster.java:154) at com.mongodb.internal.time.Timeout.lambda$onExistsAndExpired$16(Timeout.java:236) at com.mongodb.internal.time.Timeout.lambda$run$10(Timeout.java:201) at com.mongodb.internal.time.TimePoint.checkedCall(TimePoint.java:98) at com.mongodb.internal.time.Timeout.call(Timeout.java:174) at com.mongodb.internal.time.Timeout.run(Timeout.java:194) at com.mongodb.internal.time.Timeout.onExistsAndExpired(Timeout.java:233) at com.mongodb.internal.time.Timeout.onExpired(Timeout.java:226) at com.mongodb.internal.connection.BaseCluster.selectServer(BaseCluster.java:153) at com.mongodb.internal.connection.AbstractMultiServerCluster.selectServer(AbstractMultiServerCluster.java:59) at com.mongodb.internal.binding.ClusterBinding.getReadConnectionSource(ClusterBinding.java:82) at com.mongodb.client.internal.ClientSessionBinding.getConnectionSource(ClientSessionBinding.java:108) at com.mongodb.client.internal.ClientSessionBinding.getReadConnectionSource(ClientSessionBinding.java:88) at com.mongodb.internal.operation.SyncOperationHelper.withSuppliedResource(SyncOperationHelper.java:148) at com.mongodb.internal.operation.SyncOperationHelper.withSourceAndConnection(SyncOperationHelper.java:129) at com.mongodb.internal.operation.SyncOperationHelper.lambda$executeRetryableRead$4(SyncOperationHelper.java:195) at com.mongodb.internal.operation.SyncOperationHelper.lambda$decorateReadWithRetries$13(SyncOperationHelper.java:317) at com.mongodb.internal.async.function.RetryingSyncSupplier.get(RetryingSyncSupplier.java:67) at com.mongodb.internal.operation.SyncOperationHelper.executeRetryableRead(SyncOperationHelper.java:201) at com.mongodb.internal.operation.SyncOperationHelper.executeRetryableRead(SyncOperationHelper.java:180) at com.mongodb.internal.operation.AggregateOperationImpl.execute(AggregateOperationImpl.java:190) at com.mongodb.internal.operation.ChangeStreamOperation.execute(ChangeStreamOperation.java:198) at com.mongodb.internal.operation.ChangeStreamOperation.execute(ChangeStreamOperation.java:54) at com.mongodb.client.internal.MongoClusterImpl$OperationExecutorImpl.execute(MongoClusterImpl.java:358) at com.mongodb.client.internal.ChangeStreamIterableImpl.execute(ChangeStreamIterableImpl.java:223) at com.mongodb.client.internal.ChangeStreamIterableImpl.cursor(ChangeStreamIterableImpl.java:193) at io.debezium.connector.mongodb.connection.MongoDbConnection.lambda$isValidResumeToken$10(MongoDbConnection.java:219) at io.debezium.connector.mongodb.connection.MongoDbConnection.execute(MongoDbConnection.java:105) at io.debezium.connector.mongodb.connection.MongoDbConnection.isValidResumeToken(MongoDbConnection.java:215) at io.debezium.connector.mongodb.connection.MongoDbConnection.validateLogPosition(MongoDbConnection.java:205) at io.debezium.connector.mongodb.MongoDbConnectorTask.validate(MongoDbConnectorTask.java:292) at io.debezium.connector.mongodb.MongoDbConnectorTask.start(MongoDbConnectorTask.java:137) at io.debezium.connector.common.BaseSourceTask.startIfNeededAndPossible(BaseSourceTask.java:403) at io.debezium.connector.common.BaseSourceTask.poll(BaseSourceTask.java:313) at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.poll(AbstractWorkerSourceTask.java:466) at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.execute(AbstractWorkerSourceTask.java:354) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:226) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:281) at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:79) at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:238) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) 2025-03-07 07:59:34,575 ERROR || WorkerSourceTask{id=mongo-connector-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted [org.apache.kafka.connect.runtime.WorkerTask] io.debezium.DebeziumException: Error while attempting to Checking change stream at io.debezium.connector.mongodb.connection.MongoDbConnections.lambda$eventSourcingErrorHandler$1(MongoDbConnections.java:53) at io.debezium.connector.mongodb.connection.MongoDbConnection.execute(MongoDbConnection.java:111) at io.debezium.connector.mongodb.connection.MongoDbConnection.isValidResumeToken(MongoDbConnection.java:215) at io.debezium.connector.mongodb.connection.MongoDbConnection.validateLogPosition(MongoDbConnection.java:205) at io.debezium.connector.mongodb.MongoDbConnectorTask.validate(MongoDbConnectorTask.java:292) at io.debezium.connector.mongodb.MongoDbConnectorTask.start(MongoDbConnectorTask.java:137) at io.debezium.connector.common.BaseSourceTask.startIfNeededAndPossible(BaseSourceTask.java:403) at io.debezium.connector.common.BaseSourceTask.poll(BaseSourceTask.java:313) at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.poll(AbstractWorkerSourceTask.java:466) at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.execute(AbstractWorkerSourceTask.java:354) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:226) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:281) at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:79) at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:238) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: com.mongodb.MongoTimeoutException: Timed out while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=REPLICA_SET, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}] at com.mongodb.internal.connection.BaseCluster.logAndThrowTimeoutException(BaseCluster.java:427) at com.mongodb.internal.connection.BaseCluster.lambda$selectServer$0(BaseCluster.java:154) at com.mongodb.internal.time.Timeout.lambda$onExistsAndExpired$16(Timeout.java:236) at com.mongodb.internal.time.Timeout.lambda$run$10(Timeout.java:201) at com.mongodb.internal.time.TimePoint.checkedCall(TimePoint.java:98) at com.mongodb.internal.time.Timeout.call(Timeout.java:174) at com.mongodb.internal.time.Timeout.run(Timeout.java:194) at com.mongodb.internal.time.Timeout.onExistsAndExpired(Timeout.java:233) at com.mongodb.internal.time.Timeout.onExpired(Timeout.java:226) at com.mongodb.internal.connection.BaseCluster.selectServer(BaseCluster.java:153) at com.mongodb.internal.connection.AbstractMultiServerCluster.selectServer(AbstractMultiServerCluster.java:59) at com.mongodb.internal.binding.ClusterBinding.getReadConnectionSource(ClusterBinding.java:82) at com.mongodb.client.internal.ClientSessionBinding.getConnectionSource(ClientSessionBinding.java:108) at com.mongodb.client.internal.ClientSessionBinding.getReadConnectionSource(ClientSessionBinding.java:88) at com.mongodb.internal.operation.SyncOperationHelper.withSuppliedResource(SyncOperationHelper.java:148) at com.mongodb.internal.operation.SyncOperationHelper.withSourceAndConnection(SyncOperationHelper.java:129) at com.mongodb.internal.operation.SyncOperationHelper.lambda$executeRetryableRead$4(SyncOperationHelper.java:195) at com.mongodb.internal.operation.SyncOperationHelper.lambda$decorateReadWithRetries$13(SyncOperationHelper.java:317) at com.mongodb.internal.async.function.RetryingSyncSupplier.get(RetryingSyncSupplier.java:67) at com.mongodb.internal.operation.SyncOperationHelper.executeRetryableRead(SyncOperationHelper.java:201) at com.mongodb.internal.operation.SyncOperationHelper.executeRetryableRead(SyncOperationHelper.java:180) at com.mongodb.internal.operation.AggregateOperationImpl.execute(AggregateOperationImpl.java:190) at com.mongodb.internal.operation.ChangeStreamOperation.execute(ChangeStreamOperation.java:198) at com.mongodb.internal.operation.ChangeStreamOperation.execute(ChangeStreamOperation.java:54) at com.mongodb.client.internal.MongoClusterImpl$OperationExecutorImpl.execute(MongoClusterImpl.java:358) at com.mongodb.client.internal.ChangeStreamIterableImpl.execute(ChangeStreamIterableImpl.java:223) at com.mongodb.client.internal.ChangeStreamIterableImpl.cursor(ChangeStreamIterableImpl.java:193) at io.debezium.connector.mongodb.connection.MongoDbConnection.lambda$isValidResumeToken$10(MongoDbConnection.java:219) at io.debezium.connector.mongodb.connection.MongoDbConnection.execute(MongoDbConnection.java:105) ... 17 more

可以可以使用多个Kafka设置？

在Helidon SE 4.1.6中，如何使用Kafka生产商将数据发送到特定分区

我想使用Helidon SE 4.1.6并使用生产者将数据与Apache Kafka的特定分区一起生产。 细节 ： 我已经通过https://helidon.io/docs/latest/se/reactive-messaging#

KafkaConnect JDBC源连接器无法通过SMT

我已经使用io.confluent.connect.json.jsonschemaconconverter编写了一个用于连接器的SMT。同一过程正常工作的地方。

关于任务的灌注。max具有自定义kafka接收器中的单分区主题。 我已经实现了一个自定义的Kafka接收器连接器，以消耗17个主题的数据，每个都有1个分区。从我的理解来看，最大任务的数量应等于每个分区的数量...

否则，在阅读Mickael Maison和Kate Stanley的《 Kafka Connect：Build and of Data Pipelines》一书时，我遇到了本段：

有一种方法可以使用MSK-Connect API更新连接器配置？

如何确保NEO4J交易更改被以单个KAFKA消息发送在Spring Boot应用程序中

关于Strimzi的Kafka Connection中的有关连接器插件版本的问题 我正在使用Kafka Connect的功能来构建图像，我有一些问题。在所有示例中，我都使用了Strimzi操作员版本0.45.0。 当我创建一个strimzi kafka connect ins ...

如何在kafka connect运行时升级码头客户端依赖性？

Kafka与Vireguard集成：不可折断的元地址0.0.0.0

写入 Kafka 主题时在消息标头中添加记录偏移量和位置

我是 Kafka 和 Kafka Connect 的新手，我正在使用 File Pulse 源连接器并从文件中读取数据并将其写入 Kafka 主题。 标头包含类似这样的元数据...

Kafka Connect - S3 Sink 不从主题消费

我有一个 S3 接收器连接器，它从 Kafka 主题读取数据并溢出到 S3。连接器不消耗该主题。 这是连接器配置： { “名称”：“我的连接器&

Kafka 连接器在 MSK Connect 3.7.x 中每 6 分钟重新启动一次

我正在尝试将 MSK Connect (AWS) 中的 Kafka 连接器从 2.7.1 升级到 3.7.x。从功能上来说，升级后它们似乎工作得很好。但在 CloudWatch 中观察这些连接器的日志后，我...

Confluence 连接器任务失败

启动docker容器（cp-kafka-connect-base：7.0.1）并安装自我管理连接器（debezium-connector-mysql：最新）。 Docker 容器运行良好。 当我尝试配置

如何将 Oracle CDC Source Connector 生成的 CDC 事件提取到 Snowflake 中

我们当前的管道遵循与此处概述类似的结构，只是我们从 Oracle 中提取事件并将其推送到雪花。流程是这样的： 汇合...

Kafka Connect REST 接口“PUT /connectors/(string: name)/config”返回错误代码 500

我有一个分布式模式的 3 节点 kafka-connect 工作集群，带有正在运行的 s3 接收器连接器。要在运行时更新连接器的配置，我运行以下命令： 卷曲-X P...

kafka connect 使用的默认 acks 值是多少？

我们正在使用 Debezium 源连接器。向 kafka 生成消息时默认的 acks 值 [1,0,all] 是多少？ 生产者配置 - https://docs.confluence.io/current/installation/configuration/

Kafka connect：kafka connect 使用的默认 acks 值是多少？

我们正在使用 Debezium 源连接器。向 kafka 生成消息时默认的 acks 值 [1,0,all] 是多少？ 生产者配置 - https://docs.confluence.io/current/installation/configuration/

为什么 Debezium Mongo Source Kafka Connector 生成字符串 `after` 字段而不是 Json 对象？

这是我正在使用的配置 - { “名称”：“mongo-debezium-连接器”， “配置”：{ "connector.class": "io.debezium.connector.mongodb.

Kafka 连接不断重新平衡

我正在部署一个kafka连接集群，由4个使用docker swarm的工作人员组成。在初始部署时存在一些情况（当环境中不存在其他 kafka 连接集群时......