org.xerial.snappy.SnappyError：[FAILED_TO_LOAD_NATIVE_LIBRARY] 找不到 os.name=Mac 和 os.arch=aarch64 的本机库

Question

我正在构建一个cdc管道，通过maxwell读取mysql binlog并将它们放入kafka中，我的压缩类型在maxwell配置中是snappy。但是在我的spring项目的消费者端，我收到了这个错误。

org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64
at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:361) ~[snappy-java-1.1.7.7.jar:1.1.7.7]
at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:195) ~[snappy-java-1.1.7.7.jar:1.1.7.7]
at org.xerial.snappy.SnappyLoader.loadSnappyApi(SnappyLoader.java:167) ~[snappy-java-1.1.7.7.jar:1.1.7.7]
at org.xerial.snappy.Snappy.init(Snappy.java:69) ~[snappy-java-1.1.7.7.jar:1.1.7.7]
at org.xerial.snappy.Snappy.<clinit>(Snappy.java:46) ~[snappy-java-1.1.7.7.jar:1.1.7.7]
at org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:435) ~[snappy-java-1.1.7.7.jar:1.1.7.7]
at org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:466) ~[snappy-java-1.1.7.7.jar:1.1.7.7]
at java.base/java.io.DataInputStream.readByte(DataInputStream.java:271) ~[na:na]
at org.apache.kafka.common.utils.ByteUtils.readUnsignedVarint(ByteUtils.java:170) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.common.utils.ByteUtils.readVarint(ByteUtils.java:205) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.common.record.DefaultRecord.readFrom(DefaultRecord.java:296) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.common.record.DefaultRecordBatch$2.doReadRecord(DefaultRecordBatch.java:278) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.common.record.DefaultRecordBatch$StreamRecordIterator.readNext(DefaultRecordBatch.java:617) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.common.record.DefaultRecordBatch$RecordIterator.next(DefaultRecordBatch.java:582) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.common.record.DefaultRecordBatch$RecordIterator.next(DefaultRecordBatch.java:551) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.nextFetchedRecord(Fetcher.java:1578) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1613) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.access$1700(Fetcher.java:1454) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:687) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:638) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1299) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1233) ~[kafka-clients-2.7.2.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1206) ~[kafka-clients-2.7.2.jar:na]

我的java版本“17.0.2”2022-01-18 LTS 并使用 Apple M1 芯片。

Answer 1

您需要将 snappy 版本更新到

1.1.8.4

或更高版本：

    <dependency>
      <groupId>org.xerial.snappy</groupId>
      <artifactId>snappy-java</artifactId>
      <version>1.1.8.4</version>
    </dependency>

来源： https://issues.apache.org/jira/browse/BEAM-11863?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=17318180#comment-17318180

Answer 2

我正在使用kafka客户端并遇到同样的错误。我通过下载 snappy-java jar 并将其放入 /libs 来修复它。

snappy-java 的一个版本可能在这里：https://repo1.maven.org/maven2/org/xerial/snappy/snappy-java/1.1.10.7/