活动 NameNode 崩溃

Question

我需要一些帮助来解决根本原因。我怀疑这个问题可能与在 VMS 中运行的集群以及使用外部 Ceph 存储进行所有操作有关，但我需要确认。以下是我在崩溃之前看到的日志消息。

活动名称节点：

 2024-11-21 13:48:54,368 WARN  client.QuorumJournalManager (IPCLoggerChannel.java:call(404)) - Remote journal 192.168.105.23:8485 failed to write txns 140138-140138. Will try to write to this JN again after the next log roll.
    org.apache.hadoop.ipc.RemoteException(java.io.IOException): IPC's epoch 18 is less than the last promised epoch 19 ; journal id: odphbasedev
            at org.apache.hadoop.hdfs.qjournal.server.Journal.checkRequest(Journal.java:489)
            at org.apache.hadoop.hdfs.qjournal.server.Journal.checkWriteRequest(Journal.java:515)
            at org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:397)
            at org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:191)
            at org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:164)
            at org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:31924)
            at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:621)
            at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:589)
            at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:573)
            at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1227)
            at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1094)
            at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1017)
            at java.base/java.security.AccessController.doPrivileged(Native Method)
            at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
            at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3048)

期刊节点：

2024-11-21 13:48:56,019 INFO  ipc.Server (Server.java:logException(3109)) - IPC Server handler 0 on default port 8485, call Call#19984 Retry#0 org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocol.journal from hbase-dev-master02.sgdctroy.net:53826 / 192.168.105.22:53826
java.io.IOException: IPC's epoch 18 is less than the last promised epoch 19 ; journal id: odphbasedev
        at org.apache.hadoop.hdfs.qjournal.server.Journal.checkRequest(Journal.java:489)
        at org.apache.hadoop.hdfs.qjournal.server.Journal.checkWriteRequest(Journal.java:515)
        at org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:397)
        at org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:191)
        at org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:164)
        at org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:31924)
        at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:621)
        at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:589)
        at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:573)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1227)
        at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1094)
        at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1017)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)

Answer 1

只是尝试将其用作云