为什么 JanusGraph 会发现过期的 KeyColumn 声明？

问题描述 投票：0回答：1

问题

是什么导致 JanusGraph 在全面重试时抛出这么多警告？ JanusGraph 似乎找到了据称丢失的顶点，如果有延迟，这很好；
我只想在顶点删除后添加一个新顶点。

[WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn

Temporary exception during backend operation [CacheMutation]. Attempting backoff retry

繁殖

步骤

  1. 创建并启动一个 Cassandra [Docker 容器] 
    docker run --name jg-cassandra -d -e CASSANDRA_START_RPC=true -p 9160:9160 -p 9042:9042 -p 7199:7199 -p 7001:7001 -p 7000:7000 cassandra:3.11
  2. 将大型（或至少非平凡的）数据集加载到 JanusGraph 中
    1. “中型”数据集样本是ICIJ Offshore Dataleaks
    2. 中等意味着比某人可以硬编码不同和“随机”数据更大的数据集
  3. 创建并运行这个 Java + Maven 项目（代码如下）
  4. 结果
    1. 预期：几乎没有错误或警告，每个实例限制为 1，然后继续
    2. 实际上：抛出一堵警告墙，每次将 k 值增加 100 
      Discarded expired claim on KeyColumn [k=0x0x7800000000008080, c=0x0x02] with timestamp 2023-05-09T20:16:08.988543Z

日志

2023-05-10 09:23:19,141 [INFO] [c.d.o.d.i.c.ContactPoints.main] ::   Contact point localhost:9042 resolves to multiple addresses, will use them all ([localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1])
2023-05-10 09:23:19,226 [INFO] [c.d.o.d.i.c.DefaultMavenCoordinates.main] ::     DataStax Java driver for Apache Cassandra(R) (com.datastax.oss:java-driver-core) version 4.15.0
2023-05-10 09:23:19,726 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] ::   Using native clock for microsecond precision
2023-05-10 09:23:19,987 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] ::     [JanusGraph Session|default] You specified datacenter1 as the local DC, but some contact points are from a different DC: Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=37423099)=null; please provide the correct local DC, or check your contact points
2023-05-10 09:23:20,231 [INFO] [o.j.g.i.UniqueInstanceIdRetriever.main] ::   Generated unique-instance-id=c0a8563c404-rmt-lap-win201
2023-05-10 09:23:20,252 [INFO] [c.d.o.d.i.c.ContactPoints.main] ::   Contact point localhost:9042 resolves to multiple addresses, will use them all ([localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1])
2023-05-10 09:23:20,297 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] ::   Using native clock for microsecond precision
2023-05-10 09:23:20,346 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] ::     [JanusGraph Session|default] You specified datacenter1 as the local DC, but some contact points are from a different DC: Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=1ec16fcd)=null; please provide the correct local DC, or check your contact points
2023-05-10 09:23:20,362 [INFO] [o.j.d.c.ExecutorServiceBuilder.main] ::  Initiated fixed thread pool of size 40
2023-05-10 09:23:20,469 [INFO] [o.j.g.d.StandardJanusGraph.main] ::  Gremlin script evaluation is disabled
2023-05-10 09:23:20,494 [INFO] [o.j.d.l.k.KCVSLog.main] ::   Loaded unidentified ReadMarker start time 2023-05-10T14:23:20.494021Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@f1d0004
2023-05-10 09:23:20,549 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::    Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:23:21,835 [INFO] [Main.main] ::    g.V().count().next():  70005
2023-05-10 09:23:21,839 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::    Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:26:13,733 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::    Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-10 09:26:15,427 [INFO] [Main.main] ::    g.V().count().next():  0
2023-05-10 09:33:40,131 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Lock write succeeded but took too long: duration PT0.197258S exceeded limit PT0.1S
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn [k=0x0x7800000000008080, c=0x0x02] with timestamp 2023-05-09T20:16:07.308532Z
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn [k=0x0x7800000000008180, c=0x0x02] with timestamp 2023-05-09T20:16:08.146591Z
2023-05-10 09:38:03,812 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn [k=0x0x7800000000008280, c=0x0x02] with timestamp 2023-05-09T20:16:08.988543Z

...

2023-05-10 09:38:03,876 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-10T14:28:09.380600Z
2023-05-10 09:38:03,876 [INFO] [o.j.d.u.BackendOperation.main] ::    Temporary exception during backend operation [CacheMutation]. Attempting backoff retry.
org.janusgraph.diskstorage.locking.consistentkey.ExpiredLockException: Expired lock on KeyColumn [k=0x0x7800000000018080, c=0x0x02]: lock timestamp 2023-05-10T14:28:09.380600Z Micros is older than storage.lock.expiry-time=PT5M
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:472) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:131) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:348) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:178) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:157) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:77) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:99) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:96) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:66) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:52) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:96) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:147) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:210) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:147) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:917) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1558) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:322) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) [gremlin-core-3.6.2.jar:3.6.2]
    at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.commit(JanusGraphBlueprintsGraph.java:300) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at Test6.main(Test6.java:28) [test-classes/:?]
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:09:05.345276Z
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:14:34.917340Z
2023-05-10 09:38:03,925 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-09T20:21:07.864503Z
2023-05-10 09:38:03,933 [WARN] [o.j.d.l.c.ConsistentKeyLocker.main] ::   Discarded expired claim on KeyColumn [k=0x0x7800000000018080, c=0x0x02] with timestamp 2023-05-10T14:28:09.380600Z

...

2023-05-10 09:38:45,394 [INFO] [o.j.d.u.BackendOperation.main] ::    Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. ...

Process finished with exit code -1

代码

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.janusgraph.core.JanusGraph;
import org.janusgraph.core.JanusGraphFactory;
import org.janusgraph.core.JanusGraphVertex;
import org.janusgraph.core.PropertyKey;
import org.janusgraph.core.schema.JanusGraphManagement;

public class Test6 {
    private static final Logger logger = LogManager.getLogger(Main.class);

    public static void main(String[] args) {
        JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "cql").set("storage.hostname", "localhost:9042").open();
        GraphTraversalSource g = janusGraph.traversal();
        logger.info("g.V().count().next():\t" + g.V().count().next());
        g.V().drop().iterate();
        logger.info("g.V().count().next():\t" + g.V().count().next());
        JanusGraphManagement janusGraphManagement = janusGraph.openManagement();
        PropertyKey propertyKey = janusGraphManagement.getOrCreatePropertyKey("_id");
        if (!janusGraphManagement.containsGraphIndex("_id"))
            janusGraphManagement.buildIndex("_id", Vertex.class).addKey(propertyKey).buildCompositeIndex();
        janusGraphManagement.commit();
        JanusGraphVertex janusGraphVertex = janusGraph.addVertex();
        janusGraphVertex.property("test", "test1");
        janusGraph.tx().commit();
        janusGraph.close();
    }
}

资源

  1. Maven 3.8.1
  2. Java 11.0.19 (corretto-11)
  3. JanusGraph 1.0.0-20230504-014643.988c094
  4. Windows 10
  5. 码头工人
  6. Cassandra:latest [容器]
  7. ICIJ 离岸数据泄露
cassandra-3.0 janusgraph tinkerpop3 java-17 gremlin-server
1个回答
0
投票

JanusGraph 不会丢弃它们。

因为JanusGraph的服务器是Cassandra；这个实例不是“JanusGraph Server”

in-memory
或
local-distribution
由你明确Stephen Mallette

Gremlin 服务器确实会自动提交事务，但您没有连接到 Gremlin 服务器。您直接连接到 JanusGraph 实例，因此 Gremlin Server 没有做任何事情来管理交易。只是一个建议，因为您似乎以实时方式工作并在进行时提出问题，stackoverflow 确实不适合对话帮助。我强烈建议您将您的对话带到 JanusGraph 和/或 TinkerPop discord 服务器以获得此类支持。你可以在项目主页上找到 discord 的链接。 - Stephen M.

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.