Oracle Golden Gate Cassandra 处理程序

问题描述 投票：0回答：1

我有这种类型的配置：

  • Oracle GoldenGate 大数据版本 
    21.9.0.0.3
  • Cassandra Handler 
    4.17.0
    使用 OGG 通过 
    /DependencyDownloader/cassandra.sh
    • 直接提供的脚本下载

该流程使用此参数文件启动

SPECIALRUN
END RUNTIME
TARGETDB LIBFILE libggjava.so SET property=dirprm/inievo.props
REPORTCOUNT EVERY 1 MINUTES, RATE
GROUPTRANSOPS 1000
EXTFILE ${EXTFILE}
MAP U_ALLIN.*, TARGET U_ALLIN.*;

我使用此配置运行该过程

inievo.props


#The handler properties
gg.handler.cassandra.type=cassandra
gg.handler.cassandra.mode=op
gg.handler.cassandra.contactPoints=10.152.13.7,10.152.13.8,10.152.13.9
gg.handler.cassandra.ddlHandling=CREATE,ADD,DROP
gg.handler.cassandra.compressedUpdates=true
gg.handler.cassandra.cassandraMode=async
gg.handler.cassandra.consistencyLevel=LOCAL_QUORUM
gg.handler.cassandra.dataCenter=cas

#Regex properties
gg.schemareplaceregex=[$]
gg.schemareplacestring=_

#Javawriter properties
javawriter.stats.display=TRUE
javawriter.stats.full=TRUE
javawriter.bootoptions=-Xmx1G -Xms1G -Duser.timezone=UTC -Djava.class.path=.:ggjava/ggjava.jar:./dirprm

#Logging properties
gg.log=log4j
gg.log.level=INFO
gg.report.time=3600sec

gg.classpath=/shared/Middleware/GGPRE0T5/ggjava/lib/cassandra_4.17.0/*

我有两种类型的错误：

  1. Oracle GoldenGate 写入速度很快，我收到此错误。此错误不是由错误日志中报告的表引起的，因为有时重新启动同一进程不会导致错误再次发生。有时它出现在不同的表上，因此它不链接到表本身。为了降低写入速度，我注意到增加了 
    .props
    文件中的日志级别，例如从 
    INFO
    到 
    DEBUG
    。还有另一种更正确的方法来降低
    OGG
    侧的书写速度吗？
    =ERROR 2024-01-23 11:52:50.000560 [main] - Async statement execution on Cassandra failed for table [U_ALLIN.IVF_RISERVAPREMI] at position [-0000000000000000001].
    com.datastax.oss.driver.api.core.AllNodesFailedException: All 3 node(s) tried for the query failed (showing first 3 nodes, use getAllErrors() for more): Node(endPoint=10.152.13.9:9042, hostId=9b44e6db-1334-41f8-b073-73506d94e879, hashCode=
    2186c2ff): [com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.9:9042, hostId=9b44e6db-1334-41f8-b073-73506d94e879, hashCode=2186c2ff)], Node(endPoint=bdpre35cas.gruppoitas.loc
    al/10.152.13.8:9042, hostId=c993410a-4520-4b19-a84d-75bb857eff5a, hashCode=4d8e4253): [com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.8:9042, hostId=c993410a-4520-4b19-a84d
    -75bb857eff5a, hashCode=4d8e4253)], Node(endPoint=10.152.13.7:9042, hostId=083aba02-71b9-4bf3-a38f-c05045f89ef5, hashCode=21e99a7): [com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=bd
    pre34cas.gruppoitas.local/10.152.13.7:9042, hostId=083aba02-71b9-4bf3-a38f-c05045f89ef5, hashCode=21e99a7)]
            Suppressed: com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.9:9042, hostId=9b44e6db-1334-41f8-b073-73506d94e879, hashCode=2186c2ff)
            Suppressed: com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.8:9042, hostId=c993410a-4520-4b19-a84d-75bb857eff5a, hashCode=4d8e4253)
            Suppressed: com.datastax.oss.driver.api.core.NodeUnavailableException: No connection was available to Node(endPoint=10.152.13.7:9042, hostId=083aba02-71b9-4bf3-a38f-c05045f89ef5, hashCode=21e99a7)
    Exception in thread "main" oracle.goldengate.util.GGException: Error detected handling transaction commit event.
            at oracle.goldengate.datasource.UserExitDataSource.commitTransaction(UserExitDataSource.java:2480)
            at oracle.goldengate.datasource.UserExitDataSource.commitTx(UserExitDataSource.java:2081)
    
    Source Context :
      SourceModule            : [gglib.ggdal.adapter.java]
      SourceID                : [ggdal/Adapter/Java/JavaAdapter.cpp]
      SourceMethod            : [HandleJavaException]
      SourceLine              : [301]
      ThreadBacktrace         : [19] elements
                              : [/shared/Middleware/GGPRE0T5/libgglog.so(CMessageContext::AddThreadContext())]
                              : [/shared/Middleware/GGPRE0T5/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...))]
                              : [/shared/Middleware/GGPRE0T5/libgglog.so(_MSG_String(CSourceContext*, int, char const*, CMessageFactory::MessageDisposition))]
                              : [/shared/Middleware/GGPRE0T5/libggjava.so()]
                              : [/shared/Middleware/GGPRE0T5/libggjava.so(ggs::gglib::ggdal::CJavaRecordWriter::CommitTransaction(int, int, int, int, ggs::gglib::ggdal::CDALError&))]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::gglib::ggdal::CDALRecordWriter::CommitTransaction(int, int, int, int, ggs::gglib::ggdal::CDALError&))]
                              : [/shared/Middleware/GGPRE0T5/replicat(GenericImpl::CommitTransaction(int, int, int, int))]
                              : [/shared/Middleware/GGPRE0T5/replicat(odbc_commit_transaction())]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::er::ReplicatContext::commitAndCheckpoint(short, bool&, RepCsn&))]
                              : [/shared/Middleware/GGPRE0T5/replicat(ClassicWorker::CommitAndCheckpoint(short))]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::er::ReplicatContext::processReplicatLoop())]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::er::ReplicatContext::run())]
                              : [/shared/Middleware/GGPRE0T5/replicat()]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::gglib::MultiThreading::MainThread::ExecMain())]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*))]
                              : [/shared/Middleware/GGPRE0T5/replicat(ggs::gglib::MultiThreading::MainThread::Run(int, char**))]
                              : [/shared/Middleware/GGPRE0T5/replicat(main)]
                              : [/lib64/libc.so.6(__libc_start_main)]
                              : [/shared/Middleware/GGPRE0T5/replicat()]
    
    2024-01-23 11:52:50  ERROR   OGG-15051  Java or JNI exception:
    oracle.goldengate.util.GGException: Error detected handling transaction commit event.
    
    ***********************************************************************
    *                   ** Run Time Statistics **                         *
  1. Cassandra 处理程序无法将 Oracle 
    TIMESTAMP
    类型转换为 Cassandra 
    TIMESTAMP
    ，但可以将它们转换为 
    TEXT
    ，有没有办法在 Oracle 
    TIMESTAMP
    和 Cassandra 
    TIMESTAMP
    之间进行映射？

这是 Cassandra Hanlder 日志

Cassandra column [dta_effetto] maps to GG column [DTA_EFFETTO] Cassandra data type [TEXT] GG index [17].
Cassandra column [dta_evento] maps to GG column [DTA_EVENTO] Cassandra data type [TEXT] GG index [18].

这是Oracle这边的类型

SQL> desc table_name
Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 DTA_EFFETTO                                        TIMESTAMP(9)
 DTA_EVENTO                                         TIMESTAMP(9)
oracle cassandra bigdata cassandra-3.0 oracle-golden-gate
1个回答
0
投票

增加写请求超时时间。这是通过 Cassandra 中的 write_request_timeout_in_ms 属性控制的，位于 cassandra.yaml 文件中。默认值为 2000 毫秒。您可以增加此值以消除错误，然后重新启动 Cassandra 节点以使更改生效。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.