使用DataFlowRunner当光束/数据流意外错误ProtocolMessageEnum未实现

问题描述 投票:0回答:1

当运行我的梁管道本地所有的工作如预期,但试图在DataflowRunner运行它时,我突然得到下面的错误。老实说,我甚至不知道从哪里开始评估这一点,因为DataflowRunner似乎是一个黑盒子。

Jan 14, 2019 11:26:51 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 165 files. Enable logging at DEBUG level to see which files will be staged.
Exception in thread "main" java.lang.IncompatibleClassChangeError: Class org.apache.beam.model.pipeline.v1.RunnerApi$StandardPTransforms$Primitives does not implement the requested interface com.google.protobuf.ProtocolMessageEnum
        at org.apache.beam.runners.core.construction.BeamUrns.getUrn(BeamUrns.java:27)
        at org.apache.beam.runners.core.construction.PTransformTranslation.<clinit>(PTransformTranslation.java:58)
        at org.apache.beam.runners.core.construction.UnconsumedReads$1.visitValue(UnconsumedReads.java:49)
        at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:666)
        at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
        at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
        at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
        at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
        at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
        at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
        at org.apache.beam.runners.core.construction.UnconsumedReads.ensureAllReadsConsumed(UnconsumedReads.java:40)
        at org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java:868)
        at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:660)
        at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:173)
        at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
        at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
        at (my code: pipe.run().waitUntilFinish();) 
protocol-buffers google-cloud-dataflow apache-beam dataflow
1个回答
0
投票

检查梁等的版本和升级你的依赖在可能的情况。

我有同样的错误,看到你得到这个错误后,我想这一定是依赖性冲突,因为它之前并不存在。

我使用的是国务院新闻办公室部署到数据流,他们正在使用的只是引用。 https://github.com/spotify/scio/blob/v0.7.1/build.sbt我更新番石榴的protobuf也。

我知道你在使用Java,但尝试更新梁2.9.0,也许番石榴,protobuf的...

© www.soinside.com 2019 - 2024. All rights reserved.