无法在 Google Pub/Sub 主题的 Protobuf 架构中设置动态 JSON

问题描述 投票:0回答:1

我想将 protobuf 模式关联到 Google Pub/Sub 主题,

这是将收到的有关该主题的消息的示例:

{
    "event": {
        "original": "{ STRING JSON }"
    },
    "eventName": "STRING",
    "eventParams": {
        "DYNAMIC JSON"
    },
    "eventTimestamp": "2024-01-24 13:42:46.000",
    "eventUUID": "e548a0eb-3dee-4fbc-9302-2139684bb115",
    "sessionID": "65f9dd1c-3d76-4541-8296-a4233ce92775",
    "userID": "ae08f2df-7f54-472f-b3e0-857ef141607a"
}

请注意,

eventParams
字段是一个动态JSON对象,这意味着我事先不知道它将包含的字段,尽管我知道它将包含有效的JSON对象。

我已将 Protobuf 架构设置为 Pub/Sub 主题上的以下内容

syntax = "proto3";

message Test {

    // `Struct` represents a structured data value, consisting of fields
    // which map to dynamically typed values. In some languages, `Struct`
    // might be supported by a native representation. For example, in
    // scripting languages like JS a struct is represented as an
    // object. The details of that representation are described together
    // with the proto support for the language.
    //
    // The JSON representation for `Struct` is JSON object.
    message Struct {
        // Unordered map of dynamically typed values.
        map<string, Value> fields = 1;
    }

    // `Value` represents a dynamically typed value which can be either
    // null, a number, a string, a boolean, a recursive struct value, or a
    // list of values. A producer of value is expected to set one of these
    // variants. Absence of any variant indicates an error.
    //
    // The JSON representation for `Value` is JSON value.
    message Value {
        // The kind of value.
        oneof kind {
            // Represents a null value.
            NullValue null_value = 1;
            // Represents a double value.
            double number_value = 2;
            // Represents a string value.
            string string_value = 3;
            // Represents a boolean value.
            bool bool_value = 4;
            // Represents a structured value.
            Struct struct_value = 5;
            // Represents a repeated `Value`.
            ListValue list_value = 6;
        }
    }

    // `NullValue` is a singleton enumeration to represent the null value for the
    // `Value` type union.
    //
    // The JSON representation for `NullValue` is JSON `null`.
    enum NullValue {
        // Null value.
        NULL_VALUE = 0;
    }

    // `ListValue` is a wrapper around a repeated field of values.
    //
    // The JSON representation for `ListValue` is JSON array.
    message ListValue {
        // Repeated field of dynamically typed values.
        repeated Value values = 1;
    }

    message Event {
        string original = 1;
    }

    optional Event event = 1;
    optional string eventName = 2;
    optional Struct eventParams = 3;
    optional string eventTimestamp = 4;
    optional string eventUUID = 5;
    optional string sessionID = 6;
    optional string userID = 7;
}

但是,当我用消息测试时,它不起作用。我使用以下 JSON 消息进行了测试


{
    "event": {
        "original": "{ STRING }"
    },
    "eventName": "giftSent",
    "eventParams": {
        "a": 10600,
        "b": 20,
        "c": "WEB",
        "d": "35841161-f1b3-4947-a75f-057419c36988",
        "e": 1
    },
    "eventTimestamp": "2018-01-24 13:42:46.000",
    "eventUUID": "e548a0eb-3dee-4fbc-9302-65461541",
    "sessionID": "65f9dd1c-3d76-4541-8296-54654168",
    "userID": "ae08f2df-7f54-472f-b3e0-85645467a"
}

我收到此错误:

Invalid schema message: (eventParams) e: Cannot find field..

这是否可以在 Pub/Sub 上设置,是否有其他设置方法?

google-cloud-platform protocol-buffers google-cloud-pubsub
1个回答
0
投票

尽管 Google 众所周知的类型 (WKT) StructValue 提供了一种将 JSON 等动态类型值表示为 Protobuf 模式的机制,但您无法简单地映射此表示形式。

这是使用 Go 的示例:

m := &pb.Test{}

j := `
{
    "event": {
        "original": "{ STRING }"
    },
    "eventName": "giftSent",
    "eventParams": {
        "a": 10600,
        "b": 20,
        "c": "WEB",
        "d": "35841161-f1b3-4947-a75f-057419c36988",
        "e": 1
    },
    "eventTimestamp": "2018-01-24 13:42:46.000",
    "eventUUID": "e548a0eb-3dee-4fbc-9302-65461541",
    "sessionID": "65f9dd1c-3d76-4541-8296-54654168",
    "userID": "ae08f2df-7f54-472f-b3e0-85645467a"
}`

if err := protojson.Unmarshal([]byte(j), m); err == nil {
    t.Error("expected unsuccessful marshal")
}

无法将 JSON 直接解组为消息类型

Test
(此处用
pb.Test
表示,因为
eventParams
在 JSON 中包含一个字段
a
,但 Protobuf 架构表示这是
Struct 的一部分) 
(
map[string]Value
) 和
Value
oneof
(但是是哪一个!?)。在 Go 中,这必须是明确的:

m := &pb.Test{
    ...
    EventParams: &pb.Test_Struct{
        Fields: map[string]*pb.Test_Value{
            "a": {
                Kind: &pb.Test_Value_NumberValue{
                    NumberValue: 10600,
                },
            },
        },
    },
}

有一个帮助程序包

structpb
,其中包含一个方法
NewValue
,这是实现作为所有可能性的切换,但它仍然需要Go中的类型注释:

m, err := structpb.NewValue(map[string]interface{}{
    "event": map[string]interface{}{
        "original": "{ STRING }",
    },
    "eventName": "giftSent",
    "eventParams": map[string]interface{}{
        "a": 10600,
        "b": 20,
        "c": "WEB",
        "d": "35841161-f1b3-4947-a75f-057419c36988",
        "e": 1,
    },
    "eventTimestamp": "2018-01-24 13:42:46.000",
    "eventUUID":      "e548a0eb-3dee-4fbc-9302-65461541",
    "sessionID":      "65f9dd1c-3d76-4541-8296-54654168",
    "userID":         "ae08f2df-7f54-472f-b3e0-85645467a",
})
if err != nil {
    t.Fatal(err)
}

fmt.Printf("+%v", m)

b, err := protojson.Marshal(m)
if err != nil {
    t.Fatal("expected successful marshal")
}

fmt.Printf("%s", string(b))

这是

structpb
NewValue
为 JSON 消息构造的 Protobuf 消息:

struct_value:{
  fields:{
    key:"event"
    value:{
      struct_value:{
        fields:{
          key:"original"
          value:{
            string_value:"{ STRING }"
          }
        }
      }
    }
  }
  fields:{
    key:"eventName"
    value:{
      string_value:"giftSent"
    }
  }
  fields:{
    key:"eventParams"
    value:{
      struct_value:{
        fields:{
          key:"a"
          value:{
            number_value:10600
          }
        }
        fields:{
          key:"b"
          value:{
            number_value:20
          }
        }
        fields:{
          key:"c"
          value:{
            string_value:"WEB"
          }
        }
        fields:{
          key:"d"
          value:{
            string_value:"35841161-f1b3-4947-a75f-057419c36988"
          }
        }
        fields:{
          key:"e"
          value:{
            number_value:1
          }
        }
      }
    }
  }
  fields:{
    key:"eventTimestamp"
    value:{
      string_value:"2018-01-24 13:42:46.000"
    }
  }
  fields:{
    key:"eventUUID"
    value:{
      string_value:"e548a0eb-3dee-4fbc-9302-65461541"
    }
  }
  fields:{
    key:"sessionID"
    value:{
      string_value:"65f9dd1c-3d76-4541-8296-54654168"
    }
  }
  fields:{
    key:"userID"
    value:{
      string_value:"ae08f2df-7f54-472f-b3e0-85645467a"
    }
  }
}

这将正确地编组为 JSON:

{
    "event": {
        "original": "{ STRING }"
    },
    "eventName": "giftSent",
    "eventParams": {
        "a": 10600,
        "b": 20,
        "c": "WEB",
        "d": "35841161-f1b3-4947-a75f-057419c36988",
        "e": 1
    },
    "eventTimestamp": "2018-01-24 13:42:46.000",
    "eventUUID": "e548a0eb-3dee-4fbc-9302-65461541",
    "sessionID": "65f9dd1c-3d76-4541-8296-54654168",
    "userID": "ae08f2df-7f54-472f-b3e0-85645467a"
}

所以,是的,您可以使用

Struct
Value
来表示任意 JSON 消息,但您无法简单地验证这些消息。

© www.soinside.com 2019 - 2024. All rights reserved.