从 Map<Person, list<Property>、List<Feature>> 到 Map<Property, Map<Feature, List<Person>>>

问题描述 投票:0回答:1

我想知道是否可以为以下用例进行摇动变换

Person {
 String id;
}

Property {
 Id
 name
 area
 address
}

Feature{
 String featureId;
}

基本上我们的 Person 具有多种属性和特征。我们希望使用颠簸变换以这样的方式反转映射,即我们希望按功能分组的所有人员的每个属性列表。

输入为:

[
 {
  personId: "personId1",
  properties: [{id: 1, name: "property1", area: "abc"}, {id: 2, name: "property2", area: "def"}],
  features: [{featureId: "feature1"}, {featureId: "feature2"}, {featureId: "feature3"}}]
 },
 {
  personId: "personId2",
  properties: [{id: 1, name: "property1", area: "abc"}, {id: 3, name: "property3"}],
  features: [{featureId: "feature1"}, {featureId: "feature3"}, {featureId: "feature5"}}]
 }
]

预计输出为:

[
{
 id: 1,
 name: "property1",
 area: "abc",
 featureMap: [
 {
 featureId: "feature1",
 persons: [personId1, personId2]
 },
 {
 featureId: "feature2",
 persons: [personId1]
 },
 {
 featureId: "feature3",
 persons: [personId1, personId2]
 },
 {
 featureId: "feature5",
 persons: [personId2]
 }]
},
{
 id: 2,
 name: "property2",
 area: "def",
 featureMap: [
 {
 featureId: "feature1",
 persons: [personId1]
 },
 {
 featureId: "feature2",
 persons: [personId1]
 },
 {
 featureId: "feature3",
 persons: [personId1]
 }]
},
{
 id: 3,
 name: "property3",
 featureMap: [
 {
 featureId: "feature1",
 persons: [personId2]
 },
 {
 featureId: "feature3",
 persons: [personId2]
 },
 {
 featureId: "feature5",
 persons: [personId2]
 }]
}
]

以上数据输入将包含数百个具有多种特征和属性的输入人员。

我无法理解这里应该使用什么作为震动变换。即使是这种复杂的转变也是可能的。它会聚合每个属性的所有数据,因此这样做需要一些 jolt 转换中的技巧,将 propertyId 转换为所有输入 json 中的键,然后使用该键进行聚合以合并所有常见内容。可能需要添加逻辑来删除重复项。

apache-nifi transformation jolt
1个回答
0
投票

首先,请确保当您需要此类帮助时提供有效的 json 数据。当我复制您的数据进行处理时,我注意到键中缺少双引号,并且在一种情况下还有一个额外的大括号。

关于规格,我认为以下应该可行。基本上,您需要创建一种按属性和特征对人员进行分组的方法(二维移位转换)。为了轻松分组,我将所有可能的属性值连接到一个字符串中(第一次修改规范)。最后,我重构了分组输出,以根据属性串联的可能组合获得所需的结果(因为某些值可能不存在,如果不存在会更容易)。

[

  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        "properties": {
          "*": {
            "allfields": "=toList(@(1,id),@(1,name),@(1,area),@(1,address))",
            "allfieldsConcat": "=join('-',@(1,allfields))"
          }
        }
      }
    }
    }

  ,
  {
    "operation": "shift",
    "spec": {
      "*": {
        "personid": {
          "*": {
            "@(2,properties)": {
              "*": {
                "allfieldsConcat": {
                  "@(5,features)": {
                    "*": {
                      "featureid": {
                        "*": {
                          "@(8)": "[#1].@(5).&1"
                          
                        }
                      } //feature id
                    } //each feature element
                  } //each feature
                } //allfieldsConcat
              } //each property element 
            } //eah property
          } //each personid value
        }
      } // each person
    }
  }
 ,
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*-*-*-*": {
          "$(0,1)": "[#2].id",
          "$(0,2)": "[#2].name",
          "$(0,3)": "[#2].area",
          "$(0,4)": "[#2].address",
          "feature*": {
            "$": "&2.featureMap[].featureId",
            "@": "&2.featureMap[].persons"
          }
        },
        "*-*-*-": {
          "$(0,1)": "[#2].id",
          "$(0,2)": "[#2].name",
          "$(0,3)": "[#2].area",
          "feature*": {
            "$": "[#3].featureMap[#2].featureId",
            "@": "[#3].featureMap[#2].persons"
          }
        },
        "*-*-": {
          "$(0,1)": "[#2].id",
          "$(0,2)": "[#2].name",
          "feature*": {
            "$": "[#3].featureMap[#2].featureId",
            "@": "[#3].featureMap[#2].persons"
          }
        }
      }
    }
  }
  /**/
]

@Barbaros Özhan,我想知道我们是否可以进一步简化规范。

谢谢

© www.soinside.com 2019 - 2024. All rights reserved.