如何在Elasticsearch索引中同时使用ngram和Edge ngram标记器?

问题描述 投票:0回答:1

我有一个包含3个文档的索引。

            {
                    "firstname": "Anne",
                    "lastname": "Borg",
                }

            {
                    "firstname": "Leanne",
                    "lastname": "Ray"
                },

            {
                    "firstname": "Anne",
                    "middlename": "M",
                    "lastname": "Stone"
                }

最初,我使用的是ngram标记器。我的索引映射中还有一个生成的字段,称为“ full_name”,其中包含名字,中间名和姓氏字符串。当我搜索“ Anne”时,所有3个文档都在结果集中。但是,安妮·M·斯通的分数与莱安·雷相同。安妮·斯通(Ann M Stone)的得分应该比莱安(Leanne)高。

为了解决这个问题,我将我的ngram标记生成器更改为edge_ngram标记生成器。这具有将Leanne Ray从结果集中完全删除的效果。我们希望将此结果保留在结果集中-因为它仍然包含查询字符串-但得分比其他两个更好的匹配要低。

我读到某处可能在同一索引中将边缘ngram过滤器与ngram过滤器一起使用。如果是这样,我应该如何重新创建索引呢?有更好的解决方案吗?

这里是初始索引设置。

{
    "settings": {
        "analysis": {
            "analyzer": {
                "my_analyzer": {
                    "filter": [
                        "lowercase"
                    ],
                    "type": "custom",
                    "tokenizer": "my_tokenizer"
                }
            },
            "tokenizer": {
                "my_tokenizer": {
                    "token_chars": [
                        "letter",
                        "digit",
                        "custom"
                    ],
                    "custom_token_chars": "'-",
                    "min_gram": "3",
                    "type": "ngram",
                    "max_gram": "4"
                }
            }
        }
    },
    "mappings": {
        "properties": {
            "contact_id": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword",
                        "ignore_above": 256
                    }
                }
            },

            "firstname": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword"
                    }
                },
                "copy_to": [
                    "full_name"
                ]
            },


            "lastname": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword"
                    }
                },
                "copy_to": [
                    "full_name"
                ]
            },

            "middlename": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword",
                        "ignore_above": 256
                    }
                },
                "copy_to": [
                    "full_name"
                ]
            },

            "full_name": {
                "type": "text",
                "analyzer": "my_analyzer",
                "fields": {
                    "keyword": {
                        "type": "keyword"
                    }
                }
            }
        }
    }
}

这是我的查询

{
    "query": {
        "bool": {
            "should": [
                {
                    "query_string": {
                        "query": "Anne",
                        "fields": [
                            "full_name"
                        ]
                    }
                }
            ]
        }
    }
}

这带来了这些结果

    "hits": {
        "total": {
            "value": 3,
            "relation": "eq"
        },
        "max_score": 0.59604377,
        "hits": [
            {
                "_index": "contacts_15",
                "_type": "_doc",
                "_id": "3",
                "_score": 0.59604377,
                "_source": {
                    "firstname": "Anne",
                    "lastname": "Borg"
                }
            },
            {
                "_index": "contacts_15",
                "_type": "_doc",
                "_id": "1",
                "_score": 0.5592099,
                "_source": {
                    "firstname": "Anne",
                    "middlename": "M",
                    "lastname": "Stone"
                }
            },
            {
                "_index": "contacts_15",
                "_type": "_doc",
                "_id": "2",
                "_score": 0.5592099,
                "_source": {
                    "firstname": "Leanne",
                    "lastname": "Ray"
                }
            }
        ]
    }

如果我改用边缘ngram标记器,这就是索引设置的样子...

{
    "settings": {
        "max_ngram_diff": "10",
        "analysis": {
            "analyzer": {
                "my_analyzer": {
                    "filter": [
                        "lowercase"
                    ],
                    "type": "custom",
                    "tokenizer": ["edge_ngram_tokenizer"]
                }
            },
            "tokenizer": {
                "edge_ngram_tokenizer": {
                    "token_chars": [
                        "letter",
                        "digit",
                        "custom"
                    ],
                    "custom_token_chars": "'-",
                    "min_gram": "2",
                    "type": "edge_ngram",
                    "max_gram": "10"
                }
            }
        }
    },
    "mappings": {
        "properties": {
            "contact_id": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword",
                        "ignore_above": 256
                    }
                }
            },

            "firstname": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword"
                    }
                },
                "copy_to": [
                    "full_name"
                ]
            },


            "lastname": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword"
                    }
                },
                "copy_to": [
                    "full_name"
                ]
            },

            "middlename": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword",
                        "ignore_above": 256
                    }
                },
                "copy_to": [
                    "full_name"
                ]
            },

            "full_name": {
                "type": "text",
                "analyzer": "my_analyzer",
                "fields": {
                    "keyword": {
                        "type": "keyword"
                    }
                }
            }
        }
    }
}

并且同一查询带回了这个新结果集...

   "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 1.5131824,
        "hits": [
            {
                "_index": "contacts_16",
                "_type": "_doc",
                "_id": "3",
                "_score": 1.5131824,
                "_source": {
                    "firstname": "Anne",
                    "middlename": "M",
                    "lastname": "Stone"
                }
            },
            {
                "_index": "contacts_16",
                "_type": "_doc",
                "_id": "1",
                "_score": 1.4100108,
                "_source": {
                    "firstname": "Anne",
                    "lastname": "Borg"
                }
            }
        ]
    }
elasticsearch n-gram relevance
1个回答
0
投票

您可以继续使用ngram(即第一个解决方案),但随后您需要更改查询以提高相关性。它的工作方式是在multi_match子句中添加增强的should查询,以增加其名字或姓氏与输入完全匹配的文档的分数:

{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "Anne",
            "fields": [
              "full_name"
            ]
          }
        }
      ],
      "should": [
        {
          "multi_match": {
            "query": "Anne",
            "fields": [
              "firstname",
              "lastname"
            ],
            "boost": 10
          }
        }
      ]
    }
  }
}

此查询会将Anne BorgAnne M Stone带到Leanne Ray之前。

© www.soinside.com 2019 - 2024. All rights reserved.