基于索引的路由分配不起作用 - Elasticsearch 5.x

Question

作为现有开放线程的一个相关问题，我怀疑修改集群和索引时设置的策略或实现细节。虽然我尝试配置每个索引级别的分配，但该配置并未真正按预期动态工作。

执行相关命令后，我们当前的配置反映的是最新的：

GET user/_settings?include_defaults=true&filter_path=**.routing.**

{
  "user": {
    "settings": {
      "index": {
        "routing": {
          "allocation": {
            "include": {
              "size": "big",
              "rack": "compute"
            },
            "exclude": {
              "rack": "storage"
            }
          }
        }
      }
    },
    "defaults": {
      "index": {
        "routing": {
          "rebalance": {
            "enable": "ALL"
          },
          "allocation": {
            "enable": "ALL",
            "total_shards_per_node": "-1"
          }
        }
      }
    }
  }
}

具有集群设置：

GET _cluster/settings?include_defaults=true&filter_path=**.routing.**

{
  "transient": {
    "cluster": {
      "routing": {
        "rebalance": {
          "enable": "all"
        },
        "allocation": {
          "disk": {
            "watermark": {
              "low": "90%"
            }
          },
          "enable": "all",
          "total_shards_per_node": "1000"
        }
      }
    }
  },
  "defaults": {
    "cluster": {
      "routing": {
        "allocation": {
          "node_concurrent_incoming_recoveries": "2",
          "disk": {
            "threshold_enabled": "true",
            "watermark": {
              "high": "90%"
            },
            "include_relocations": "true",
            "reroute_interval": "60s"
          },
          "node_initial_primaries_recoveries": "4",
          "awareness": {
            "attributes": ""
          },
          "balance": {
            "index": "0.55",
            "threshold": "1.0",
            "shard": "0.45"
          },
          "same_shard": {
            "host": "false"
          },
          "node_concurrent_outgoing_recoveries": "2",
          "allow_rebalance": "indices_all_active",
          "cluster_concurrent_rebalance": "2",
          "node_concurrent_recoveries": "2",
          "type": "balanced",
          "snapshot": {
            "relocation_enabled": "false"
          }
        }
      }
    }
  }
}

我之所以肯定这个结论，是因为

_cat/shards

中提到的

user

索引的节点涉及所有类型的节点，甚至排除了标签！

user    1   r   STARTED ... ... search02
user    1   p   STARTED ... ... search05
user    1   r   STARTED ... ... search22
user    3   r   STARTED ... ... search21
user    3   r   STARTED ... ... search10
user    3   p   STARTED ... ... search26
user    2   r   STARTED ... ... search09
user    2   r   STARTED ... ... search08
user    2   p   STARTED ... ... search11
user    4   p   STARTED ... ... search06
user    4   r   STARTED ... ... search15
user    4   r   STARTED ... ... search27
user    0   r   STARTED ... ... search35
user    0   p   STARTED ... ... search36
user    0   r   STARTED ... ... search20

问：我错过了什么吗？或者此更改是否需要滚动重新启动我的数据节点（K8s 中的 pod）集群才能反映更改？

更新：正如Val在

评论中所问的那样

GET _cat/nodeattrs?v


node     host        ip          attr       value
search17 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search20 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search13 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search15 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search28 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search01 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search21 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search12 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search31 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search19 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search08 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search35 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search10 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search14 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search27 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search36 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search33 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search09 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search16 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search29 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search24 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search11 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search18 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search07 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search34 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search23 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search22 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search26 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search37 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search06 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search32 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search04 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search02 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search03 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search00 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search05 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search30 10.xx.xx.xx 10.xx.xx.xx ml.enabled true
search25 10.xx.xx.xx 10.xx.xx.xx ml.enabled true

Answer 1

如果您查看

GET _cat/nodeattrs?v

的输出，您可以看到修改配置文件后是否已考虑您的节点属性。

尽管您可以动态更改与分片分配过滤相关的索引和集群级别设置，但修改节点属性需要重新启动节点。

在您的情况下，如果对

user

索引使用索引级分片过滤并指定

index.routing.allocation.require

而不是

index.routing.allocation.include

，后者只是一个提示，而前者是一个硬约束。

这样您就可以强制

user

索引位于

big

节点上，而其他索引不位于

big

节点上。