EKS集群中AWS EBS的HDFS Namenode格式问题

Question

我有EKS集群与EBS存储类volume。我有elasticsearch集群与EBS存储一起运行得很好（作为持久化的volumepvc）.我试图使用statefulset部署hdfs namenode镜像（bde2020hadoop-namenode），但它总是给我以下错误。

2020-05-09 08:59:02,400 INFO util.GSet: capacity      = 2^15 = 32768 entries
2020-05-09 08:59:02,415 INFO common.Storage: Lock on /hadoop/dfs/name/in_use.lock acquired by nodename [email protected]
2020-05-09 08:59:02,417 WARN namenode.FSNamesystem: Encountered exception loading fsimage
java.io.IOException: NameNode is not formatted.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:252)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1105)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:720)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:648)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:710)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:953)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:926)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1692)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)

我检查了这个iameg的run.sh，如果dir是空的，它似乎确实在格式化namenode。但在可能的情况下，这并不工作（用EBS作为PVC）。任何帮助将是非常感激的。

我的部署yml是。

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: hdfs-name
  labels:
    component: hdfs-name
spec:
  serviceName: hdfs-name
  replicas: 1
  selector:
    matchLabels:
      component: hdfs-name
  template:
    metadata:
      labels:
        component: hdfs-name
    spec:
      containers:
      - name: hdfs-name
        image: bde2020/hadoop-namenode
        env:
        - name: CLUSTER_NAME
          value: hdfs-k8s
        ports:
        - containerPort: 8020
          name: nn-rpc
        - containerPort: 50070
          name: nn-web
        volumeMounts:
        - name: hdfs-name-pv-claim
          mountPath: /hadoop/dfs/name 
  volumeClaimTemplates:
  - metadata:
      name: hdfs-name-pv-claim
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: ebs
      resources:
        requests:
          storage: 1Gi

Answer 1

使用EBS存储类，会自动创建失物招领文件夹。由于这个原因，没有发生 namenode 格式化。用initcontainer删除lost+found文件夹似乎可以。

initContainers:
  - name: delete-lost-found
    image: busybox
    command: ["sh", "-c", "rm -rf /hadoop/dfs/name/lost+found"]
    volumeMounts:
    - name: hdfs-name-pv-claim
      mountPath: /hadoop/dfs/name