我一直在尝试使用以下方法在我的kubernetes集群上运行glusterfs集群:
glusterfs-service.json
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "glusterfs-cluster"
},
"spec": {
"type": "NodePort",
"selector": {
"name": "gluster"
},
"ports": [
{
"port": 1
}
]
}
}
和glusterfs-server.json
:
{
"apiVersion": "extensions/v1beta1",
"kind": "DaemonSet",
"metadata": {
"labels": {
"name": "gluster"
},
"name": "gluster"
},
"spec": {
"selector": {
"matchLabels": {
"name": "gluster"
}
},
"template": {
"metadata": {
"labels": {
"name": "gluster"
}
},
"spec": {
"containers": [
{
"name": "gluster",
"image": "gluster/gluster-centos",
"livenessProbe": {
"exec": {
"command": [
"/bin/bash",
"-c",
"systemctl status glusterd.service"
]
}
},
"readinessProbe": {
"exec": {
"command": [
"/bin/bash",
"-c",
"systemctl status glusterd.service"
]
}
},
"securityContext": {
"privileged": true
},
"volumeMounts": [
{
"mountPath": "/mnt/brick1",
"name": "gluster-brick"
},
{
"mountPath": "/etc/gluster",
"name": "gluster-etc"
},
{
"mountPath": "/var/log/gluster",
"name": "gluster-logs"
},
{
"mountPath": "/var/lib/glusterd",
"name": "gluster-config"
},
{
"mountPath": "/dev",
"name": "gluster-dev"
},
{
"mountPath": "/sys/fs/cgroup",
"name": "gluster-cgroup"
}
]
}
],
"dnsPolicy": "ClusterFirst",
"hostNetwork": true,
"volumes": [
{
"hostPath": {
"path": "/mnt/brick1"
},
"name": "gluster-brick"
},
{
"hostPath": {
"path": "/etc/gluster"
},
"name": "gluster-etc"
},
{
"hostPath": {
"path": "/var/log/gluster"
},
"name": "gluster-logs"
},
{
"hostPath": {
"path": "/var/lib/glusterd"
},
"name": "gluster-config"
},
{
"hostPath": {
"path": "/dev"
},
"name": "gluster-dev"
},
{
"hostPath": {
"path": "/sys/fs/cgroup"
},
"name": "gluster-cgroup"
}
]
}
}
}
}
然后在我的pod定义中,我正在做:
"volumes": [
{
"name": "< volume name >",
"glusterfs": {
"endpoints": "glusterfs-cluster.default.svc.cluster.local",
"path": "< gluster path >",
"readOnly": false
}
}
]
但是pod创建已超时,因为它无法装入卷
它看起来只有一个glusterfs pod正在运行
这是我的日志:http://imgur.com/a/j2I8r
然后我尝试在我的gluster集群运行时在同一名称空间上运行我的pod,我现在收到此错误:
Operation for "\"kubernetes.io/glusterfs/01a0834e-64ab-11e6-af52-42010a840072-ssl-certificates\" (\"01a0834e-64ab-11e6-af52-42010a840072\")" failed.
No retries permitted until 2016-08-17 18:51:20.61133778 +0000 UTC (durationBeforeRetry 2m0s).
Error: MountVolume.SetUp failed for volume "kubernetes.io/glusterfs/01a0834e-64ab-11e6-af52-42010a840072-ssl-certificates" (spec.Name: "ssl-certificates") pod "01a0834e-64ab-11e6-af52-42010a840072" (UID: "01a0834e-64ab-11e6-af52-42010a840072") with: glusterfs: mount failed:
mount failed: exit status 1
Mounting arguments:
10.132.0.7:ssl_certificates /var/lib/kubelet/pods/01a0834e-64ab-11e6-af52-42010a840072/volumes/kubernetes.io~glusterfs/ssl-certificates
glusterfs [log-level=ERROR log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/ssl-certificates/caddy-server-1648321103-epvdi-glusterfs.log]
Output: Mount failed. Please check the log file for more details. the following error information was pulled from the glusterfs log to help diagnose this issue:
[2016-08-17 18:49:20.583585] E [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-mgmt: failed to fetch volume file (key:ssl_certificates)
[2016-08-17 18:49:20.610531] E [glusterfsd-mgmt.c:1494:mgmt_getspec_cbk] 0-glusterfs: failed to get the 'volume file' from server
日志清楚地说明发生了什么:
未能获得端点glusterfs-cluster [端点“glusterfs-cluster”未找到]
因为:
"ports": [
{
"port": 1
}
在某些方面是虚假的。首先,端口“1”非常可疑。第二,它在containerPort:
方面没有匹配的DaemonSet
,kubernetes可以指向Service
- 因此,它不会为Endpoints
元组创建(podIP, protocol, port)
。因为glusterfs(合理地)想要直接接触底层Pod
s,而不通过Service
,然后它无法发现Pod
s并且一切都突然停止。