我必须通过Terraform创建一个k8s作业,并以某种方式组织等待该作业完成的机制(因为Terraform无法执行此操作https://github.com/terraform-providers/terraform-provider-kubernetes/issues/534)。我发现没有什么比在等待k8s作业的命令(即kubectl wait
)中使用空资源更好的了。这些东西将在CI上的Docker容器中执行。此外,我需要仔细研究堡垒才能进入k8s集群。我为此使用SSH隧道:
provider "ssh" {
port = ....
}
provider "kubernetes" {
config_context = "..."
config_context_cluster = "..."
host = "api.${k8s_host}:${data.ssh_tunnel.k8s.port}"
}
data "ssh_tunnel" "k8s" {
host = "bastion.....com"
local_address = "localhost:0"
remote_address = "api.${k8s_host}:443"
}
所有k8s资源均已成功创建,因此我认为SSH隧道可以正常工作。但是如何将其用于空资源?就是这里:
resource "null_resource" "wait" {
provisioner "local-exec" {
connection {
type = "ssh"
bastion_host = data.ssh_tunnel.k8s.host
bastion_private_key = file("~/.ssh/id_rsa")
bastion_port = data.ssh_tunnel.k8s.port
host = "api.${k8s_host}"
port = 443
}
command = "kubectl wait ...."
}
triggers = {
job_ids = join(", ", kubernetes_job.a-job.*.id)
}
}
但是不走运,我收到“与服务器api的连接。$ {k8s_host}被拒绝-您指定了正确的主机或端口吗?”
所以有两个问题:1.如何以其他方式等待工作2.如果1是不可能的(我确定是这样),如何以正确的方式重用SSH隧道。
P.S是,我阅读了文档https://www.terraform.io/docs/provisioners/connection.html,但我肯定做错了。
我想出了如何重用Terraform打开的SSH隧道:
resource "null_resource" "wait" {
provisioner "local-exec" {
command = "kubectl wait --server=https://api.${k8s_host}:${data.ssh_tunnel.k8s.port} --for=condition=complete --timeout=3000s job/a-job"
}
triggers = {
job_ids = join(", ", kubernetes_job.a-job.*.id)
}
}