我有 Jenkins 管道来构建和运行集成测试，这需要 docker 环境来运行测试容器，这些测试容器使用 Kubedock 在 Kubernetes 中创建 Mysql 数据库。
Jenkins 代理 Pod 和 Kubedock Pod 已成功创建，但测试失败，出现以下连接错误。
下面是在maven测试中尝试连接Mysql数据库时的应用程序日志。
2024-10-17T06:25:41.663Z INFO 231 --- [ main] org.testcontainers.DockerClientFactory : Testcontainers version: 1.20.2
2024-10-17T06:25:42.403Z INFO 231 --- [ main] o.t.d.DockerClientProviderStrategy : Found Docker environment with Environment variables, system properties and defaults. Resolved dockerHost=tcp://127.0.0.1:2475
2024-10-17T06:25:42.403Z WARN 231 --- [ main] o.t.d.DockerClientProviderStrategy : Could not determine Docker OS type
2024-10-17T06:25:42.404Z INFO 231 --- [ main] org.testcontainers.DockerClientFactory : Docker host IP address is 127.0.0.1
2024-10-17T06:25:42.418Z INFO 231 --- [ main] org.testcontainers.DockerClientFactory : Connected to docker:
Server Version: 0.17.0-2-g6353a5f
API Version: 1.25
Operating System: kubernetes
Total Memory: 0 MB
2024-10-17T06:25:42.421Z WARN 231 --- [ main] o.testcontainers.utility.ResourceReaper :
********************************************************************************
Ryuk has been disabled. This can cause unexpected behavior in your environment.
********************************************************************************
2024-10-17T06:25:46.458Z INFO 231 --- [ main] tc.mysql:8.0.39 : Creating container for image: mysql:8.0.39
2024-10-17T06:25:46.467Z INFO 231 --- [ main] o.t.utility.RegistryAuthLocator : Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: mysql:8.0.39, configFile: /root/.docker/config.json, configEnv: DOCKER_AUTH_CONFIG). Falling back to docker-java default behaviour. Exception message: Status 404: No config supplied. Checked in order: /root/.docker/config.json (file not found), DOCKER_AUTH_CONFIG (not set)
2024-10-17T06:25:57.153Z INFO 231 --- [ main] tc.mysql:8.0.39 : Container mysql:8.0.39 is starting: f279144bfdbe5dbfa6e90c1103675e65572dc335843cb4c99fcc5d4d2f1b1c16
2024-10-17T06:25:57.193Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17 06:25:50+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.39-1.el9 started.
2024-10-17T06:25:57.194Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17 06:25:54+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2024-10-17T06:25:57.194Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17 06:25:54+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.39-1.el9 started.
2024-10-17T06:25:57.194Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17 06:25:55+00:00 [Note] [Entrypoint]: Initializing database files
2024-10-17T06:25:57.194Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17T06:25:55.178476Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
2024-10-17T06:25:57.194Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17T06:25:55.190788Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.39) initializing of server in progress as process 80
2024-10-17T06:25:57.234Z INFO 231 --- [ main] tc.mysql:8.0.39 : Waiting for database connection to become available at jdbc:mysql://127.0.0.1:38259/blogger using query 'SELECT 1'
2024-10-17T06:25:57.289Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17T06:25:55.568910Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2024-10-17T06:26:15.307Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17T06:26:15.156186Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2024-10-17T06:26:56.726Z INFO 231 --- [eam--1797563069] z.co.customtech.AbstractIntegrationTest : STDOUT: 2024-10-17T06:26:56.586681Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2024-10-17T06:27:57.691Z ERROR 231 --- [ main] tc.mysql:8.0.39 : Could not start container
java.lang.IllegalStateException: Container is started, but cannot be accessed by (JDBC URL: jdbc:mysql://127.0.0.1:38259/blogger), please check container logs
at org.testcontainers.containers.JdbcDatabaseContainer.waitUntilContainerStarted(JdbcDatabaseContainer.java:210) ~[jdbc-1.20.2.jar:na]
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500) ~[testcontainers-1.20.2.jar:na]
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:354) ~[testcontainers-1.20.2.jar:na]
Kubedock Pod
[GIN-debug] Listening and serving HTTP on :2475
I1017 06:23:48.773547 1 main.go:66] start cleaning lingering objects...
I1017 06:23:49.026240 1 main.go:68] finished cleaning lingering objects...
I1017 06:24:49.026772 1 main.go:66] start cleaning lingering objects...
I1017 06:24:50.706661 1 main.go:68] finished cleaning lingering objects...
[GIN] 2024/10/17 - 06:25:42 |[97;42m 200 [0m| 449.762µs | 127.0.0.1 |[97;44m GET [0m "/info"
[GIN] 2024/10/17 - 06:25:42 |[97;42m 200 [0m| 30.871µs | 127.0.0.1 |[97;44m GET [0m "/version"
[GIN] 2024/10/17 - 06:25:42 |[97;42m 200 [0m| 427.77µs | 127.0.0.1 |[97;44m GET [0m "/images/json"
[GIN] 2024/10/17 - 06:25:46 |[97;42m 200 [0m| 3.9216542s | 127.0.0.1 |[97;44m GET [0m "/images/mysql:8.0.39/json"
[GIN] 2024/10/17 - 06:25:46 |[97;42m 201 [0m| 455.79µs | 127.0.0.1 |[97;46m POST [0m "/containers/create"
W1017 06:25:46.619401 1 container.go:226] user not set, will run as user defined in image
I1017 06:25:50.707259 1 main.go:66] start cleaning lingering objects...
I1017 06:25:50.818169 1 main.go:68] finished cleaning lingering objects...
I1017 06:25:51.925403 1 deploy.go:223] reverse proxy for 38259 to 3306
I1017 06:25:51.925440 1 tcpproxy.go:37] start reverse-proxy 0.0.0.0:38259->10.244.1.147:3306
I1017 06:25:56.969437 1 copy.go:30] copy archive to f279144bfdbe:/
I1017 06:25:56.969671 1 exec.go:59] exec kubedock-f279144bfdbe:[tar -xf - -C /]
[GIN] 2024/10/17 - 06:25:57 |[97;42m 200 [0m| 10.554630777s | 127.0.0.1 |[90;43m PUT [0m "/containers/f279144bfdbe5dbfa6e90c1103675e65572dc335843cb4c99fcc5d4d2f1b1c16/archive?noOverwriteDirNonDir=false&path=%2F©UIDGID=false"
W1017 06:25:57.156293 1 containers.go:36] container f279144bfdbe5dbfa6e90c1103675e65572dc335843cb4c99fcc5d4d2f1b1c16 already running
[GIN] 2024/10/17 - 06:25:57 |[97;42m 204 [0m| 112.701µs | 127.0.0.1 |[97;46m POST [0m "/containers/f279144bfdbe5dbfa6e90c1103675e65572dc335843cb4c99fcc5d4d2f1b1c16/start"
[GIN] 2024/10/17 - 06:25:57 |[97;42m 200 [0m| 3.135396ms | 127.0.0.1 |[97;44m GET [0m "/containers/f279144bfdbe5dbfa6e90c1103675e65572dc335843cb4c99fcc5d4d2f1b1c16/json"
[GIN] 2024/10/17 - 06:25:57 |[97;42m 200 [0m| 44.687µs | 127.0.0.1 |[97;44m GET [0m "/images/mysql:8.0.39/json"
[GIN] 2024/10/17 - 06:25:57 |[97;42m 200 [0m| 19.1µs | 127.0.0.1 |[97;44m GET [0m "/version"
[GIN] 2024/10/17 - 06:25:57 |[97;42m 200 [0m| 3.106078ms | 127.0.0.1 |[97;44m GET [0m "/containers/f279144bfdbe5dbfa6e90c1103675e65572dc335843cb4c99fcc5d4d2f1b1c16/json"
[GIN] 2024/10/17 - 06:25:57 |[97;42m 200 [0m| 3.112197ms | 127.0.0.1 |[97;44m GET [0m "/containers/f279144bfdbe5dbfa6e90c1103675e65572dc335843cb4c99fcc5d4d2f1b1c16/json"
I1017 06:26:27.211428 1 tcpproxy.go:68] accepted connection for 0.0.0.0:38259 to 10.244.1.147:3306
W1017 06:26:27.222916 1 tcpproxy.go:99] error dialing 10.244.1.147:3306: dial tcp 10.244.1.147:3306: connect: connection refused (attempt: 0)
W1017 06:26:27.226730 1 tcpproxy.go:99] error dialing 10.244.1.147:3306: dial tcp 10.244.1.147:3306: connect: connection refused (attempt: 1)
参见詹金斯管道
def pod =
'''
apiVersion: v1
kind: Pod
metadata:
name: java-docker-build-pod
spec:
serviceAccountName: kubedock
volumes:
- name: socket
hostPath:
path: /var/run/docker.sock
containers:
- name: kubedock
image: joyrex2001/kubedock:latest
command:
- /usr/local/bin/kubedock
args:
- server
- --service-account=kubedock
- --reverse-proxy
- --timeout=2m
- -v=3
- --inspector
tty: true
- name: maven
image: maven:3.9.9-sapmachine-17
tty: true
command: ["cat"]
'''
pipeline {
agent {
kubernetes {
yaml pod
}
}
options {
skipStagesAfterUnstable()
}
triggers {
pollSCM 'H/2 * * * *'
}
environment {
TESTCONTAINERS_RYUK_DISABLED = "true"
TESTCONTAINERS_CHECKS_DISABLE = "true"
DOCKER_HOST = "tcp://127.0.0.1:2475"
}
stages {
stage('Test') {
steps {
container('maven') {
script {
sh "mvn test"
}
}
}
}
}
}
这就是有效的方法 - 我已经从 kubedock 切换到在 Docker(DIND) 中使用 Docker。例如 https://www.docker.com/blog/how-to-use-testcontainers-on-jenkins-ci/
在我的 Spring Boot 应用程序中，我添加了 Maven 包装文件（mvnw 和 mvnw.cmd） - 这样我就不再需要在 jenkinsFile 管道中使用 Maven 容器。
这是最终的 JenkinsFile。
def pod =
'''
apiVersion: v1
kind: Pod
metadata:
labels:
name: worker
spec:
serviceAccountName: kubedock
containers:
- name: java17
image: eclipse-temurin:17.0.9_9-jdk-jammy
resources:
requests:
cpu: 1000m
memory: 2048Mi
imagePullPolicy: Always
tty: true
command:
- cat
- name: dind
image: docker:dind
imagePullPolicy: Always
tty: true
env:
- name: DOCKER_TLS_CERTDIR
value: ""
securityContext:
privileged: true
'''
pipeline {
agent {
kubernetes {
yaml pod
}
}
options {
skipStagesAfterUnstable()
}
triggers {
pollSCM 'H/2 * * * *'
}
environment {
DOCKER_TLS_VERIFY = 0
DOCKER_HOST = 'tcp://localhost:2375'
}
stages {
stage('Test') {
steps {
container('java17') {
script {
sh "./mvnw test"
}
}
}
}
}
}