我正在使用 Flink cli 在会话模型 Flink 集群中提交一个分离的作业,它成功并显示了作业 ID,但 cli 不会返回并阻止控制台。
我的docker-compose文件段:
jobmanager:
image: flink:1.15.3-scala_2.12-java8
logging: *default-logging
network_mode: host
command: jobmanager
volumes:
- /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
- ./libs:/opt/flink/lib/libs
- ./jobmanager/processWorkDir:/processWorkDir
- ./jobmanager/checkpointsDir:/checkpointsDir
- ./jobmanager/savepointsDir:/savepointsDir
- ./jobmanager/jobs:/opt/flink/usrlib
environment:
- |
FLINK_PROPERTIES=
process.working-dir: /processWorkDir
jobmanager.rpc.address: 10.0.9.75
jobmanager.host: 10.0.9.75
state.backend: rocksdb
state.checkpoints.dir: file:///checkpointsDir/
state.checkpoint-storage: filesystem
state.savepoints.dir: file:///savepointsDir/
execution.checkpointing.interval: 60000
我的java代码就像
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
StreamTableEnvironment tableEnvironment = StreamTableEnvironment.create(env, streamSettings);
tableEnvironment.executeSql("create TEMPORARY table source_table ....");
tableEnvironment.executeSql("create TEMPORARY table redis_sink ....");
final TableResult sinkResult = tableEnvironment.executeSql("insert into redis_sink select * from source_table");
sinkResult.print();
env.execute("test job");
我的命令就像:
docker-compose exec jobmanager /opt/flink/bin/flink run --detached -t remote -m 10.0.9.75:8081 -c com.test.TestJob /opt/flink/usrlib/dev2/test-0.0.1-RELEASE.jar
我使用了 --detached 选项。但是命令在显示作业 ID 后永远不会返回,虽然作业图已成功创建,但我可以使用 Ctrl+C 中断命令并且作业仍然运行数小时,但我想要的是在我的 CI 管道中使用此命令,我需要命令显示作业ID并正常返回。
我的解决方案是使用
nohup
运行deploy命令行。