自动删除 QuestDB 上的分区

Question

我在 QuestDB 数据库上有几个按每日分区的时间序列表。我想每天从我的表中删除所有超过 21 天的分区。最好的方法是什么？

谢谢

Answer 1

您可以使用

ALTER TABLE...DROP PARTITION

从特定表中删除旧分区。您可以通过 pgwire API 或 REST API 执行此命令。使用 cronjob 时，API 可能是最方便的方法，您可以执行类似的操作

curl -G --data-urlencode "query=ALTER TABLE measurements DROP PARTITION WHERE timestamp < to_timestamp('2018-01-01:00:00:00', 'yyyy-MM-dd:HH:mm:ss');" http://localhost:9000/exec

为了让它更有趣，您可以按每日分区查看数据库中的所有表，并自动计算要删除的日期范围。我正在粘贴一个简单的 bash 脚本来执行此操作。该脚本依赖于

curl

和

jq

并已在 Ubuntu 和 OSx 上进行了测试。

#!/bin/bash

# This script needs both curl and jq installed.
# It will go over all the tables with daily partitioning and will remove all partitions older than 21 days 
# It uses jq to parse the JSON output from the REST API, extracting the "dataset" element and flatten all the rows.
# Then it reads line by line and calls the QuestDB API with each ALTER TABLE statement.


# We get all the tables with daily partitioning and compose the ALTER TABLE statements

TABLES=`curl -G  --data-urlencode "query=with daily_tables AS (
select name, designatedTimestamp, timestamp_floor('d',dateadd('d', -21, systimestamp())) as deadline from tables where partitionBy = 'DAY'
)
select CONCAT('ALTER TABLE ', name, ' DROP PARTITION WHERE ', designatedTimestamp, ' <= ', deadline) FROM daily_tables;" "http://localhost:9000/exec?nm=true"|jq ".dataset | flatten[]"`


# Splitting the output line by line and issuing the ALTER TABLE
printf '%s\n' "$TABLES" |
while IFS= read -r sql; do
        # echo $sql #uncomment if you want to output each statement we are sending
        #we need to remove starting and trailing double quote from the line, so using :1:-1 syntax
        curl -G --data-urlencode "query=${sql:1:-1}" http://localhost:9000/exec
done