如何从配置文件中获取增量数据以仅获取最近 5 天的数据

Question

目前我正在使用 python 中的配置文件从 api 获取增量数据。下面是配置文件的片段

[
 {
   "id" : 123,
   "input":{
    "name" :    ,
    "path" :     ,
    "format":     , 
    "paramters":{
                 "table_name" : "test"
      },
     "query" : "select * from test where type_id='1234' and time>= '2023-04-20T00:46:26+00:00'"
  },
]

现在我正在手动更改 sql 查询的时间路径，但我想要一个不需要手动更改日期的解决方案。

我确实尝试了很多不同的方法，但最后我的输出文件没有任何价值，除非我给出日期

1. time >= 'DATE(NOW())-INTERVAL 7 DAY'

2. time = DateDiff(left(time,19), today(), day) <5 - this should ideally give incremental value in sql but here i am not able to pass this statement in python

Answer 1

你不必把它变成一个json，但是你解析一个json配置可能会更容易，因为它可以变成一个字典，使得它很容易解析。但如果你想要的话，txt 文件也可以工作。

#import datetime module
import datetime
#Get datetime for right now
tod = datetime.datetime.now()
#create time delta of how many days to go back
d = datetime.timedelta(days = 5)
#simple subtraction
a = tod - d
#format the date to the format you want. The 00:00 at the end is to just set milliseconds to 0 because datetime can't parse ms like that (I think?)
formatted_string = a.strftime("%Y-%m-%dT%H:%M:%S+00:00")
print(formatted_string)
#prints 2023-04-19T18:39:38+00:00 (ie. 5 days before when I ran this)

#I will assume here you open and read your config file
#assuming query looks like this in the config: "query" : "select * from test where type_id='1234' and time>= "
#query variable is your query from the config file
#Initialising query below because I don't have your config file
query = "select * from test where type_id='1234' and time>= "
query += formatted_string
print(query)
#prints select * from test where type_id='1234' and time>= 2023-04-19T18:42:23+00:00

#now you can write this back to your config and do whatever else is needed