从 SQL Server 中的表中提取增量数据的 SQL 查询

Question

我正在创建一个 AWS 粘合作业，以将数据从 OLTP 数据库提取到 Redshift 数据库中。我希望 SQL 查询从表中提取增量数据。 我的表有 CreatedOn 和 LastUpdatedOn 等列来跟踪表中所做的更改。

CREATE TABLE [dbo].[Table](
    [Id] [int] IDENTITY(1,1) NOT NULL,
    [CreatedOn] [datetimeoffset](7) NOT NULL,
    [CreatedBy] [nvarchar](255) NOT NULL,
    [LastUpdatedBy] [nvarchar](255) NOT NULL,
    [LastUpdatedOn] [datetimeoffset](7) NOT NULL,
    [RowVersion] [timestamp] NOT NULL,
    [Type] [nvarchar](128) NULL,
    [Title] [nvarchar](256) NULL,
    [FirstName] [nvarchar](256) NULL,
    [MiddleName] [nvarchar](256) NULL,
    [HasMiddleName] [bit] NULL,
    [Surname] [nvarchar](256) NULL,
    [DateOfBirth] [datetime] NULL,
    [Gender] [nvarchar](16) NULL,
 CONSTRAINT [PK_Table] PRIMARY KEY CLUSTERED 
 (
    [Id] ASC
 )
)

以下是满载查询

SELECT 
        Id
        ,CONVERT(VARCHAR, CreatedOn, 20) AS CreatedOn
        ,CreatedBy
        ,LastUpdatedBy
        ,CONVERT(VARCHAR, LastUpdatedOn, 20) AS LastUpdatedOn
        ,Type
        ,Title
        ,TRIM(REPLACE(FirstName,CHAR(10),' ')) AS FirstName
        ,TRIM(REPLACE(MiddleName,CHAR(10),' ')) AS MiddleName
        ,HasMiddleName
        ,REPLACE(Surname,CHAR(10),' ') AS Surname
        ,TRIM(CONVERT(VARCHAR, DateOfBirth, 20)) AS DateOfBirth
        ,Gender
    FROM
        table

Answer 1

获取上次成功运行作业的执行时间并对其进行过滤，如下所示

SELECT 
        Id
        ,CONVERT(VARCHAR, CreatedOn, 20) AS CreatedOn
        ,CreatedBy
        ,LastUpdatedBy
        ,CONVERT(VARCHAR, LastUpdatedOn, 20) AS LastUpdatedOn
        ,Type
        ,Title
        ,TRIM(REPLACE(FirstName,CHAR(10),' ')) AS FirstName
        ,TRIM(REPLACE(MiddleName,CHAR(10),' ')) AS MiddleName
        ,HasMiddleName
        ,REPLACE(Surname,CHAR(10),' ') AS Surname
        ,TRIM(CONVERT(VARCHAR, DateOfBirth, 20)) AS DateOfBirth
        ,Gender
    FROM table
WHERE CreatedOn > '<LastSuccessful Job Execution Time>'
OR LastUpdatedOn > '<LastSuccessful Job Execution Time>'