单个 UPDATE Postgres 查询比应用程序代码的批量更新花费更多时间

Question

我有这个更新查询：

UPDATE articlestockhistory
SET articlerotationnature = cross_address_histories.rotation
FROM (
    SELECT
        articlestockhistory.id,
        articlestockhistory.delta,
        CASE
            WHEN articlestockhistory.delta > 0 THEN 'RECEPTION_ONLY_IN_SPECIFIC_STOCK_ADDRESS'
            ELSE 'CONSUMPTION_ONLY_IN_SPECIFIC_STOCK_ADDRESS' END
        AS rotation
    FROM stockmovementline
    INNER JOIN stocklocation source_location on stockmovementline.sourcephysicallocation_id = source_location.id
    INNER JOIN stockwarehouse source_warehouse ON source_location.warehouse_id = source_warehouse.id
    INNER JOIN stocklocation target_location on stockmovementline.targetphysicallocation_id = target_location.id
    INNER JOIN stockwarehouse target_warehouse ON target_location.warehouse_id = target_warehouse.id
    INNER JOIN articlestockhistory ON articlestockhistory.stockchangecauserid = stockmovementline.id
    WHERE
            source_warehouse.address_id != target_warehouse.address_id
      AND articlerotationoverriddenby_id IS NULL
) AS cross_address_histories
WHERE articlestockhistory.id = cross_address_histories.id;

这击中了我在文章stockhistory 中拥有的 2700 万条记录中的大约 300 万条记录。

现在，我尝试跑步并坚持跑了40分钟。我看到 postgres 一直使用大量 CPU 和磁盘，但查询没有完成。

如果我删除了articlestockhistory 表上的所有索引，大约需要4 分钟即可完成。但这在生产中是不可行的。

然后我将此 UPDATE 查询转换为应用程序代码：

• 运行 SELECT 查询，选择所有 3M 行到应用程序内存
• 使用 JDBC 的PreparedStatement，每 10000 条记录使用方法

  addBatch

  和

  executeBatch

  执行 UPDATE 查询。

大约 15 分钟就完成了，我可以在运行时查看进度。

这是在本地数据库上，没有收到其他查询。

我不明白为什么我被迫编写应用程序代码来有效地进行相同的更新。有什么见解吗？

Answer 1

另一种方法是使用 CTE 选择并更新一条语句中的相应行，然后评估性能

WITH cross_address_histories AS (
   SELECT
        articlestockhistory.id,
        articlestockhistory.delta,
        CASE
            WHEN articlestockhistory.delta > 0 THEN 'RECEPTION_ONLY_IN_SPECIFIC_STOCK_ADDRESS'
            ELSE 'CONSUMPTION_ONLY_IN_SPECIFIC_STOCK_ADDRESS' END
        AS rotation
    FROM stockmovementline
    INNER JOIN stocklocation source_location on stockmovementline.sourcephysicallocation_id = source_location.id
    INNER JOIN stockwarehouse source_warehouse ON source_location.warehouse_id = source_warehouse.id
    INNER JOIN stocklocation target_location on stockmovementline.targetphysicallocation_id = target_location.id
    INNER JOIN stockwarehouse target_warehouse ON target_location.warehouse_id = target_warehouse.id
    INNER JOIN articlestockhistory ON articlestockhistory.stockchangecauserid = stockmovementline.id
    WHERE
            source_warehouse.address_id != target_warehouse.address_id
      AND articlerotationoverriddenby_id IS NULL
)
UPDATE articlestockhistory update_articlestockhistory
SET articlerotationnature = cross_address_histories.rotation
FROM cross_address_histories
WHERE update_articlestockhistory.id = cross_address_histories.id