SQL计算组中的异常值数

问题描述 投票:0回答:1

我想计算每个group by中有多少元素大于$ \ mu + \ sigma $,$ \ mu + 2 \ sigma $等等。

到目前为止,我找到了一个解决方案,首先使用$ \ mu $和$ \ sigma $创建一个表gp

CREATE TABLE gp AS
SELECT col_a,
       col_b,
       AVG(y) AS y_mean,
       STDDEV(y) AS y_std
FROM my_table
GROUP BY col_a, col_b;

然后我用原始表做left join并通过计数

SELECT col_a,
       col_b,
       SUM(CASE
             WHEN y>y_mean+y_std THEN 1
             ELSE 0
           END) AS std1,
       SUM(CASE
             WHEN y>y_mean+2*y_std THEN 1
             ELSE 0
           END) AS std2,
       SUM(CASE
             WHEN y>y_mean+3*y_std THEN 1
             ELSE 0
           END) AS std3, 
FROM (
SELECT a.*,
       b.y_mean,
       b.y_std
FROM(
(SELECT col_a,
       col_b,
       y
FROM my_table) a
LEFT JOIN (SELECT * FROM gp) b
ON a.col_a=b.col_a AND a.col_b=b.col_b)
)
GROUP BY col_a, col_b

我想知道是否有更有效的方法来实现相同的结果。

sql amazon-redshift
1个回答
1
投票

使用窗口功能:

SELECT col_a, col_b,
       SUM(CASE WHEN y > y_mean + y_std THEN 1 ELSE 0
           END) AS std1,
       SUM(CASE WHEN y > y_mean + 2 * y_std THEN 1 ELSE 0
           END) AS std2,
       SUM(CASE WHEN y > y_mean + 3 * y_std THEN 1 ELSE 0
           END) AS std3
FROM (SELECT t.*,
             AVG(y) OVER (PARTITION BY col_a, col_b) as y_mean,
             STDDEV(y) OVER (PARTITION BY col_a, col_b) as y_std
      FROM my_table t
     ) t
GROUP BY col_a, col_b;

从统计角度来看,您也应该关注下限。如果分布仅在正方向上倾斜,则标准差可能不是最佳度量(尽管在使用数据库时没有太多选择)。

© www.soinside.com 2019 - 2024. All rights reserved.