pandas按组值过滤行

问题描述 投票:3回答:2

这是我正在练习的数据

import pandas as pd
df = pd.read_csv("https://raw.githubusercontent.com/mwaskom/seaborn-data/master/tips.csv")

我想按分组值过滤单个行。我知道我可以执行以下操作来过滤组

df.groupby("day").filter(lambda x: x['total_bill'].mean() > 20).day.unique()

哪个日子的平均账单大于20美元。这是因为groupby.filter将函数应用于应该返回True或False的每个子帧。但是,如果我想找到total_bill的值大于当天的total_bill的每一餐(行),该怎么办?例如,如果一行有total_bill22并且是星期日,那么它应该保留,因为周日的total_bill平均值是21.41

这是我的尝试:

df.groupby('day').apply(lambda x: x['total_bill'] > x['total_bill'].mean())

但是,这会产生一些看起来像这样的东西(前几行)

day    
Fri  90     True
     91     True
     92    False
     93    False
     94     True
Name: total_bill, dtype: bool

这与数据帧的顺序不同,所以我不能只使用布尔列并使用它来索引数据。

所以现在我做以下事情:

grouped = (df
           .groupby('day')
           .apply(lambda x: x['total_bill'] > x['total_bill'].mean())
           .reset_index())

index_bill = (grouped
             .loc[grouped.total_bill == True, 'level_1'].values)
df.loc[index_bill]

这给了我想要的结果......必须有一个更简单的方法,对吧?如果有正确的方法,请告诉我。如果没有,至少有一种方法可以将这两个步骤合二为一吗?我可以做groupby,但不知道如何在不将分组对象存储为变量然后引用它的情况下获取值。谢谢!

python pandas pandas-groupby
2个回答
1
投票

我认为最好的方法是使用groupbytransfrom的布尔索引。首先,您按天分组以查找当天的平均值,然后使用transform将该平均值应用于每一行,然后将该平均值与当天的实际total_billed进行比较,然后使用该布尔系列来使用布尔索引过滤您的数据帧。

df[df.groupby('day')['total_bill'].transform('mean') < df['total_bill']]

输出:

     total_bill   tip     sex smoker   day    time  size
3         23.68  3.31    Male     No   Sun  Dinner     2
4         24.59  3.61  Female     No   Sun  Dinner     4
5         25.29  4.71    Male     No   Sun  Dinner     4
7         26.88  3.12    Male     No   Sun  Dinner     4
11        35.26  5.00  Female     No   Sun  Dinner     4
15        21.58  3.92    Male     No   Sun  Dinner     2
19        20.65  3.35    Male     No   Sat  Dinner     3
23        39.42  7.58    Male     No   Sat  Dinner     4
28        21.70  4.30    Male     No   Sat  Dinner     2
33        20.69  2.45  Female     No   Sat  Dinner     4
35        24.06  3.60    Male     No   Sat  Dinner     3
39        31.27  5.00    Male     No   Sat  Dinner     3
44        30.40  5.60    Male     No   Sun  Dinner     4
46        22.23  5.00    Male     No   Sun  Dinner     2
47        32.40  6.00    Male     No   Sun  Dinner     4
48        28.55  2.05    Male     No   Sun  Dinner     3
52        34.81  5.20  Female     No   Sun  Dinner     4
54        25.56  4.34    Male     No   Sun  Dinner     4
56        38.01  3.00    Male    Yes   Sat  Dinner     4
57        26.41  1.50  Female     No   Sat  Dinner     2
59        48.27  6.73    Male     No   Sat  Dinner     4
72        26.86  3.14  Female    Yes   Sat  Dinner     2
73        25.28  5.00  Female    Yes   Sat  Dinner     2
77        27.20  4.00    Male     No  Thur   Lunch     4
78        22.76  3.00    Male     No  Thur   Lunch     2
80        19.44  3.00    Male    Yes  Thur   Lunch     2
83        32.68  5.00    Male    Yes  Thur   Lunch     2
85        34.83  5.17  Female     No  Thur   Lunch     4
87        18.28  4.00    Male     No  Thur   Lunch     2
88        24.71  5.85    Male     No  Thur   Lunch     2
..          ...   ...     ...    ...   ...     ...   ...
180       34.65  3.68    Male    Yes   Sun  Dinner     4
181       23.33  5.65    Male    Yes   Sun  Dinner     2
182       45.35  3.50    Male    Yes   Sun  Dinner     3
183       23.17  6.50    Male    Yes   Sun  Dinner     4
184       40.55  3.00    Male    Yes   Sun  Dinner     2
187       30.46  2.00    Male    Yes   Sun  Dinner     5
189       23.10  4.00    Male    Yes   Sun  Dinner     3
191       19.81  4.19  Female    Yes  Thur   Lunch     2
192       28.44  2.56    Male    Yes  Thur   Lunch     2
197       43.11  5.00  Female    Yes  Thur   Lunch     4
200       18.71  4.00    Male    Yes  Thur   Lunch     3
204       20.53  4.00    Male    Yes  Thur   Lunch     4
206       26.59  3.41    Male    Yes   Sat  Dinner     3
207       38.73  3.00    Male    Yes   Sat  Dinner     4
208       24.27  2.03    Male    Yes   Sat  Dinner     2
210       30.06  2.00    Male    Yes   Sat  Dinner     3
211       25.89  5.16    Male    Yes   Sat  Dinner     4
212       48.33  9.00    Male     No   Sat  Dinner     4
214       28.17  6.50  Female    Yes   Sat  Dinner     3
216       28.15  3.00    Male    Yes   Sat  Dinner     5
219       30.14  3.09  Female    Yes   Sat  Dinner     4
227       20.45  3.00    Male     No   Sat  Dinner     4
229       22.12  2.88  Female    Yes   Sat  Dinner     2
230       24.01  2.00    Male    Yes   Sat  Dinner     4
237       32.83  1.17    Male    Yes   Sat  Dinner     2
238       35.83  4.67  Female     No   Sat  Dinner     3
239       29.03  5.92    Male     No   Sat  Dinner     3
240       27.18  2.00  Female    Yes   Sat  Dinner     2
241       22.67  2.00    Male    Yes   Sat  Dinner     2
243       18.78  3.00  Female     No  Thur  Dinner     2

[97 rows x 7 columns]

1
投票

对于相同的输出需要transform返回相同的Seriesmean每组,然后过滤boolean indexing和最后添加sort_values

a=df[df['total_bill'] > df.groupby('day')['total_bill'].transform('mean')].sort_values('day')
print (a.head(20))
     total_bill   tip     sex smoker  day    time  size
90        28.97  3.00    Male    Yes  Fri  Dinner     2
91        22.49  3.50    Male     No  Fri  Dinner     2
94        22.75  3.25  Female     No  Fri  Dinner     2
95        40.17  4.73    Male    Yes  Fri  Dinner     4
96        27.28  4.00    Male    Yes  Fri  Dinner     2
98        21.01  3.00    Male    Yes  Fri  Dinner     2
102       44.30  2.50  Female    Yes  Sat  Dinner     3
206       26.59  3.41    Male    Yes  Sat  Dinner     3
229       22.12  2.88  Female    Yes  Sat  Dinner     2
227       20.45  3.00    Male     No  Sat  Dinner     4
219       30.14  3.09  Female    Yes  Sat  Dinner     4
237       32.83  1.17    Male    Yes  Sat  Dinner     2
103       22.42  3.48  Female    Yes  Sat  Dinner     2
106       20.49  4.06    Male    Yes  Sat  Dinner     2
107       25.21  4.29    Male    Yes  Sat  Dinner     2
216       28.15  3.00    Male    Yes  Sat  Dinner     5
214       28.17  6.50  Female    Yes  Sat  Dinner     3
241       22.67  2.00    Male    Yes  Sat  Dinner     2
212       48.33  9.00    Male     No  Sat  Dinner     4
211       25.89  5.16    Male    Yes  Sat  Dinner     4

编辑:

要获得days的正确排序,可以使用ordered categorical

cats = ['Mon','Tue','Wed','Thur','Fri','Sat','Sun']
df['day'] = pd.Categorical(df['day'], categories=cats, ordered=True)
means = df.groupby('day')['total_bill'].transform('mean')
df1 = df[df['total_bill'] > means].sort_values('day')
print (df1.head(20))
     total_bill   tip     sex smoker   day   time  size
129       22.82  2.18    Male     No  Thur  Lunch     3
80        19.44  3.00    Male    Yes  Thur  Lunch     2
83        32.68  5.00    Male    Yes  Thur  Lunch     2
85        34.83  5.17  Female     No  Thur  Lunch     4
87        18.28  4.00    Male     No  Thur  Lunch     2
88        24.71  5.85    Male     No  Thur  Lunch     2
89        21.16  3.00    Male     No  Thur  Lunch     2
119       24.08  2.92  Female     No  Thur  Lunch     4
125       29.80  4.20  Female     No  Thur  Lunch     6
130       19.08  1.50    Male     No  Thur  Lunch     2
78        22.76  3.00    Male     No  Thur  Lunch     2
131       20.27  2.83  Female     No  Thur  Lunch     2
141       34.30  6.70    Male     No  Thur  Lunch     6
142       41.19  5.00    Male     No  Thur  Lunch     5
143       27.05  5.00  Female     No  Thur  Lunch     6
146       18.64  1.36  Female     No  Thur  Lunch     3
191       19.81  4.19  Female    Yes  Thur  Lunch     2
192       28.44  2.56    Male    Yes  Thur  Lunch     2
197       43.11  5.00  Female    Yes  Thur  Lunch     4
200       18.71  4.00    Male    Yes  Thur  Lunch     3
© www.soinside.com 2019 - 2024. All rights reserved.