【pandas】goupby工具使用

1.init a dataframe

import pandas as pd
import numpy as np

df = pd.DataFrame({'key1':['a', 'a', 'b', 'b', 'a','a','a','a'],
                   'key2':['one', 'two', 'one', 'two', 'one','two','two','two'],
                   'data1':np.random.randn(8),
                   'data2':np.random.randn(8)})
df.head(10)

2.data1按key1，key2分组后的平均值

df_group = df.groupby(["key1","key2"])["data1"].mean()
print(df_group)

result:

key1  key2
a     one     1.197223
      two    -0.224934
b     one     1.779484
      two     1.193350
Name: data1, dtype: float64

3.group 操作后还恢复到正常dataframe索引

df_reset = df_group.reset_index()
df_reset.head()

result:

key1    key2    data1
0   a   one 1.197223
1   a   two -0.224934
2   b   one 1.779484
3   b   two 1.193350

4.同一组数据做多类型数据统计

df_group_m = df.groupby(["key1","key2"])["data1"].agg(["mean","max"])
df_reset_m = df_group_m.reset_index()
print(df_group_m)
df_reset_m.head()

result:

               mean       max
key1 key2                    
a    one   1.197223  2.319615
     two  -0.224934 -0.173460
b    one   1.779484  1.779484
     two   1.193350  1.357037

key1    key2    mean    max
0   a   one 1.197223    2.319615
1   a   two -0.224934   -0.173460
2   b   one 1.779484    1.779484
3   b   two 1.193350    1.357037

5.自定义统计函数

系统提供了丰富的统计函数，比如：最大值、最小值、count、求和、平均值等常见的统计属性，但是有时候仍然不能满足我们的需求，需要自己给grouby写统计函数（这里要特别小心，在数据量大时，任何的时间消耗都会被放大，统计要尽可能简单）。
比如要获取大于平均数的数据的中位数附近的数据。

def median_m(arrs):
    length = len(arrs) - 1
    idx = int(0.75*length)
    return arrs.iat[idx]
    
df.sort_values(by=["data1"],inplace=True,ascending=True)
df_group_d = df.groupby(["key1","key2"])["data1"].agg(["mean",median_m])
df_reset_d = df_group_d.reset_index()
print(df_group_d)
df_reset_d.head()

result:

               mean  median_m
key1 key2                    
a    one   1.223088  0.146005
     two  -1.033850 -0.334123
b    one  -0.194909 -0.194909
     two  -1.798123 -1.798123

key1    key2    mean    median_m
0   a   one 1.223088    0.146005
1   a   two -1.033850   -0.334123
2   b   one -0.194909   -0.194909
3   b   two -1.798123   -1.798123