Groupby

示例代码：

import pandas as pd
import numpy as np
df = pd.DataFrame({
  'key1':['a', 'a', 'b', 'b', 'a'],
  'key2':['one', 'two', 'one', 'two', 'one'],
  'data1':np.random.randn(5),
  'data2':np.random.randn(5)
})
grouped = df.groupby(df['key1'])
print(grouped.mean())

------------------------------------------------------------------------------------------------------------------
         data1     data2
key1                    
a    -0.464027 -0.992397
b    -0.629515 -0.391474

• 可以使用字符串形式的列名：df.groupby('key1')
• grouped返回的是一个DataFrameGroupBy对象，他没有做任何运算，所以速度非常快。
• groupby可以接受的参数可以有很多种，比如一个list：

print df.groupby([1,2,2,3,3]).mean()
------------------------------------------------------------------------------------------------------------------

      data1     data2
1 -0.664111  0.964810
2 -0.074053  0.802726
3  0.122837 -0.035785

• 如果只需要data1的数据，以下几种写法是一样的。
  1. grouped = df['data1'].groupby(df['key1']).mean()
  2. grouped = df.groupby(df['key1'])['data1'].mean()
  3. grouped = df.groupby(df['key1']).mean()['data1']
• 返回一个层次化的结果：

print grouped = df.groupby([df['key1'],df['key2']]).mean()
------------------------------------------------------------------------------------------------------------------

              data1     data2
key1 key2                    
a    one   0.190639 -1.077724
     two   1.523810 -0.753498
b    one  -1.026170 -0.893146
     two  -0.051379  1.461553

• 可以用grouped.size()查看分组的大小，比如：

grouped = df.groupby([df['key1'],df['key2']])
print(grouped.size()) #grouped.size()是一个拥有MultiIndex的Series
print(type(grouped.size()))
print(type(grouped.size().index))
------------------------------------------------------------------------------------------------------------------

key1  key2
a     one     2
      two     1
b     one     1
      two     1
dtype: int64
<class 'pandas.core.series.Series'>
<class 'pandas.core.indexes.multi.MultiIndex'>

• 可以对group对象进行迭代

for i,j in df.groupby([df['key1'],df['key2']]):
    print(i)  # i其实是个tuple
    print('-----------')
    print(j)  # j是个DataFrame
------------------------------------------------------------------------------------------------------------------

('a', 'one')
-----------
      data1     data2 key1 key2
0  0.815046  1.269757    a  one
4 -0.604281 -0.956418    a  one
('a', 'two')
-----------
      data1     data2 key1 key2
1 -0.938286  2.636096    a  two
('b', 'one')
-----------
      data1     data2 key1 key2
2 -0.454884  0.141963    b  one
('b', 'two')
-----------
      data1     data2 key1 key2
3 -1.042242  0.618984    b  two

• 以函数作为groupby的参数：

print df.groupby(lambda x:'even' if x%2==0 else 'odd').mean()
------------------------------------------------------------------------------------------------------------------

         data1     data2
even  0.645358 -0.642165
odd   0.160585 -0.429005

这个不难理解，单数行和双数行分别作为两组进行聚合。当然，如果给df加一个字符串形式的index，这样的写法就有问题了，因为传进来的x就不能进行对2取余操作了。

• 可以根据索引进行分组

index = pd.MultiIndex.from_arrays([['even','odd','even','odd','even'],
                                  [0,1,2,3,4]],names=['a','b'])
df.index = index
print(df.groupby(level='a').mean())
------------------------------------------------------------------------------------------------------------------

         data1     data2
a                       
even -0.113491  0.730719
odd   0.076897  0.016876

• 除了内置的函数比如mean()，sum()等，还可以自定义聚合函数：

df.groupby('key1')['data1','data2'].agg(lambda arr:arr.max()-arr.min())
------------------------------------------------------------------------------------------------------------------
         data1     data2
key1                    
a     2.508309  2.334477
b     0.107973  0.203492

2.508309其实就是取出'key1==a'的所有data1的值，组成一个数组。然后用最大值减去最小值。其他3项同理。

• agg可以接受一个函数列表：

print df.groupby('key1')['data1','data2'].agg(['min','max'])
print df.groupby('key1')['data1','data2'].agg(['min','max']).columns
------------------------------------------------------------------------------------------------------------------
         data1               data2          
           min       max       min       max
key1                                        
a    -1.586040  0.922269 -1.312042  1.022435
b     0.527926  0.635899  0.279316  0.482807
MultiIndex(levels=[[u'data1', u'data2'], [u'min', u'max']],labels=[[0, 0, 1, 1], [0, 1, 0, 1]])

• 或者可以提供一个从列名到函数的映射：

print df.groupby('key1').agg({'data1':'min','data2':'max'})
------------------------------------------------------------------------------------------------------------------

         data1     data2
key1                    
a    -1.586040  1.022435
b     0.527926  0.482807

• 除了聚合，还可以进行transform()

>>> df
           data1     data2 key1 key2
a    b                              
even 0  0.922269  0.110285    a  one
odd  1 -0.181773  1.022435    a  two
even 2  0.635899  0.279316    b  one
odd  3  0.527926  0.482807    b  two
even 4 -1.586040 -1.312042    a  one
[5 rows x 4 columns]

>>> df.groupby('key1').transform('mean')
           data1     data2
a    b                    
even 0 -0.281848 -0.059774
odd  1 -0.281848 -0.059774
even 2  0.581912  0.381061
odd  3  0.581912  0.381061
even 4 -0.281848 -0.059774

其实就是.mean()以后又把结果反向传播到df

参考：https://my.oschina.net/lionets/blog/280332