美文网首页
python pandas的使用

python pandas的使用

作者: 风___________ | 来源:发表于2021-04-26 19:14 被阅读0次

    总结:

    1. Series是一维数据,代表了DateFrame的一行数据或一列数据。是有多个不同类型的标量构成的一行数据
    2. DateFrame是二维数据,是有多行构成的,可以理解为一个表。
    3. data.loc 函数用来通过表名取表中指定数据的,第一个参数是行名称,第二个参数是列名称
    4. data.iloc 函数用来通过下标取表中指定数据的,第一个参数行下标,第二个参数是列下标
    

    数据类型:

    1. Series:带标签的一维同构数组

    (1).用值列表生成Series对象,Pandas 默认自动生成整数索引:

    s =  pd.Series([1, 3, 5, np.nan, 6, 8]) # 一列数据
    >>> s:
    0    1.0
    1    3.0
    2    5.0
    3    NaN
    4    6.0
    5    8.0
    dtype: float64
    
    
    
    

    2. DataFrame:带标签的,大小可变的,二维异构表格

    (1. )生成DataFrame对象

    # DataFrame 表格数据,包含index(行)或 columns(列)
    # 用这种方式迭代 DataFrame 的列
    for colum in df.columns:
        series = df[colum]
        # do something with series
    
    1. 用含日期时间索引与标签的 NumPy 数组生成DataFrame:
    dates = pd.date_range('20130101', periods=6)
    >>>> dates:
    DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
                   '2013-01-05', '2013-01-06'],
                  dtype='datetime64[ns]', freq='D')
    
    
    df = pd.DataFrame(np.random.randn(6, 4), index=dates, columns=list('ABCD'))
    >>>> df:
                       A         B         C         D
    2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
    2013-01-02  1.212112 -0.173215  0.119209 -1.044236
    2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
    2013-01-04  0.721555 -0.706771 -1.039575  0.271860
    2013-01-05 -0.424972  0.567020  0.276232 -1.087401
    2013-01-06 -0.673690  0.113648 -1.478427  0.524988
    
    用 Series 字典对象生成 DataFrame:
    df2 = pd.DataFrame({'A': 1.,
       ...:                     'B': pd.Timestamp('20130102'),
       ...:                     'C': pd.Series(1, index=list(range(4)), dtype='float32'),
       ...:                     'D': np.array([3] * 4, dtype='int32'),
       ...:                     'E': pd.Categorical(["test", "train", "test", "train"]),
       ...:                     'F': 'foo'})
    >>>>: df2
         A          B    C  D      E    F
    0  1.0 2013-01-02  1.0  3   test  foo
    1  1.0 2013-01-02  1.0  3  train  foo
    2  1.0 2013-01-02  1.0  3   test  foo
    3  1.0 2013-01-02  1.0  3  train  foo
    DataFrame 的列有不同数据类型:
    >>>>:df2.dtypes
    A           float64
    B    datetime64[ns]
    C           float32
    D             int32
    E          category
    F            object
    dtype: object
    

    (2.)查看数据:

    1. 查看 DataFrame 头部和尾部数据:
    
    >>>> df.head()
                       A         B         C         D
    2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
    2013-01-02  1.212112 -0.173215  0.119209 -1.044236
    2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
    2013-01-04  0.721555 -0.706771 -1.039575  0.271860
    2013-01-05 -0.424972  0.567020  0.276232 -1.087401
    
    >>>> df.tail(3)
                       A         B         C         D
    2013-01-04  0.721555 -0.706771 -1.039575  0.271860
    2013-01-05 -0.424972  0.567020  0.276232 -1.087401
    2013-01-06 -0.673690  0.113648 -1.478427  0.524988
    
    2. 显示索引与列名:
    >>> df.index
    DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
                   '2013-01-05', '2013-01-06'],
                  dtype='datetime64[ns]', freq='D')
    
    >>>> df.columns
    Index(['A', 'B', 'C', 'D'], dtype='object')
    
    

    (3.) 转化为低层 NumPy 对象:输出不包含行索引和列标签

    >>>> df.to_numpy()
    array([[ 0.4691, -0.2829, -1.5091, -1.1356],
           [ 1.2121, -0.1732,  0.1192, -1.0442],
           [-0.8618, -2.1046, -0.4949,  1.0718],
           [ 0.7216, -0.7068, -1.0396,  0.2719],
           [-0.425 ,  0.567 ,  0.2762, -1.0874],
           [-0.6737,  0.1136, -1.4784,  0.525 ]])
    
    >>>> df2.to_numpy()
    array([[1.0, Timestamp('2013-01-02 00:00:00'), 1.0, 3, 'test', 'foo'],
           [1.0, Timestamp('2013-01-02 00:00:00'), 1.0, 3, 'train', 'foo'],
           [1.0, Timestamp('2013-01-02 00:00:00'), 1.0, 3, 'test', 'foo'],
           [1.0, Timestamp('2013-01-02 00:00:00'), 1.0, 3, 'train', 'foo']], dtype=object)
    
    

    (3.)摘要:

    // 快速查看数据的统计
    >>>> df.describe()
                  A         B         C         D
    count  6.000000  6.000000  6.000000  6.000000
    mean   0.073711 -0.431125 -0.687758 -0.233103
    std    0.843157  0.922818  0.779887  0.973118
    min   -0.861849 -2.104569 -1.509059 -1.135632
    25%   -0.611510 -0.600794 -1.368714 -1.076610
    50%    0.022070 -0.228039 -0.767252 -0.386188
    75%    0.658444  0.041933 -0.034326  0.461706
    max    1.212112  0.567020  0.276232  1.071804
    // 转置数据:
    >>>> df.T
       2013-01-01  2013-01-02  2013-01-03  2013-01-04  2013-01-05  2013-01-06
    A    0.469112    1.212112   -0.861849    0.721555   -0.424972   -0.673690
    B   -0.282863   -0.173215   -2.104569   -0.706771    0.567020    0.113648
    C   -1.509059    0.119209   -0.494929   -1.039575    0.276232   -1.478427
    D   -1.135632   -1.044236    1.071804    0.271860   -1.087401    0.524988
    // 按轴排序:
    >>>> df.sort_index(axis=1, ascending=False)
                       D         C         B         A
    2013-01-01 -1.135632 -1.509059 -0.282863  0.469112
    2013-01-02 -1.044236  0.119209 -0.173215  1.212112
    2013-01-03  1.071804 -0.494929 -2.104569 -0.861849
    2013-01-04  0.271860 -1.039575 -0.706771  0.721555
    2013-01-05 -1.087401  0.276232  0.567020 -0.424972
    2013-01-06  0.524988 -1.478427  0.113648 -0.673690
    // 按值排序:
    >>>> df.sort_values(by='B')
                       A         B         C         D
    2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
    2013-01-04  0.721555 -0.706771 -1.039575  0.271860
    2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
    2013-01-02  1.212112 -0.173215  0.119209 -1.044236
    2013-01-06 -0.673690  0.113648 -1.478427  0.524988
    2013-01-05 -0.424972  0.567020  0.276232 -1.087401
    

    (4.)数据访问:.at、.iat、.loc 和 .iloc

    // 获取数据:
    >>>> df['A']
    2013-01-01    0.469112
    2013-01-02    1.212112
    2013-01-03   -0.861849
    2013-01-04    0.721555
    2013-01-05   -0.424972
    2013-01-06   -0.673690
    
    // 用 [ ] 切片行:
    >>>> df[0:3]
                       A         B         C         D
    2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
    2013-01-02  1.212112 -0.173215  0.119209 -1.044236
    2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
    >>>> df['20130102':'20130104']
                       A         B         C         D
    2013-01-02  1.212112 -0.173215  0.119209 -1.044236
    2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
    2013-01-04  0.721555 -0.706771 -1.039575  0.271860
    
    // 用标签提取一行数据:返回了Series
    >>>> df.loc[dates[0]]
    A    0.469112
    B   -0.282863
    C   -1.509059
    D   -1.135632
    Name: 2013-01-01 00:00:00, dtype: float64
    
    // 用标签选择多列数据:
    >>>> df.loc[:, ['A', 'B']]
                       A         B
    2013-01-01  0.469112 -0.282863
    2013-01-02  1.212112 -0.173215
    2013-01-03 -0.861849 -2.104569
    2013-01-04  0.721555 -0.706771
    2013-01-05 -0.424972  0.567020
    2013-01-06 -0.673690  0.113648
    
    // 标签切片,包含行与列结束点:
    >>>> df.loc['20130102':'20130104', ['A', 'B']]
                       A         B
    2013-01-02  1.212112 -0.173215
    2013-01-03 -0.861849 -2.104569
    2013-01-04  0.721555 -0.706771
    
    // 对象降维: DataFrame返回了Series(一行数据)
    >>>> df.loc['20130102', ['A', 'B']]
    A    1.212112
    B   -0.173215
    Name: 2013-01-02 00:00:00, dtype: float64
    
    // 提取标量值:返回一个横纵坐标下准确标量值
    >>>> df.loc[dates[0], 'A']
    0.46911229990718628
     # 快速访问标量,与上述方法等效:
    >>>> df.at[dates[0], 'A']
    0.46911229990718628
    
    
    // 按位置选择:
    >>>>  df.iloc[3]
    A    0.721555
    B   -0.706771
    C   -1.039575
    D    0.271860
    Name: 2013-01-04 00:00:00, dtype: float64
    >>>> df.iloc[3:5, 0:2] # 意思是返回index从3到5(不包含5) colum从0到2(不包含2)的DateFrame
                       A         B
    2013-01-04  0.721555 -0.706771
    2013-01-05 -0.424972  0.567020
    
    >>>> df.iloc[[1, 2, 4], [0, 2]] # 意思是返回index = 1 and 2 and 4 colum = 0 and 2 的DateFrame
                       A         C
    2013-01-02  1.212112  0.119209
    2013-01-03 -0.861849 -0.494929
    2013-01-05 -0.424972  0.276232
    
    >>>> df.iloc[1:3, :] # 意思是返回index从1到3(不包含3)的所有列的DateFrame
                       A         B         C         D
    2013-01-02  1.212112 -0.173215  0.119209 -1.044236
    2013-01-03 -0.861849 -2.104569 -0.494929  1.071804
    
    >>>> df.iloc[:, 1:3] # 意思是返回colum从1到3(不包含3)的所有行的DateFrame
                       B         C
    2013-01-01 -0.282863 -1.509059
    2013-01-02 -0.173215  0.119209
    2013-01-03 -2.104569 -0.494929
    2013-01-04 -0.706771 -1.039575
    2013-01-05  0.567020  0.276232
    2013-01-06  0.113648 -1.478427
    
    // 提取标量值:返回一个横纵坐标下准确标量值
    >>>> df.iloc[1, 1]
     -0.17321464905330858
    # 快速访问标量,与上述方法等效:
     >>>> df.iat[1, 1]
    -0.17321464905330858
    
    布尔索引:
    // 单列的值选择数据:
    >>>> df[df.A > 0] # 筛选'A' 这一列大于0的数据然后返回DateFrame
                       A         B         C         D
    2013-01-01  0.469112 -0.282863 -1.509059 -1.135632
    2013-01-02  1.212112 -0.173215  0.119209 -1.044236
    2013-01-04  0.721555 -0.706771 -1.039575  0.271860
    
    >>>> df[df > 0] # 筛选出标中的正数
                       A         B         C         D
    2013-01-01  0.469112       NaN       NaN       NaN
    2013-01-02  1.212112       NaN  0.119209       NaN
    2013-01-03       NaN       NaN       NaN  1.071804
    2013-01-04  0.721555       NaN       NaN  0.271860
    2013-01-05       NaN  0.567020  0.276232       NaN
    2013-01-06       NaN  0.113648       NaN  0.524988
    
    // 用 isin筛选:
    >>>> df2 = df.copy()
    >>>> df2['E'] = ['one', 'one', 'two', 'three', 'four', 'three']
    >>>> df2
                       A         B         C         D      E
    2013-01-01  0.469112 -0.282863 -1.509059 -1.135632    one
    2013-01-02  1.212112 -0.173215  0.119209 -1.044236    one
    2013-01-03 -0.861849 -2.104569 -0.494929  1.071804    two
    2013-01-04  0.721555 -0.706771 -1.039575  0.271860  three
    2013-01-05 -0.424972  0.567020  0.276232 -1.087401   four
    2013-01-06 -0.673690  0.113648 -1.478427  0.524988  three
    >>>> df2[df2['E'].isin(['two', 'four'])] # 筛选df2这个表中名为‘E’的这一列中值在['two', 'four']中的Series然后组成新的DataFrame返回
                       A         B         C         D     E
    2013-01-03 -0.861849 -2.104569 -0.494929  1.071804   two
    2013-01-05 -0.424972  0.567020  0.276232 -1.087401  four
    

    (5.) 赋值

    // 用位置赋值
    >>>> s1 = pd.Series([1, 2, 3, 4, 5, 6], index=pd.date_range('20130102', periods=6)) # 创建一个标量为[1, 2, 3, 4, 5, 6],多行一列的一组数据,s1 就是一个新增列数据
    2013-01-02    1
    2013-01-03    2
    2013-01-04    3
    2013-01-05    4
    2013-01-06    5
    2013-01-07    6
    Freq: D, dtype: int64
    >>>> df['F'] = s1 # 意思是名为df的dataFrame表,新增一列,列名为'F'的数据。
                       A         B         C          D    F
    2013-01-01 -0.861849 -2.104569 -1.509059  -1.135632  NaN
    2013-01-02  1.212112 -0.173215  0.119209  -1.044236  1.0
    2013-01-03 -0.861849 -2.104569 -0.494929   1.071804  2.0
    2013-01-04  0.721555 -0.706771 -1.039575   0.271860  3.0
    2013-01-05 -0.424972  0.567020  0.276232  -1.087401  4.0
    2013-01-06 -0.673690  0.113648 -1.478427   0.524988  5.0
    >>>> df.at[dates[0], 'A'] = 0 # 改变的是上面的  2013-01-01 'A' 对应的值-0.861849
    >>>> df.iat[0, 1] = 0   # 改变的是上面的  2013-01-01 'B' 对应的值-2.104569
    >>>> df.loc[:, 'D'] = np.array([5] * len(df)) # 按 NumPy 数组赋值,改变的是‘D’这一列, len(df)就是所有行,改成5
    >>>> df
                       A         B         C  D    F
    2013-01-01  0.000000  0.000000 -1.509059  5  NaN
    2013-01-02  1.212112 -0.173215  0.119209  5  1.0
    2013-01-03 -0.861849 -2.104569 -0.494929  5  2.0
    2013-01-04  0.721555 -0.706771 -1.039575  5  3.0
    2013-01-05 -0.424972  0.567020  0.276232  5  4.0
    2013-01-06 -0.673690  0.113648 -1.478427  5  5.0
    
    
    // 条件赋值:
    >>>> df2 = df.copy()
    >>>> df2[df2 > 0] = -df2 # df2中所有正数都变成负数
    >>>> df2
                       A         B         C  D    F
    2013-01-01  0.000000  0.000000 -1.509059 -5  NaN
    2013-01-02 -1.212112 -0.173215 -0.119209 -5 -1.0
    2013-01-03 -0.861849 -2.104569 -0.494929 -5 -2.0
    2013-01-04 -0.721555 -0.706771 -1.039575 -5 -3.0
    2013-01-05 -0.424972 -0.567020 -0.276232 -5 -4.0
    2013-01-06 -0.673690 -0.113648 -1.478427 -5 -5.0
    
    

    (6.)缺失值:np.nan 表示缺失数据

    >>>> df1 = df.reindex(index=dates[0:4], columns=list(df.columns) + ['E'])
    >>>> df1.loc[dates[0]:dates[1], 'E'] = 1
    >>>> df1
                       A         B         C  D    F    E
    2013-01-01  0.000000  0.000000 -1.509059  5  NaN  1.0
    2013-01-02  1.212112 -0.173215  0.119209  5  1.0  1.0
    2013-01-03 -0.861849 -2.104569 -0.494929  5  2.0  NaN
    2013-01-04  0.721555 -0.706771 -1.039575  5  3.0  NaN
    // 删除所有含缺失值的行:
    >>>> df1.dropna(how='any')
    
                       A         B         C  D    F    E
    2013-01-02  1.212112 -0.173215  0.119209  5  1.0  1.0
    
    // 填充缺失值:
    >>>> df1.fillna(value=5)
                       A         B         C  D    F    E
    2013-01-01  0.000000  0.000000 -1.509059  5  5.0  1.0
    2013-01-02  1.212112 -0.173215  0.119209  5  1.0  1.0
    2013-01-03 -0.861849 -2.104569 -0.494929  5  2.0  5.0
    2013-01-04  0.721555 -0.706771 -1.039575  5  3.0  5.0
    
    // 提取 nan 值的布尔掩码:
    >>>> pd.isna(df1)
                    A      B      C      D      F      E
    2013-01-01  False  False  False  False   True  False
    2013-01-02  False  False  False  False  False  False
    2013-01-03  False  False  False  False  False   True
    2013-01-04  False  False  False  False  False   True
    

    (7.)统计

    // 描述性统计
    >>>> df.mean() # 纵轴统计
    A   -0.004474
    B   -0.383981
    C   -0.687758
    D    5.000000
    F    3.000000
    dtype: float64
    >>>> df.mean(1) #另一个轴(即,行)上执行同样的操作
    2013-01-01    0.872735
    2013-01-02    1.431621
    2013-01-03    0.707731
    2013-01-04    1.395042
    2013-01-05    1.883656
    2013-01-06    1.592306
    Freq: D, dtype: float64
    
    // 不同维度对象运算时,要先对齐, 此外,Pandas 自动沿指定维度广播。
    >>>> s = pd.Series([1, 3, 5, np.nan, 6, 8], index=dates).shift(2) # shift(2): 偏移2行对齐
    2013-01-01    NaN
    2013-01-02    NaN
    2013-01-03    1.0
    2013-01-04    3.0
    2013-01-05    5.0
    2013-01-06    NaN
    Freq: D, dtype: float64
    
    >>>> df.sub(s, axis='index') # 啥意思是按列运算,NaN+ 标量 = NaN,按列为组运算,index相同的做和,返回一个新的dataframe,不会改变df,只是用df运算。
                       A         B         C    D    F
    2013-01-01       NaN       NaN       NaN  NaN  NaN
    2013-01-02       NaN       NaN       NaN  NaN  NaN
    2013-01-03 -1.861849 -3.104569 -1.494929  4.0  1.0 # -2.861849+1 = -1.861849
    2013-01-04 -2.278445 -3.706771 -4.039575  2.0  0.0
    2013-01-05 -5.424972 -4.432980 -4.723768  0.0 -1.0
    2013-01-06       NaN       NaN       NaN  NaN  NaN
    
    // Apply 函数处理数据
    >>>> df.apply(np.cumsum) # np.cumsum 是一个函数,列累加的函数,apply和.sub一样,返回一个新的dataframe,不会改变df,只是用df运算。
                       A         B         C   D     F
    2013-01-01  0.000000  0.000000 -1.509059   5   NaN # 0.000000 =  df.iat[0,0]
    2013-01-02  1.212112 -0.173215 -1.389850  10   1.0 # 1.212112 = df.iat[0,0] + df.iat[0,1]
    2013-01-03  0.350263 -2.277784 -1.884779  15   3.0 # -0.350263 = df.iat[0,1] + df.iat[0,2]
    2013-01-04  1.071818 -2.984555 -2.924354  20   6.0 # ...
    2013-01-05  0.646846 -2.417535 -2.648122  25  10.0 # ...
    2013-01-06 -0.026844 -2.303886 -4.126549  30  15.0 # ...
    >>>> df.apply(lambda x: x.max() - x.min()) # 先找到df每列的最大值和最小值,然后做差,然后返回一个Series,列统计。
    A    2.073961 #  1.212112 - (-0.861849) = 2.073961
    B    2.671590 
    C    1.785291 
    D    0.000000 
    F    4.000000
    dtype: float64
    
    

    (8.)直方图

    >>>> test = np.random.randint(0, 7, size=10)
    >>>> test
    array([4, 2, 1, 2, 6, 4, 4, 6, 4, 4])
    
    >>>>s = pd.Series(test) 
    0    4
    1    2
    2    1
    3    2
    4    6
    5    4
    6    4
    7    6
    8    4
    9    4
    dtype: int64
    
    >>>> s.value_counts() # 4 出现了5次 6 出现了2次 。。。。。
    4    5
    6    2
    2    2
    1    1
    dtype: int64
    

    (9.)字符串方法:Series 的 str 属性包含一组字符串处理功能

    >>>> s = pd.Series(['A', 'B', 'C', 'Aaba', 'Baca', np.nan, 'CABA', 'dog', 'cat'])
    >>>> s.str.lower()
    0       a
    1       b
    2       c
    3    aaba
    4    baca
    5     NaN
    6    caba
    7     dog
    8     cat
    dtype: object
    

    (10.)合并

    >>> test = np.random.randn(10, 4)
    >>> test
    array([[-0.81741352,  1.84903603,  0.49850415,  0.44634968],
           [-0.36911923, -0.62801459,  1.17384241,  0.10961811],
           [-2.02391694, -1.13128739,  0.23789725, -0.34718204],
           [ 0.48488279, -1.99957237, -0.11090257, -0.00836397],
           [ 1.12524972, -1.06405041,  0.23928546,  1.49336223],
           [-0.72348469, -0.47378583, -0.34307547, -0.35879956],
           [-1.34686762,  1.24929498,  0.66233861,  0.262106  ],
           [-0.38460465, -0.02032119, -1.16112247,  0.11717697],
           [-0.22309295, -0.14498124,  0.01653987, -0.92529958],
           [ 0.78572275,  1.85298325, -0.85724784,  0.08748715]])
    >>> df = pd.DataFrame(test)
    >>> df
              0         1         2         3
    0 -0.817414  1.849036  0.498504  0.446350
    1 -0.369119 -0.628015  1.173842  0.109618
    2 -2.023917 -1.131287  0.237897 -0.347182
    3  0.484883 -1.999572 -0.110903 -0.008364
    4  1.125250 -1.064050  0.239285  1.493362
    5 -0.723485 -0.473786 -0.343075 -0.358800
    6 -1.346868  1.249295  0.662339  0.262106
    7 -0.384605 -0.020321 -1.161122  0.117177
    8 -0.223093 -0.144981  0.016540 -0.925300
    9  0.785723  1.852983 -0.857248  0.087487
    
    >>>> pieces = [df[:3], df[3:7], df[7:]]# 分解为多组,从第0行到第3行(不包括3),。。。pieces是dataframe数组
    >>>> pd.concat(pieces)
              0         1         2         3
    0 -0.817414  1.849036  0.498504  0.446350
    1 -0.369119 -0.628015  1.173842  0.109618
    2 -2.023917 -1.131287  0.237897 -0.347182
    3  0.484883 -1.999572 -0.110903 -0.008364
    4  1.125250 -1.064050  0.239285  1.493362
    5 -0.723485 -0.473786 -0.343075 -0.358800
    6 -1.346868  1.249295  0.662339  0.262106
    7 -0.384605 -0.020321 -1.161122  0.117177
    8 -0.223093 -0.144981  0.016540 -0.925300
    9  0.785723  1.852983 -0.857248  0.087487
    
    

    (11.)join连接 :SQL 风格的合并

    >>>> left = pd.DataFrame({'key': ['foo', 'foo'], 'lval': [1, 2]})
    >>>> right = pd.DataFrame({'key': ['foo', 'foo'], 'rval': [4, 5]})
    >>> left
       key  lval
    0  foo     1
    1  foo     2
    >>> right
       key  rval
    0  foo     4
    1  foo     5
    
    >>>> pd.merge(left, right, on='key') # 14 15 24 25 
       key  lval  rval
    0  foo     1     4
    1  foo     1     5
    2  foo     2     4
    3  foo     2     5
    
    
    >>>> left = pd.DataFrame({'key': ['foo', 'bar'], 'lval': [1, 2]})
    >>>> right = pd.DataFrame({'key': ['foo', 'bar'], 'rval': [4, 5]})
    >>>>  left
       key  lval
    0  foo     1
    1  bar     2
    >>>> right
       key  rval
    0  foo     4
    1  bar     5
    >>>> pd.merge(left, right, on='key')
       key  lval  rval
    0  foo     1     4
    1  bar     2     5
    

    (12.)Append 追加:

    >>> df = pd.DataFrame(np.random.randn(8, 4), columns=['A', 'B', 'C', 'D'])
    >>> s = df.iloc[3] # index = 3 的一行数据,一个Series
    >>> df.append(s, ignore_index=True) # 添加了一条数据(index = 8的那条),index = 3 和 index = 8 
              A         B         C         D
    0  1.346061  1.511763  1.627081 -0.990582
    1 -0.441652  1.211526  0.268520  0.024580
    2 -1.577585  0.396823 -0.105381 -0.532532
    3  1.453749  1.208843 -0.080952 -0.264610
    4 -0.727965 -0.589346  0.339969 -0.693205
    5 -0.339355  0.593616  0.884345  1.591431
    6  0.141809  0.220390  0.435589  0.192451
    7 -0.096701  0.803351  1.715071 -0.708758
    8  1.453749  1.208843 -0.080952 -0.264610
    
    

    (13.)分组(Grouping)

    >>> df = pd.DataFrame({'A': ['foo', 'bar', 'foo', 'bar',
       ....:                          'foo', 'bar', 'foo', 'foo'],
       ....:                    'B': ['one', 'one', 'two', 'three',
       ....:                          'two', 'two', 'one', 'three'],
       ....:                    'C': np.random.randn(8),
       ....:                    'D': np.random.randn(8)})
    
         A      B         C         D
    0  foo    one -1.202872 -0.055224
    1  bar    one -1.814470  2.395985
    2  foo    two  1.018601  1.552825
    3  bar  three -0.595447  0.166599
    4  foo    two  1.395433  0.047609
    5  bar    two -0.392670 -0.136473
    6  foo    one  0.007207 -0.561757
    7  foo  three  1.928123 -1.623033
    
    >>>> df.groupby('A').sum() # 根据'A'分组,然后用sum计算值,返回新的dataframe
                C        D
    A                     
    bar -2.802588  2.42611
    foo  3.146492 -0.63958
    >>>> df.groupby(['A', 'B']).sum() # 多列分组后,根据'A'and  'B' 分组,然后用sum计算值,生成多层索引,返回新的dataframe
                      C         D
    A   B                        
    bar one   -1.814470  2.395985
        three -0.595447  0.166599
        two   -0.392670 -0.136473
    foo one   -1.195665 -0.616981
        three  1.928123 -1.623033
        two    2.414034  1.600434
    

    (14.)重塑(Reshaping) + 数据透视表(Pivot Tables)+ 堆叠(Stack): 。。。不看了不看了,感觉用不上啊(https://www.pypandas.cn/docs/getting_started/10min.html#%E9%87%8D%E5%A1%91-reshaping

    (16.) 时间序列:

    >>> rng = pd.date_range('1/1/2012', periods=100, freq='S')
    >>> ts = pd.Series(np.random.randint(0, 500, len(rng)), index=rng)
    >>> ts.resample('5Min').sum() # 意思是把 将秒级的数据转换为 5 分钟为频率的数据,用sum()把数据合并
    2012-01-01    25083
    Freq: 5T, dtype: int64
    
    >>> rng = pd.date_range('3/6/2012 00:00', periods=5, freq='D')
    >>> ts = pd.Series(np.random.randn(len(rng)), rng)
    >>> ts
    2012-03-06    0.464000
    2012-03-07    0.227371
    2012-03-08   -0.496922
    2012-03-09    0.306389
    2012-03-10   -2.290613
    >>>ts_utc = ts.tz_localize('UTC')
    >>>ts_utc
    2012-03-06 00:00:00+00:00    0.464000
    2012-03-07 00:00:00+00:00    0.227371
    2012-03-08 00:00:00+00:00   -0.496922
    2012-03-09 00:00:00+00:00    0.306389
    2012-03-10 00:00:00+00:00   -2.290613
    Freq: D, dtype: float64
    // 转换成其它时区
    >>>ts_utc.tz_convert('US/Eastern')
    2012-03-05 19:00:00-05:00    0.464000
    2012-03-06 19:00:00-05:00    0.227371
    2012-03-07 19:00:00-05:00   -0.496922
    2012-03-08 19:00:00-05:00    0.306389
    2012-03-09 19:00:00-05:00   -2.290613
    // 转换时间段:
    >>>rng = pd.date_range('1/1/2012', periods=5, freq='M')
    >>>ts = pd.Series(np.random.randn(len(rng)), index=rng)
    >>>ts
    2012-01-31   -1.134623
    2012-02-29   -1.561819
    2012-03-31   -0.260838
    2012-04-30    0.281957
    2012-05-31    1.523962
    Freq: M, dtype: float64
    >>>ps = ts.to_period()
    2012-01   -1.134623
    2012-02   -1.561819
    2012-03   -0.260838
    2012-04    0.281957
    2012-05    1.523962
    >>>ps.to_timestamp()
    2012-01-01   -1.134623
    2012-02-01   -1.561819
    2012-03-01   -0.260838
    2012-04-01    0.281957
    2012-05-01    1.523962
    
    eg:
    // Pandas 函数可以很方便地转换时间段与时间戳。下例把以 11 月为结束年份的季度频率转换为下一季度月末上午 9 点:
    >>> prng = pd.period_range('1990Q1', '2000Q4', freq='Q-NOV')
    >>> ts = pd.Series(np.random.randn(len(prng)), prng)
    >>> ts.index = (prng.asfreq('M', 'e') + 1).asfreq('H', 's') + 9
    
    >>> ts.head()
    1990-03-01 09:00   -0.902937
    1990-06-01 09:00    0.068159
    1990-09-01 09:00   -0.057873
    1990-12-01 09:00   -0.368204
    1991-03-01 09:00   -1.144073
    Freq: H, dtype: float64
    

    (17.)类别型(Categoricals) + 可视化 + HDF5+Excel 都先不看了 用到在说

    (18.)数据输入 / 输出

     // 写入csv文件
    >>> df.to_csv('foo.csv')
    
    // 读取 CSV 文件数据
    >>> pd.read_csv('foo.csv')
    

    ✨✨✨✨:DataFrame 是 Series 的容器,Series 则是标量的容器

    相关文章

      网友评论

          本文标题:python pandas的使用

          本文链接:https://www.haomeiwen.com/subject/mycerltx.html