Pandas简介
Pandas 是基于NumPy 的一种工具，该工具是为了解决数据分析任务而创建的。Pandas 纳入了大量库和一些标准的数据模型，提供了高效地操作大型数据集所需的工具。pandas提供了大量能使我们快速便捷地处理数据的函数和方法。
Series对象
Pandas基于两种数据类型：series与dataframe。
Series是Pandas中最基本的对象，Series类似一种一维数组。事实上，Series 基本上就是基于 NumPy 的数组对象来的。和 NumPy 的数组不同，Series 能为数据自定义标签，也就是索引（index），然后通过索引来访问数组中的数据。
Dataframe是一个二维的表结构。Pandas的dataframe可以存储许多种不同的数据类型，并且每一个坐标轴都有自己的标签。你可以把它想象成一个series的字典项。
import pandas as pd
from pandas import Series,DataFrame
import numpy as np
创建series
index 参数是可省略的，你可以选择不输入这个参数。
如果不带 index 参数，Pandas 会自动用默认 index 进行索引，类似数组，索引值是 [0, ..., len(data) - 1]
sel=Series([1,2,3,4])
print(sel)
0 1
1 2
2 3
3 4
dtype: int64
#通常创建方式
sel=Series([1,2,3,4],index=['a','b','c','d'])
#sel=Series([1,2,3,4],index=list('abcd'))
print(sel)
a 1
b 2
c 3
d 4
dtype: int64
print(sel.values) #获取索引值
print(sel.index) #获取检索值
print(sel.iteritems()) #获取索引和值对
[1 2 3 4]
Index(['a', 'b', 'c', 'd'], dtype='object')
<zip object at 0x00000000085838C8>
字典转化为Series
dict={"red":100,"black":400,"green":300,"pink":900}
se3=Series(dict)
print(se3)
red 100
black 400
green 300
pink 900
dtype: int64
Series数据获取
# Series对象同时支持位置和标签两种方式获取数据
print('索引下标',sel['c'])
print('位置下标',sel[2])
索引下标 3
位置下标 3
#获取不连续的数据
print('索引下标',sel[['a','c']])
print('位置下标',sel[[1,3]])
索引下标 a 1
c 3
dtype: int64
位置下标 b 2
d 4
dtype: int64
#可以使用切片获取数据
print('索引下标',sel['a':'c'])
print('位置下标',sel[0:3])
索引下标 a 1
b 2
c 3
dtype: int64
位置下标 a 1
b 2
c 3
dtype: int64
#重新赋值索引值
sel.index=['d','f','h','g']
print(sel)
d 1
f 2
h 3
g 4
dtype: int64
#ReIndex重新索引,会返回一个新的Series(调用reindex将会重新排序，缺失值则用NaN填补)
print(sel.reindex(['b','a','c','d','e']))
print(sel)
b NaN
a NaN
c NaN
d 1.0
e NaN
dtype: float64
d 1
f 2
h 3
g 4
dtype: int64
#Drop丢弃指定轴的项
se1=pd.Series(range(10,15))
print(se1)
print(se1.drop([2,3]))
0 10
1 11
2 12
3 13
4 14
dtype: int64
0 10
1 11
4 14
dtype: int64
Series 进行算术运算操作
对 Series 的算术运算都是基于 index 进行的。
我们可以用加减乘除（+ - * /）这样的运算符对两个 Series 进行运算，
Pandas 将会根据索引 index，对响应的数据进行计算，结果将会以浮点数的形式存储，以避免丢失精度。
如果 Pandas 在两个 Series 里找不到相同的 index，对应的位置就返回一个空值 NaN
series1 = pd.Series([1,2,3,4],['London','HongKong','Humbai','lagos'])
series2 = pd.Series([1,3,6,4],['London','Accra','lagos','Delhi'])
print(series1+series2)
print('-'*20)
print(series1+series2)
print('-'*20)
print(series1*series2)
Accra NaN
Delhi NaN
HongKong NaN
Humbai NaN
London 2.0
lagos 10.0
dtype: float64
--------------------
Accra NaN
Delhi NaN
HongKong NaN
Humbai NaN
London 2.0
lagos 10.0
dtype: float64
--------------------
Accra NaN
Delhi NaN
HongKong NaN
Humbai NaN
London 1.0
lagos 24.0
dtype: float64
series支持numpy数组运算
sel =Series(data = [1,6,3,5], index =['a','b','c','d'])
print(sel[sel>3])#布尔数组过滤
print('-'*20)
print(sel*2)
print('-'*20)
print(np.square(sel))#直接加入到numpy的数学函数
b 6
d 5
dtype: int64
--------------------
a 2
b 12
c 6
d 10
dtype: int64
--------------------
a 1
b 36
c 9
d 25
dtype: int64
DataFrame
DataFrame（数据表）是一种 2 维数据结构，数据以表格的形式存储，分成若干行和列。通过 DataFrame，你能很方便地处理数据。常见的操作比如选取、替换行或列的数据，还能重组数据表、修改索引、多重筛选等。我们基本上 可以把 DataFrame 理解成一组采用同样索引的 Series 的集合。调用DataFrame()可以将多种格式的数据转换为
DataFrame对象，它的的三个参数data、index和columns分别为数据、行索引和列索引。
DataFrame的创建
df1 = DataFrame(np.random.randint(0,10,(4,4)),index=[1,2,3,4],columns=['a','b','c','d'])
print(df1)
a b c d
1 9 3 7 2
2 8 8 4 3
3 5 5 7 0
4 4 0 7 2
字典创建
行索引由index决定，列索引由字典的键决定
dict={
'Province': ['Guangdong', 'Beijing', 'Qinghai', 'Fujian'],
'pop': [1.3, 2.5, 1.1, 0.7],
'year': [2018, 2018, 2018, 2018]}
df2=pd.DataFrame(dict,index=[1,2,3,4])
print(df2)
Province pop year
1 Guangdong 1.3 2018
2 Beijing 2.5 2018
3 Qinghai 1.1 2018
4 Fujian 0.7 2018
使用from_dict
dict2={'a':[1,2,3],'b':['a','b','c']}
df3=pd.DataFrame.from_dict(dict2)
print(df3)
a b
0 1 a
1 2 b
2 3 c
#索引相同的情况下，相同索引的值会相对应，缺少的值会添加NaN
data = {
'Name':pd.Series(['zs','ls','we'],index=['a','b','c']),
'Age':pd.Series(['10','20','30','40'],index=['a','b','c','d']),
'country':pd.Series(['中国','日本','韩国'],index=['a','c','b'])
}
df = pd.DataFrame(data)
print(df)
# to_dict()方法将DataFrame对象转换为字典
dict = df.to_dict()
print(dict)
Name Age country
a zs 10 中国
b ls 20 韩国
c we 30 日本
d NaN 40 NaN
{'Name': {'a': 'zs', 'b': 'ls', 'c': 'we', 'd': nan}, 'Age': {'a': '10', 'b': '20', 'c': '30', 'd': '40'}, 'country': {'a': '中国', 'b': '韩国', 'c': '日本', 'd': nan}}
DataFrame对象常用属性
import pandas as pd
from pandas import Series,DataFrame
import numpy as np
#数据
df_dict = {
'name':['James','Curry','Iversion'],
'age':['18','20','19'],
'national':['us','China','us']
}
df=pd.DataFrame(df_dict,index=[0,1,2])
print(df)
name age national
0 James 18 us
1 Curry 20 China
2 Iversion 19 us
#获取行列数
print(df.shape)
(3, 3)
#获取行索引
print(df.index.tolist())
[0, 1, 2]
#获取列索引
print(df.columns.tolist())
['name', 'age', 'national']
#获取数据类型
print(df.dtypes)
name object
age object
national object
dtype: object
#获取数据维度
print(df.ndim)
2
#values属性也会以二维ndarray的形式返回DataFrame的数据
print(df.values)
[['James' '18' 'us']
['Curry' '20' 'China']
['Iversion' '19' 'us']]
#展示概况
print(df.info())
<class 'pandas.core.frame.DataFrame'>
Int64Index: 3 entries, 0 to 2
Data columns (total 3 columns):
name 3 non-null object
age 3 non-null object
national 3 non-null object
dtypes: object(3)
memory usage: 96.0+ bytes
None
#显示头两行，默认头五行
print(df.head(2))
name age national
0 James 18 us
1 Curry 20 China
#显示后几行
print(df.tail(2))
name age national
1 Curry 20 China
2 Iversion 19 us
#获取DataFrame的列
print(df['name'])
print('-'*20)
#因为我们只获取一列，所以返回的就是一个 Series
print(type(df['name']))
print('-'*20)
# 如果获取多个列，那返回的就是一个 DataFrame 类型：
print(df[['name','age']])
print(type(df[['name','age']]))
0 James
1 Curry
2 Iversion
Name: name, dtype: object
--------------------
<class 'pandas.core.series.Series'>
--------------------
name age
0 James 18
1 Curry 20
2 Iversion 19
<class 'pandas.core.frame.DataFrame'>
# 获取一行
print(df[0:1])
print('-'*20)
# 获取多行
print(df[1:3])
print('-'*20)
# 取多行里面的某一列（不能进行多行多列的选择）
print(df[1:3][['name','age']])
# 注意： df[]只能进行行选择，或列选择，不能同时多行多列选择。
name age national
0 James 18 us
--------------------
name age national
1 Curry 20 China
2 Iversion 19 us
--------------------
name age
1 Curry 20
2 Iversion 19
'''df.loc 通过标签索引行数据
df.iloc 通过位置获取行数据
'''
# 获取某一行某一列的数据
print(df.loc[0,'name'])
print('-'*20)
# 一行所有列
print(df.loc[0,:])
print('-'*20)
# 某一行多列的数据
print(df.loc[0,['name','age']])
print('-'*20)
# 选择间隔的多行多列
print(df.loc[[0,2],['name','national']])
print('-'*20)
# 选择连续的多行和间隔的多列
print(df.loc[0:2,['name','national']])
print('-'*20)
# 取一行
print(df.iloc[1])
print('-'*20)
# 取连续多行
print(df.iloc[0:2])
print('-'*20)
# 取间断的多行
print(df.iloc[[0,2],:])
print('-'*20)
# 取某一列
print(df.iloc[:,1])
print('-'*20)
# 某一个值
print(df.iloc[1,0])
James
--------------------
name James
age 18
national us
Name: 0, dtype: object
--------------------
name James
age 18
Name: 0, dtype: object
--------------------
name national
0 James us
2 Iversion us
--------------------
name national
0 James us
1 Curry China
2 Iversion us
--------------------
name Curry
age 20
national China
Name: 1, dtype: object
--------------------
name age national
0 James 18 us
1 Curry 20 China
--------------------
name age national
0 James 18 us
2 Iversion 19 us
--------------------
0 18
1 20
2 19
Name: age, dtype: object
--------------------
Curry
# 修改值
df.iloc[0,0]='panda'
print(df)
print('-'*40)
# dataframe中的排序方法
df = df.sort_values(by='age',ascending=False)
# ascending=False： 降序排列，默认是升序
print(df)
name age national
1 panda 20 China
2 Iversion 19 us
0 panda 18 us
----------------------------------------
name age national
1 panda 20 China
2 Iversion 19 us
0 panda 18 us
dataframe修改index、columns
df1=pd.DataFrame(np.arange(9).reshape(3,3),index=['bi','sh','oz'],columns=['a','b','c'])
print(df1)
a b c
bi 0 1 2
sh 3 4 5
oz 6 7 8
#修改df1的index
df1.index=['beijing','shanghai','xiamen']
print(df1)
a b c
beijing 0 1 2
shanghai 3 4 5
xiamen 6 7 8
#定义map函数（x是原有行列）
def test_map(x):
return x+'_ABC'
print(df1.rename(index=test_map,columns=test_map))
a_ABC b_ABC c_ABC
beijing_ABC 0 1 2
shanghai_ABC 3 4 5
xiamen_ABC 6 7 8
# 同时，renme可以传入字典，为某个index单独修改名称
df2=df1.rename(index={'beijing':'bi'},columns={'a':'aa'})
print(df2)
aa b c
bi 0 1 2
shanghai 3 4 5
xiamen 6 7 8
# 列转化为索引
df3=pd.DataFrame({'X':range(5),'Y':range(5),'S':['a','b','c','d','e'],'Z':[1,1,2,2,2]})
print(df3)
print('-'*30)
# 指定一列为索引 (drop=False 指定同时保留作为索引的列)
result = df3.set_index('S',drop=False)
result.index.name=None
print(result)
X Y S Z
0 0 0 a 1
1 1 1 b 1
2 2 2 c 2
3 3 3 d 2
4 4 4 e 2
------------------------------
X Y S Z
a 0 0 a 1
b 1 1 b 1
c 2 2 c 2
d 3 3 d 2
e 4 4 e 2
添加数据
# 增加数据
df1 = pd.DataFrame([['Snow','M',22],['Tyrion','M',32],['Sansa','F',18],['Arya','F',14]], columns=['name','gender','age'])
print(df1)
name gender age
0 Snow M 22
1 Tyrion M 32
2 Sansa F 18
3 Arya F 14
#在数据框最后加上score
df1['score']=[80,90,67,78]#与增加数据一样
print(df1)
name gender age score
0 Snow M 22 80
1 Tyrion M 32 90
2 Sansa F 18 67
3 Arya F 14 78
在具体某个位置插入一列可以用insert的方法
- 语法格式：列表.insert(index, obj)
- index --->对象 obj 需要插入的索引位置。
- obj ---> 要插入列表中的对象（列名）
col_name=df1.columns.tolist()#将数据框中列明全部提取出存在列表里
col_name.insert(2,'city') #在列索引为2的位置后面插入列名为city的列
#没有值，所有列为NaN
df1=df1.reindex(columns=col_name)# DataFrame.reindex() 对原行/列索引重新构建索引值
print(df1)
name gender city age score
0 Snow M NaN 22 80
1 Tyrion M NaN 32 90
2 Sansa F NaN 18 67
3 Arya F NaN 14 78
df1['city']=['北京','上海','广东','厦门']
print(df1)
name gender city age score
0 Snow M 北京 22 80
1 Tyrion M 上海 32 90
2 Sansa F 广东 18 67
3 Arya F 厦门 14 78
df中的insert,插入一列
df.insert(iloc,column,value)
- iloc:要插入的位置
- colunm:列名
- value:值
df1.insert(2,'score2',[79,43,67,89])
print(df1)
name gender score2 city age score
0 Snow M 79 北京 22 80
1 Tyrion M 43 上海 32 90
2 Sansa F 67 广东 18 67
3 Arya F 89 厦门 14 78
# 插入行
row=['Ryan','M','100','厦门','18','100']
df1.loc[3]=row
print(df1)
name gender score2 city age score
0 Snow M 79 北京 22 80
1 Tyrion M 43 上海 32 90
2 Sansa F 67 广东 18 67
3 Ryan M 100 厦门 18 100
# 增加数据
df1 = pd.DataFrame([['Snow','M',22],
['Tyrion','M',32],
['Sansa','F',18],
['Arya','F',14]],
columns=['name','gender','age'])
print(df1)
# 先创建一个DataFrame，用来增加进数据框的最后一行
new=pd.DataFrame({'name':'lisa',
'gender':'F', 'age':19},
index=[0])
print('-'*30)
print(new)
print('-'*30)
df1=df1.append(new,ignore_index=True) # ignore_index=False,表示不按原来的索引，从0开始自动递增
print(df1)
name gender age
0 Snow M 22
1 Tyrion M 32
2 Sansa F 18
3 Arya F 14
------------------------------
name gender age
0 lisa F 19
------------------------------
name gender age
0 Snow M 22
1 Tyrion M 32
2 Sansa F 18
3 Arya F 14
4 lisa F 19
插入多行多列
- objs:合并对象
- axis:合并方式，默认0表示按列合并，1表示按行合并
- ignore_index:是否忽略索引
df1 = pd.DataFrame(np.arange(6).reshape(3,2),columns=['four','five'])
df2 = pd.DataFrame(np.arange(6).reshape(2,3),columns=['one','two','three'])
print(df2)
print('-'*30)
# 按行合并
result=pd.concat([df2,df1],axis=1)
print(result)
print('-'*30)
# 按列合并
result=pd.concat([df2,df1],ignore_index=True,)
print(result)
one two three
0 0 1 2
1 3 4 5
------------------------------
one two three four five
0 0.0 1.0 2.0 0 1
1 3.0 4.0 5.0 2 3
2 NaN NaN NaN 4 5
------------------------------
five four one three two
0 NaN NaN 0.0 2.0 1.0
1 NaN NaN 3.0 5.0 4.0
2 1.0 0.0 NaN NaN NaN
3 3.0 2.0 NaN NaN NaN
4 5.0 4.0 NaN NaN NaN
E:\anaconda\lib\site-packages\ipykernel_launcher.py:14: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=True'.
To retain the current behavior and silence the warning, pass sort=False
DataFrame的删除
- lables：要删除数据的标签
- axis：0表示删除行，1表示删除列，默认0
- inplace:是否在当前df中执行此操作
df2 = pd.DataFrame(np.arange(9).reshape(3,3),columns=['one','two','three'])
print(df2)
df4=df2.drop(['one'],axis=1, inplace=False)
df3=df2.drop([0,1],axis=0, inplace=False)
print('-'*30)
print(df4)
print('-'*30)
print(df3)
one two three
0 0 1 2
1 3 4 5
2 6 7 8
------------------------------
two three
0 1 2
1 4 5
2 7 8
------------------------------
one two three
2 6 7 8
数据处理
通过dropna()滤除缺失数据：
se=pd.Series([4,None,8,None,5])
print(se)
print('-'*20)
print(se.dropna())
print('-'*20)
print(se.notnull())
print('-'*20)
print(se.isnull())
print('-'*20)
# 通过布尔序列也能滤除：
print(se[se.notnull()])
0 4.0
1 NaN
2 8.0
3 NaN
4 5.0
dtype: float64
--------------------
0 4.0
2 8.0
4 5.0
dtype: float64
--------------------
0 True
1 False
2 True
3 False
4 True
dtype: bool
--------------------
0 False
1 True
2 False
3 True
4 False
dtype: bool
--------------------
0 4.0
2 8.0
4 5.0
dtype: float64
处理DataFrame对象
df1=pd.DataFrame([[1,2,3],[None,None,2],[None,None,None],[8,8,None]])
print(df1)
print('-'*20)
print(df1.dropna())
print('-'*20)
# 传入how=‘all’滤除全为NaN的行：
print(df1.dropna(how='all')) # 默认情况下是how='any'，只要有nan就删除
print('-'*20)
# 传入axis=1滤除列：
print(df1.dropna(axis=1,how="any"))
print('-'*20)
#传入thresh=n保留至少有n个非NaN数据的行：
print(df1.dropna(thresh=3))
0 1 2
0 1.0 2.0 3.0
1 NaN NaN 2.0
2 NaN NaN NaN
3 8.0 8.0 NaN
--------------------
0 1 2
0 1.0 2.0 3.0
--------------------
0 1 2
0 1.0 2.0 3.0
1 NaN NaN 2.0
3 8.0 8.0 NaN
--------------------
Empty DataFrame
Columns: []
Index: [0, 1, 2, 3]
--------------------
0 1 2
0 1.0 2.0 3.0
填充缺失数据
df1=pd.DataFrame([[1,2,3],[None,None,2],[None,None,None],[8,8,None]])
print(df1)
0 1 2
0 1.0 2.0 3.0
1 NaN NaN 2.0
2 NaN NaN NaN
3 8.0 8.0 NaN
# 用常数填充fillna
print(df1.fillna(0))
0 1 2
0 1.0 2.0 3.0
1 0.0 0.0 2.0
2 0.0 0.0 0.0
3 8.0 8.0 0.0
#传入inplace=True直接修改原对象：
df1.fillna(0,inplace=True)
print(df1)
0 1 2
0 1.0 2.0 3.0
1 0.0 0.0 2.0
2 0.0 0.0 0.0
3 8.0 8.0 0.0
df1=pd.DataFrame([[1,2,3],[None,None,2],[None,None,None],[8,8,None]])
# 通过字典填充不同的常数
print(df1.fillna({0:10,1:20,2:30}))
0 1 2
0 1.0 2.0 3.0
1 10.0 20.0 2.0
2 10.0 20.0 30.0
3 8.0 8.0 30.0
#填充平均值
print(df1.fillna(df1.mean()))
# 如果只填充一列
df1.iloc[:,1].fillna(5,inplace = True)
print(df1)
0 1 2
0 1.0 2.0 3.0
1 4.5 5.0 2.0
2 4.5 5.0 2.5
3 8.0 8.0 2.5
0 1 2
0 1.0 2.0 3.0
1 NaN 5.0 2.0
2 NaN 5.0 NaN
3 8.0 8.0 NaN
# 传 入 method=” “ 改 变 插 值 方 式 ：
df2=pd.DataFrame(np.random.randint(0,10,(5,5)))
df2.iloc[1:4,3]=None
df2.iloc[2:4,4]=None
print(df2)
print('-'*20)
#用前面的值来填充ffill 用后面的值来填充bfill
print(df2.fillna(method='ffill'))
print('-'*20)
# 传入limit=” “限制填充行数：
print(df2.fillna(method='bfill',limit=2))
print('-'*20)
# 传入axis=” “修改填充方向：
print(df2.fillna(method="ffill",limit=1,axis=1))
0 1 2 3 4
0 7 6 9 4.0 4.0
1 8 1 7 NaN 5.0
2 2 7 0 NaN NaN
3 7 3 2 NaN NaN
4 1 0 7 7.0 4.0
--------------------
0 1 2 3 4
0 7 6 9 4.0 4.0
1 8 1 7 4.0 5.0
2 2 7 0 4.0 5.0
3 7 3 2 4.0 5.0
4 1 0 7 7.0 4.0
--------------------
0 1 2 3 4
0 7 6 9 4.0 4.0
1 8 1 7 NaN 5.0
2 2 7 0 7.0 4.0
3 7 3 2 7.0 4.0
4 1 0 7 7.0 4.0
--------------------
0 1 2 3 4
0 7.0 6.0 9.0 4.0 4.0
1 8.0 1.0 7.0 7.0 5.0
2 2.0 7.0 0.0 0.0 NaN
3 7.0 3.0 2.0 2.0 NaN
4 1.0 0.0 7.0 7.0 4.0
移除重复数据
DataFrame中经常会出现重复行，利用duplicated()函数返回每一行判断是否重复的结果（重复则为True)
df1=pd.DataFrame({'A':[1,1,1,2,2,3,1],'B':list("aabbbca")})
print(df1)
print('-'*30)
# 判断每一行是否重复(结果是bool值，TRUE代表重复的)
print(df1.duplicated())
print('-'*30)
# 去除全部的重复行
print(df1.drop_duplicates())
print('-'*30)
# 指定列去除重复行
print(df1.drop_duplicates(['A']))
print('-'*30)
# 保留重复行中的最后一行
print(df1.drop_duplicates(['A'],keep='last'))
print('-'*30)
#去除重复的同时改变DataFrame对象
df1.drop_duplicates(['A','B'],inplace=True)
print(df1)
A B
0 1 a
1 1 a
2 1 b
3 2 b
4 2 b
5 3 c
6 1 a
------------------------------
0 False
1 True
2 False
3 False
4 True
5 False
6 True
dtype: bool
------------------------------
A B
0 1 a
2 1 b
3 2 b
5 3 c
------------------------------
A B
0 1 a
3 2 b
5 3 c
------------------------------
A B
4 2 b
5 3 c
6 1 a
------------------------------
A B
0 1 a
2 1 b
3 2 b
5 3 c
数据合并
join合并
着重关注行的合并
import pandas as pd
df3=pd.DataFrame({'Red':[1,3,5],'Green':[5,0,3]},index=list('abc'))
df4=pd.DataFrame({'Blue':[1,9,8],'Yellow':[6,6,7]},index=list('cde'))
print(df3)
print('-'*30)
print(df4)
# 简单合并（默认是left左连接,以左侧df3为基础）
df3.join(df4,how='left')
print('-'*30)
print(df3)
# 右链接
df3.join(df4,how='right')
# 外链接
df3.join(df4,how='outer')
Red Green
a 1 5
b 3 0
c 5 3
------------------------------
Blue Yellow
c 1 6
d 9 6
e 8 7
------------------------------
Red Green
a 1 5
b 3 0
c 5 3
| Red | Green | Blue | Yellow | |
|---|---|---|---|---|
| a | 1.0 | 5.0 | NaN | NaN |
| b | 3.0 | 0.0 | NaN | NaN |
| c | 5.0 | 3.0 | 1.0 | 6.0 |
| d | NaN | NaN | 9.0 | 6.0 |
| e | NaN | NaN | 8.0 | 7.0 |
# 合并多个DataFrame对象
df5=pd.DataFrame({'Brown':[3,4,5],'White':[1,1,2]},index=list('aed'))
df3.join([df4,df5])
| Red | Green | Blue | Yellow | Brown | White | |
|---|---|---|---|---|---|---|
| a | 1 | 5 | NaN | NaN | 3.0 | 1.0 |
| b | 3 | 0 | NaN | NaN | NaN | NaN |
| c | 5 | 3 | 1.0 | 6.0 | NaN | NaN |
merge合并
着重关注的是列的合并
df1=pd.DataFrame({'名字':list('ABCDE'),'性别':['男','女','男','男','女'],'职称':['副教授','讲 师','助教','教授','助教']},
index=range(1001,1006))
df1.columns.name='学院老师'
df1.index.name='编号'
print(df1)
print('-'*20)
df2=pd.DataFrame({'名字':list('ABDAX'),'课程':['C++','计算机导论','汇编','数据结构','马克思原理'],'职称':['副教授','讲师','教授','副教授','讲师']},index=[1001,1002,1004,1001,3001])
df2.columns.name='课程'
df2.index.name='编号'
print(df2)
学院老师 名字 性别 职称
编号
1001 A 男 副教授
1002 B 女 讲 师
1003 C 男 助教
1004 D 男 教授
1005 E 女 助教
--------------------
课程 名字 课程 职称
编号
1001 A C++ 副教授
1002 B 计算机导论 讲师
1004 D 汇编 教授
1001 A 数据结构 副教授
3001 X 马克思原理 讲师
# 默认下是根据左右对象中出现同名的列作为连接的键，且连接方式是how=’inner’
print(pd.merge(df1,df2))# 返回匹配的
# 指定列名合并
pd.merge(df1,df2,on='名字',suffixes=['_1','_2'])# 返回匹配的
# 连接方式，根据左侧为准
pd.merge(df1,df2,how='left')
# 根据左侧为准
pd.merge(df1,df2,how='right')
# 所 有
# pd.merge(df1,df2,how='outer')
# 根据多个键进行连接
pd.merge(df1,df2,on=['职称','名字'])
名字 性别 职称 课程
0 A 男 副教授 C++
1 A 男 副教授 数据结构
2 D 男 教授 汇编
| 名字 | 性别 | 职称 | 课程 | |
|---|---|---|---|---|
| 0 | A | 男 | 副教授 | C++ |
| 1 | A | 男 | 副教授 | 数据结构 |
| 2 | D | 男 | 教授 | 汇编 |
拓展——轴向连接（Concat）
- Series对象的连接
s1=pd.Series([1,2],index=list('ab'))
s2=pd.Series([3,4,5],index=list('bde'))
print(s1)
print(s2)
print(pd.concat([s1,s2]))
#横向连接
print(pd.concat([s1,s2],axis=1))
# 用内连接求交集(连接方式，共有’inner’,’left’,right’,’outer’)
pd.concat([s1,s2],axis=1,join='inner')
# 指定部分索引进行连接
pd.concat([s1,s2],axis=1,join_axes=[list('abc')])
# 创建层次化索引
pd.concat([s1,s2],keys=['A','B'])
#当纵向连接时keys为列名
pd.concat([s1,s2],keys=['A','D'],axis=1)
a 1
b 2
dtype: int64
b 3
d 4
e 5
dtype: int64
a 1
b 2
b 3
d 4
e 5
dtype: int64
0 1
a 1.0 NaN
b 2.0 3.0
d NaN 4.0
e NaN 5.0
E:\anaconda\lib\site-packages\ipykernel_launcher.py:8: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=True'.
To retain the current behavior and silence the warning, pass sort=False
E:\anaconda\lib\site-packages\ipykernel_launcher.py:20: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=True'.
To retain the current behavior and silence the warning, pass sort=False
| A | D | |
|---|---|---|
| a | 1.0 | NaN |
| b | 2.0 | 3.0 |
| d | NaN | 4.0 |
| e | NaN | 5.0 |
- DataFrame对象的连接
df3=pd.DataFrame({'Red':[1,3,5],'Green':[5,0,3]},index=list('abd'))
df4=pd.DataFrame({'Blue':[1,9],'Yellow':[6,6]},index=list('ce'))
print(df3)
print(df4)
pd.concat([df3,df4])
pd.concat([df3,df4],axis=1,keys=['A','B'])
# 用字典的方式连接同样可以创建层次化列索引
pd.concat({'A':df3,'B':df4},axis=1)
Red Green
a 1 5
b 3 0
d 5 3
Blue Yellow
c 1 6
e 9 6
E:\anaconda\lib\site-packages\ipykernel_launcher.py:5: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=True'.
To retain the current behavior and silence the warning, pass sort=False
"""
E:\anaconda\lib\site-packages\ipykernel_launcher.py:6: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=True'.
To retain the current behavior and silence the warning, pass sort=False
E:\anaconda\lib\site-packages\ipykernel_launcher.py:9: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=True'.
To retain the current behavior and silence the warning, pass sort=False
if __name__ == '__main__':
| A | B | |||
|---|---|---|---|---|
| Red | Green | Blue | Yellow | |
| a | 1.0 | 5.0 | NaN | NaN |
| b | 3.0 | 0.0 | NaN | NaN |
| c | NaN | NaN | 1.0 | 6.0 |
| d | 5.0 | 3.0 | NaN | NaN |
| e | NaN | NaN | 9.0 | 6.0 |
多层索引（拓展）
创建多层索引
import numpy as np
import pandas as pd
from pandas import Series,DataFrame
# Series也可以创建多层索引
s = Series(np.random.randint(0,150,size=6),index=list('abcdef'))
print(s)
print('-'*20)
s = Series(np.random.randint(0,150,size=6),
index=[['a','a','b','b','c','c'],['期中','期末','期中','期末','期中','期末']])
print(s)
a 146
b 137
c 39
d 16
e 142
f 142
dtype: int32
--------------------
a 期中 43
期末 53
b 期中 71
期末 114
c 期中 149
期末 13
dtype: int32
#DataFrame创建多层索引
df1 = DataFrame(np.random.randint(0,150,size=(6,4)),columns = ['zs','ls','ww','zl'],
index = [['python','python','math','math','En','En'],['期中','期末','期中','期末','期中','期末']])
print(df1)
zs ls ww zl
python 期中 7 131 141 55
期末 77 52 85 69
math 期中 139 118 80 3
期末 23 5 14 27
En 期中 142 47 28 136
期末 19 95 33 55
特定结构
class1=['python','python','math','math','En','En']
class2=['期中','期末','期中','期末','期中','期末']
m_index2=pd.MultiIndex.from_arrays([class1,class2])
df2=DataFrame(np.random.randint(0,150,(6,4)),index=m_index2)
print(df2)
print('-'*20)
class1=['期中','期中','期中','期末','期末','期末']
class2=['python','math','En','python','math','En']
m_index2=pd.MultiIndex.from_arrays([class1,class2])
df2=DataFrame(np.random.randint(0,150,(6,4)),index=m_index2)
print(df2)
0 1 2 3
python 期中 13 13 108 94
期末 117 130 87 74
math 期中 35 111 80 94
期末 144 94 111 0
En 期中 135 80 87 90
期末 108 63 119 118
--------------------
0 1 2 3
期中 python 22 36 66 41
math 142 5 139 73
En 62 130 120 140
期末 python 144 5 60 128
math 18 41 14 37
En 74 62 10 83
product构造
class1=['python','math','En']
class2=['期中','期末']
m_index2=pd.MultiIndex.from_product([class1,class2])
df2=DataFrame(np.random.randint(0,150,(6,4)),index=m_index2)
print(df2)
0 1 2 3
python 期中 23 98 144 114
期末 122 80 115 13
math 期中 79 80 43 65
期末 99 132 103 2
En 期中 1 81 27 72
期末 50 113 28 116
多层索引对象的索引
#多层索引对象的索引操作# series
s = Series(np.random.randint(0,150,size=6),index=[['a','a','b','b','c','c'],['期中','期末','期中','期末','期中','期末']]) # print(s)
#取一个第一级索引
print(s['a'])
# 取多个第一级索引
print(s[['a','b']])
# 根据索引获取值
print(s['a','期末'])
# loc方法取值
print(s.loc['a'])
print(s.loc[['a','b']])
print(s.loc['a','期末'])
# iloc方法取值(iloc计算的事最内层索引)
print(s.iloc[1])
print(s.iloc[1:4])
# dataframe
class1=['python','math','En']
class2=[' 期 中 ',' 期 末 ']
m_index2=pd.MultiIndex.from_product([class1,class2])
df2=DataFrame(np.random.randint(0,150,(6,4)),index=m_index2)
print(df2)
# 获取列
print(df2[0])
# 一级索引
print(df2.loc['python'])
# 多个一级索引
print(df2.loc[['python','math']])
# 取一行
# print(df2.loc['python','期末'])
# 取一值
# print(df2.loc['python','期末'][0])
# iloc是只取最内层的索引的
print(df2.iloc[0])
期中 110
期末 123
dtype: int32
a 期中 110
期末 123
b 期中 93
期末 2
dtype: int32
123
期中 110
期末 123
dtype: int32
a 期中 110
期末 123
b 期中 93
期末 2
dtype: int32
123
123
a 期末 123
b 期中 93
期末 2
dtype: int32
0 1 2 3
python 期 中 123 28 120 69
期 末 4 146 50 32
math 期 中 142 65 56 81
期 末 74 59 17 80
En 期 中 133 17 89 113
期 末 18 35 60 106
python 期 中 123
期 末 4
math 期 中 142
期 末 74
En 期 中 133
期 末 18
Name: 0, dtype: int32
0 1 2 3
期 中 123 28 120 69
期 末 4 146 50 32
0 1 2 3
python 期 中 123 28 120 69
期 末 4 146 50 32
math 期 中 142 65 56 81
期 末 74 59 17 80
0 123
1 28
2 120
3 69
Name: (python, 期 中 ), dtype: int32
Timestamp
时间模块datetime
datetime.date (date对象)
import datetime
# 或者from datetime import date
today=datetime.date.today()#今天的时间
print(today,type(today))
print(str(today),type(str(today)))
#输出date类的格式
print('-'*30)
t=datetime.date(1997,10,26)
print(t)
#（年，月，日）
2019-11-29 <class 'datetime.date'>
2019-11-29 <class 'str'>
------------------------------
1997-10-26
datetime.date(datetime对象)
now=datetime.datetime.now()
print(now,type(now))
print(str(now),type(str(now)))
t1 = datetime.datetime(2016,6,1)
t2 = datetime.datetime(2014,1,1,12,44,33)
print(t1,t2)
# (年，月，日，时，分，秒)，至少输入年月日
t2-t1
# 相减得到时间差 —— timedelta
2019-11-29 14:47:16.940429 <class 'datetime.datetime'>
2019-11-29 14:47:16.940429 <class 'str'>
2016-06-01 00:00:00 2014-01-01 12:44:33
datetime.timedelta(-882, 45873)
datetime.timedelta(时间差)
today = datetime.datetime.today() # datetime.datetime也有today()方法
yestoday = today - datetime.timedelta(1) #
print(today)
print(yestoday)
print(today - datetime.timedelta(7))
# 时间差主要用作时间的加减法，相当于可被识别的时间“差值”
2019-11-29 14:48:03.943359
2019-11-28 14:48:03.943359
2019-11-22 14:48:03.943359
parser.parse(日期字符串转换)
from dateutil.parser import parse
date = '12-21-2017'
t = parse(date)
print(t,type(t))
# 直接将str转化成datetime.datetime
print(parse('2000-1-1'),'\n',
parse('5/1/2014'),'\n',
parse('5/1/2014', dayfirst = True),'\n', # 国际通用格式中，日在月之前，可以通过dayfirst来设置
parse('22/1/2014'),'\n',
parse('Jan 31, 1997 10:45 PM'))
# 各种格式可以解析，但无法支持中文
2017-12-21 00:00:00 <class 'datetime.datetime'>
2000-01-01 00:00:00
2014-05-01 00:00:00
2014-01-05 00:00:00
2014-01-22 00:00:00
1997-01-31 22:45:00
Timestamp(Pandas时刻数据)
时刻数据代表时间点，是pandas的数据类型，是将值与时间点相关联的最基本类型的时间序列数据。pandas.Timestamp()
pd.Timestamp()
import numpy as np
import pandas as pd
date1 = datetime.datetime(2016,12,1,12,45,30) # 创建一个datetime.datetime
date2 = '2017-12-21' # 创建一个字符串
t1 = pd.Timestamp(date1)
t2 = pd.Timestamp(date2)
print(t1,type(t1))
print(t2)
print(pd.Timestamp('2017-12-21 15:00:22'))
# 直接生成pandas的时刻数据 → 时间戳
# 数据类型为 pandas的Timestamp
2016-12-01 12:45:30 <class 'pandas._libs.tslibs.timestamps.Timestamp'>
2017-12-21 00:00:00
2017-12-21 15:00:22
pd.to_datetime（转化为时间戳）
from datetime import datetime
date1 = datetime(2016,12,1,12,45,30)
date2 = '2017-12-21'
t1 = pd.to_datetime(date1)
t2 = pd.to_datetime(date2)
print(t1,type(t1))
print(t2,type(t2))
# pd.to_datetime()：如果是单个时间数据，转换成pandas的时刻数据，数据类型为Timestamp
lst_date = [ '2017-12-21', '2017-12-22', '2017-12-23']
t3 = pd.to_datetime(lst_date)
print(t3,type(t3))
# 多个时间数据，将会转换为pandas的DatetimeIndex
2016-12-01 12:45:30 <class 'pandas._libs.tslibs.timestamps.Timestamp'>
2017-12-21 00:00:00 <class 'pandas._libs.tslibs.timestamps.Timestamp'>
DatetimeIndex(['2017-12-21', '2017-12-22', '2017-12-23'], dtype='datetime64[ns]', freq=None) <class 'pandas.core.indexes.datetimes.DatetimeIndex'>
- 多个时间数据转换时间戳索引
date1 = [datetime(2015,6,1),datetime(2015,7,1),datetime(2015,8,1),datetime(2015,9,1),datetime(2015,10,1)]
date2 = ['2017-2-1','2017-2-2','2017-2-3','2017-2-4','2017-2-5','2017-2-6']
print(date1)
print(date2)
t1 = pd.to_datetime(date2)
t2 = pd.to_datetime(date2)
print(t1)
print(t2)
# 多个时间数据转换为 DatetimeIndex
date3 = ['2017-2-1','2017-2-2','2017-2-3','hello world!','2017-2-5','2017-2-6']
t3 = pd.to_datetime(date3, errors = 'ignore')
print(t3,type(t3))
# 当一组时间序列中夹杂其他格式数据，可用errors参数返回
# errors = 'ignore':不可解析时返回原始输入，这里就是直接生成一般数组
t4 = pd.to_datetime(date3, errors = 'coerce')
print(t4,type(t4))
# errors = 'coerce':不可扩展，缺失值返回NaT（Not a Time），结果认为DatetimeIndex
[datetime.datetime(2015, 6, 1, 0, 0), datetime.datetime(2015, 7, 1, 0, 0), datetime.datetime(2015, 8, 1, 0, 0), datetime.datetime(2015, 9, 1, 0, 0), datetime.datetime(2015, 10, 1, 0, 0)]
['2017-2-1', '2017-2-2', '2017-2-3', '2017-2-4', '2017-2-5', '2017-2-6']
DatetimeIndex(['2017-02-01', '2017-02-02', '2017-02-03', '2017-02-04',
'2017-02-05', '2017-02-06'],
dtype='datetime64[ns]', freq=None)
DatetimeIndex(['2017-02-01', '2017-02-02', '2017-02-03', '2017-02-04',
'2017-02-05', '2017-02-06'],
dtype='datetime64[ns]', freq=None)
['2017-2-1' '2017-2-2' '2017-2-3' 'hello world!' '2017-2-5' '2017-2-6'] <class 'numpy.ndarray'>
DatetimeIndex(['2017-02-01', '2017-02-02', '2017-02-03', 'NaT', '2017-02-05',
'2017-02-06'],
dtype='datetime64[ns]', freq=None) <class 'pandas.core.indexes.datetimes.DatetimeIndex'>
DatetimeIndex(Pandas时间戳索引)
pd.DatetimeIndex()与TimeSeries时间序列
rng = pd.DatetimeIndex(['12/1/2017','12/2/2017','12/3/2017','12/4/2017','12/5/2017'])
print(rng,type(rng))
print(rng[0],type(rng[0]))
# 直接生成时间戳索引，支持str、datetime.datetime
# 单个时间戳为Timestamp，多个时间戳为DatetimeIndex
st = pd.Series(np.random.rand(len(rng)), index = rng)
print(st,type(st))
print(st.index)
# 以DatetimeIndex为index的Series，为TimeSries，时间序列
DatetimeIndex(['2017-12-01', '2017-12-02', '2017-12-03', '2017-12-04',
'2017-12-05'],
dtype='datetime64[ns]', freq=None) <class 'pandas.core.indexes.datetimes.DatetimeIndex'>
2017-12-01 00:00:00 <class 'pandas._libs.tslibs.timestamps.Timestamp'>
2017-12-01 0.322258
2017-12-02 0.677879
2017-12-03 0.142017
2017-12-04 0.366208
2017-12-05 0.139923
dtype: float64 <class 'pandas.core.series.Series'>
DatetimeIndex(['2017-12-01', '2017-12-02', '2017-12-03', '2017-12-04',
'2017-12-05'],
dtype='datetime64[ns]', freq=None)
pd.date_range()-日期范围
生成日期范围
生成方法：
- start+end
- star/end+periods
默认频率：day
rng1 = pd.date_range('1/1/2017','1/10/2017', normalize=True)
rng2 = pd.date_range(start = '1/1/2017', periods = 10)
rng3 = pd.date_range(end = '1/30/2017 15:00:00', periods = 10) # 增加了时、分、秒
print(rng1,type(rng1))
print(rng2)
print(rng3)
print('-------')
# 直接生成DatetimeIndex
# pd.date_range(start=None, end=None, periods=None, freq='D', tz=None, normalize=False, name=None, closed=None, **kwargs)
# start：开始时间
# end：结束时间
# periods：偏移量
# freq：频率，默认天，pd.date_range()默认频率为日历日，pd.bdate_range()默认频率为工作日
# tz：时区
rng4 = pd.date_range(start = '1/1/2017 15:30', periods = 10, name = 'hello world!', normalize = True)
print(rng4)
print('-------')
# normalize：时间参数值正则化到午夜时间戳（这里最后就直接变成0:00:00，并不是15:30:00）
# name：索引对象名称
print(pd.date_range('20170101','20170104')) # 20170101也可读取
print(pd.date_range('20170101','20170104',closed = 'right'))
print(pd.date_range('20170101','20170104',closed = 'left'))
print('-------')
# closed：默认为None的情况下，左闭右闭，left则左闭右开，right则左开右闭
print(pd.bdate_range('20170101','20170107'))
# pd.bdate_range()默认频率为工作日
print(list(pd.date_range(start = '1/1/2017', periods = 10)))
# 直接转化为list，元素为Timestamp
DatetimeIndex(['2017-01-01', '2017-01-02', '2017-01-03', '2017-01-04',
'2017-01-05', '2017-01-06', '2017-01-07', '2017-01-08',
'2017-01-09', '2017-01-10'],
dtype='datetime64[ns]', freq='D') <class 'pandas.core.indexes.datetimes.DatetimeIndex'>
DatetimeIndex(['2017-01-01', '2017-01-02', '2017-01-03', '2017-01-04',
'2017-01-05', '2017-01-06', '2017-01-07', '2017-01-08',
'2017-01-09', '2017-01-10'],
dtype='datetime64[ns]', freq='D')
DatetimeIndex(['2017-01-21 15:00:00', '2017-01-22 15:00:00',
'2017-01-23 15:00:00', '2017-01-24 15:00:00',
'2017-01-25 15:00:00', '2017-01-26 15:00:00',
'2017-01-27 15:00:00', '2017-01-28 15:00:00',
'2017-01-29 15:00:00', '2017-01-30 15:00:00'],
dtype='datetime64[ns]', freq='D')
-------
DatetimeIndex(['2017-01-01', '2017-01-02', '2017-01-03', '2017-01-04',
'2017-01-05', '2017-01-06', '2017-01-07', '2017-01-08',
'2017-01-09', '2017-01-10'],
dtype='datetime64[ns]', name='hello world!', freq='D')
-------
DatetimeIndex(['2017-01-01', '2017-01-02', '2017-01-03', '2017-01-04'], dtype='datetime64[ns]', freq='D')
DatetimeIndex(['2017-01-02', '2017-01-03', '2017-01-04'], dtype='datetime64[ns]', freq='D')
DatetimeIndex(['2017-01-01', '2017-01-02', '2017-01-03'], dtype='datetime64[ns]', freq='D')
-------
DatetimeIndex(['2017-01-02', '2017-01-03', '2017-01-04', '2017-01-05',
'2017-01-06'],
dtype='datetime64[ns]', freq='B')
[Timestamp('2017-01-01 00:00:00', freq='D'), Timestamp('2017-01-02 00:00:00', freq='D'), Timestamp('2017-01-03 00:00:00', freq='D'), Timestamp('2017-01-04 00:00:00', freq='D'), Timestamp('2017-01-05 00:00:00', freq='D'), Timestamp('2017-01-06 00:00:00', freq='D'), Timestamp('2017-01-07 00:00:00', freq='D'), Timestamp('2017-01-08 00:00:00', freq='D'), Timestamp('2017-01-09 00:00:00', freq='D'), Timestamp('2017-01-10 00:00:00', freq='D')]
pd.date_range()-日期范围：频率(1)
print(pd.date_range('2017/1/1','2017/1/4')) # 默认freq = 'D'：每日历日
print(pd.date_range('2017/1/1','2017/1/4', freq = 'B')) # B：每工作日
print(pd.date_range('2017/1/1','2017/1/2', freq = 'H')) # H：每小时
print(pd.date_range('2017/1/1 12:00','2017/1/1 12:10', freq = 'T')) # T/MIN：每分
print(pd.date_range('2017/1/1 12:00:00','2017/1/1 12:00:10', freq = 'S')) # S：每秒
print(pd.date_range('2017/1/1 12:00:00','2017/1/1 12:00:10', freq = 'L')) # L：每毫秒（千分之一秒）
print(pd.date_range('2017/1/1 12:00:00','2017/1/1 12:00:10', freq = 'U')) # U：每微秒（百万分之一秒）
print(pd.date_range('2017/1/1','2017/2/1', freq = 'W-MON'))
# W-MON：从指定星期几开始算起，每周
# 星期几缩写：MON/TUE/WED/THU/FRI/SAT/SUN
print(pd.date_range('2017/1/1','2017/5/1', freq = 'WOM-2MON'))
# WOM-2MON：每月的第几个星期几开始算，这里是每月第二个星期一
DatetimeIndex(['2017-01-01', '2017-01-02', '2017-01-03', '2017-01-04'], dtype='datetime64[ns]', freq='D')
DatetimeIndex(['2017-01-02', '2017-01-03', '2017-01-04'], dtype='datetime64[ns]', freq='B')
DatetimeIndex(['2017-01-01 00:00:00', '2017-01-01 01:00:00',
'2017-01-01 02:00:00', '2017-01-01 03:00:00',
'2017-01-01 04:00:00', '2017-01-01 05:00:00',
'2017-01-01 06:00:00', '2017-01-01 07:00:00',
'2017-01-01 08:00:00', '2017-01-01 09:00:00',
'2017-01-01 10:00:00', '2017-01-01 11:00:00',
'2017-01-01 12:00:00', '2017-01-01 13:00:00',
'2017-01-01 14:00:00', '2017-01-01 15:00:00',
'2017-01-01 16:00:00', '2017-01-01 17:00:00',
'2017-01-01 18:00:00', '2017-01-01 19:00:00',
'2017-01-01 20:00:00', '2017-01-01 21:00:00',
'2017-01-01 22:00:00', '2017-01-01 23:00:00',
'2017-01-02 00:00:00'],
dtype='datetime64[ns]', freq='H')
DatetimeIndex(['2017-01-01 12:00:00', '2017-01-01 12:01:00',
'2017-01-01 12:02:00', '2017-01-01 12:03:00',
'2017-01-01 12:04:00', '2017-01-01 12:05:00',
'2017-01-01 12:06:00', '2017-01-01 12:07:00',
'2017-01-01 12:08:00', '2017-01-01 12:09:00',
'2017-01-01 12:10:00'],
dtype='datetime64[ns]', freq='T')
DatetimeIndex(['2017-01-01 12:00:00', '2017-01-01 12:00:01',
'2017-01-01 12:00:02', '2017-01-01 12:00:03',
'2017-01-01 12:00:04', '2017-01-01 12:00:05',
'2017-01-01 12:00:06', '2017-01-01 12:00:07',
'2017-01-01 12:00:08', '2017-01-01 12:00:09',
'2017-01-01 12:00:10'],
dtype='datetime64[ns]', freq='S')
DatetimeIndex([ '2017-01-01 12:00:00', '2017-01-01 12:00:00.001000',
'2017-01-01 12:00:00.002000', '2017-01-01 12:00:00.003000',
'2017-01-01 12:00:00.004000', '2017-01-01 12:00:00.005000',
'2017-01-01 12:00:00.006000', '2017-01-01 12:00:00.007000',
'2017-01-01 12:00:00.008000', '2017-01-01 12:00:00.009000',
...
'2017-01-01 12:00:09.991000', '2017-01-01 12:00:09.992000',
'2017-01-01 12:00:09.993000', '2017-01-01 12:00:09.994000',
'2017-01-01 12:00:09.995000', '2017-01-01 12:00:09.996000',
'2017-01-01 12:00:09.997000', '2017-01-01 12:00:09.998000',
'2017-01-01 12:00:09.999000', '2017-01-01 12:00:10'],
dtype='datetime64[ns]', length=10001, freq='L')
DatetimeIndex([ '2017-01-01 12:00:00', '2017-01-01 12:00:00.000001',
'2017-01-01 12:00:00.000002', '2017-01-01 12:00:00.000003',
'2017-01-01 12:00:00.000004', '2017-01-01 12:00:00.000005',
'2017-01-01 12:00:00.000006', '2017-01-01 12:00:00.000007',
'2017-01-01 12:00:00.000008', '2017-01-01 12:00:00.000009',
...
'2017-01-01 12:00:09.999991', '2017-01-01 12:00:09.999992',
'2017-01-01 12:00:09.999993', '2017-01-01 12:00:09.999994',
'2017-01-01 12:00:09.999995', '2017-01-01 12:00:09.999996',
'2017-01-01 12:00:09.999997', '2017-01-01 12:00:09.999998',
'2017-01-01 12:00:09.999999', '2017-01-01 12:00:10'],
dtype='datetime64[ns]', length=10000001, freq='U')
DatetimeIndex(['2017-01-02', '2017-01-09', '2017-01-16', '2017-01-23',
'2017-01-30'],
dtype='datetime64[ns]', freq='W-MON')
DatetimeIndex(['2017-01-09', '2017-02-13', '2017-03-13', '2017-04-10'], dtype='datetime64[ns]', freq='WOM-2MON')
pd.date_range()-日期范围：频率(2)
print(pd.date_range('2017','2018', freq = 'M'))
print(pd.date_range('2017','2020', freq = 'Q-DEC'))
print(pd.date_range('2017','2020', freq = 'A-DEC'))
print('------')
# M：每月最后一个日历日
# Q-月：指定月为季度末，每个季度末最后一月的最后一个日历日
# A-月：每年指定月份的最后一个日历日
# 月缩写：JAN/FEB/MAR/APR/MAY/JUN/JUL/AUG/SEP/OCT/NOV/DEC
# 所以Q-月只有三种情况：1-4-7-10,2-5-8-11,3-6-9-12
print(pd.date_range('2017','2018', freq = 'BM'))
print(pd.date_range('2017','2020', freq = 'BQ-DEC'))
print(pd.date_range('2017','2020', freq = 'BA-DEC'))
print('------')
# BM：每月最后一个工作日
# BQ-月：指定月为季度末，每个季度末最后一月的最后一个工作日
# BA-月：每年指定月份的最后一个工作日
print(pd.date_range('2017','2018', freq = 'MS'))
print(pd.date_range('2017','2020', freq = 'QS-DEC'))
print(pd.date_range('2017','2020', freq = 'AS-DEC'))
print('------')
# M：每月第一个日历日
# Q-月：指定月为季度末，每个季度末最后一月的第一个日历日
# A-月：每年指定月份的第一个日历日
print(pd.date_range('2017','2018', freq = 'BMS'))
print(pd.date_range('2017','2020', freq = 'BQS-DEC'))
print(pd.date_range('2017','2020', freq = 'BAS-DEC'))
print('------')
# BM：每月第一个工作日
# BQ-月：指定月为季度末，每个季度末最后一月的第一个工作日
# BA-月：每年指定月份的第一个工作日
DatetimeIndex(['2017-01-31', '2017-02-28', '2017-03-31', '2017-04-30',
'2017-05-31', '2017-06-30', '2017-07-31', '2017-08-31',
'2017-09-30', '2017-10-31', '2017-11-30', '2017-12-31'],
dtype='datetime64[ns]', freq='M')
DatetimeIndex(['2017-03-31', '2017-06-30', '2017-09-30', '2017-12-31',
'2018-03-31', '2018-06-30', '2018-09-30', '2018-12-31',
'2019-03-31', '2019-06-30', '2019-09-30', '2019-12-31'],
dtype='datetime64[ns]', freq='Q-DEC')
DatetimeIndex(['2017-12-31', '2018-12-31', '2019-12-31'], dtype='datetime64[ns]', freq='A-DEC')
------
DatetimeIndex(['2017-01-31', '2017-02-28', '2017-03-31', '2017-04-28',
'2017-05-31', '2017-06-30', '2017-07-31', '2017-08-31',
'2017-09-29', '2017-10-31', '2017-11-30', '2017-12-29'],
dtype='datetime64[ns]', freq='BM')
DatetimeIndex(['2017-03-31', '2017-06-30', '2017-09-29', '2017-12-29',
'2018-03-30', '2018-06-29', '2018-09-28', '2018-12-31',
'2019-03-29', '2019-06-28', '2019-09-30', '2019-12-31'],
dtype='datetime64[ns]', freq='BQ-DEC')
DatetimeIndex(['2017-12-29', '2018-12-31', '2019-12-31'], dtype='datetime64[ns]', freq='BA-DEC')
------
DatetimeIndex(['2017-01-01', '2017-02-01', '2017-03-01', '2017-04-01',
'2017-05-01', '2017-06-01', '2017-07-01', '2017-08-01',
'2017-09-01', '2017-10-01', '2017-11-01', '2017-12-01',
'2018-01-01'],
dtype='datetime64[ns]', freq='MS')
DatetimeIndex(['2017-03-01', '2017-06-01', '2017-09-01', '2017-12-01',
'2018-03-01', '2018-06-01', '2018-09-01', '2018-12-01',
'2019-03-01', '2019-06-01', '2019-09-01', '2019-12-01'],
dtype='datetime64[ns]', freq='QS-DEC')
DatetimeIndex(['2017-12-01', '2018-12-01', '2019-12-01'], dtype='datetime64[ns]', freq='AS-DEC')
------
DatetimeIndex(['2017-01-02', '2017-02-01', '2017-03-01', '2017-04-03',
'2017-05-01', '2017-06-01', '2017-07-03', '2017-08-01',
'2017-09-01', '2017-10-02', '2017-11-01', '2017-12-01',
'2018-01-01'],
dtype='datetime64[ns]', freq='BMS')
DatetimeIndex(['2017-03-01', '2017-06-01', '2017-09-01', '2017-12-01',
'2018-03-01', '2018-06-01', '2018-09-03', '2018-12-03',
'2019-03-01', '2019-06-03', '2019-09-02', '2019-12-02'],
dtype='datetime64[ns]', freq='BQS-DEC')
DatetimeIndex(['2017-12-01', '2018-12-03', '2019-12-02'], dtype='datetime64[ns]', freq='BAS-DEC')
------
pd.date_range()-日期范围：复合频率
print(pd.date_range('2017/1/1','2017/2/1', freq = '7D')) # 7天
print(pd.date_range('2017/1/1','2017/1/2', freq = '2h30min')) # 2小时30分钟
print(pd.date_range('2017','2018', freq = '2M')) # 2月，每月最后一个日历日
DatetimeIndex(['2017-01-01', '2017-01-08', '2017-01-15', '2017-01-22',
'2017-01-29'],
dtype='datetime64[ns]', freq='7D')
DatetimeIndex(['2017-01-01 00:00:00', '2017-01-01 02:30:00',
'2017-01-01 05:00:00', '2017-01-01 07:30:00',
'2017-01-01 10:00:00', '2017-01-01 12:30:00',
'2017-01-01 15:00:00', '2017-01-01 17:30:00',
'2017-01-01 20:00:00', '2017-01-01 22:30:00'],
dtype='datetime64[ns]', freq='150T')
DatetimeIndex(['2017-01-31', '2017-03-31', '2017-05-31', '2017-07-31',
'2017-09-30', '2017-11-30'],
dtype='datetime64[ns]', freq='2M')
asfreq：时期频率转换
ts = pd.Series(np.random.rand(4),
index = pd.date_range('20170101','20170104'))
print(ts)
print(ts.asfreq('4H',method = 'ffill'))
# 改变频率，这里是D改为4H
# method：插值模式，None不插值，ffill用之前值填充，bfill用之后值填充
2017-01-01 0.380807
2017-01-02 0.082508
2017-01-03 0.989844
2017-01-04 0.946656
Freq: D, dtype: float64
2017-01-01 00:00:00 0.380807
2017-01-01 04:00:00 0.380807
2017-01-01 08:00:00 0.380807
2017-01-01 12:00:00 0.380807
2017-01-01 16:00:00 0.380807
2017-01-01 20:00:00 0.380807
2017-01-02 00:00:00 0.082508
2017-01-02 04:00:00 0.082508
2017-01-02 08:00:00 0.082508
2017-01-02 12:00:00 0.082508
2017-01-02 16:00:00 0.082508
2017-01-02 20:00:00 0.082508
2017-01-03 00:00:00 0.989844
2017-01-03 04:00:00 0.989844
2017-01-03 08:00:00 0.989844
2017-01-03 12:00:00 0.989844
2017-01-03 16:00:00 0.989844
2017-01-03 20:00:00 0.989844
2017-01-04 00:00:00 0.946656
Freq: 4H, dtype: float64
pd.date_range()-日期范围：超前/滞后数据
ts = pd.Series(np.random.rand(4),
index = pd.date_range('20170101','20170104'))
print(ts)
print(ts.shift(2))
print(ts.shift(-2))
print('------')
# 正数：数值后移（滞后）；负数：数值前移（超前）
per = ts/ts.shift(1) - 1
print(per)
print('------')
# 计算变化百分比，这里计算：该时间戳与上一个时间戳相比，变化百分比
print(ts.shift(2, freq = 'D'))
print(ts.shift(2, freq = 'T'))
# 加上freq参数：对时间戳进行位移，而不是对数值进行位移
2017-01-01 0.887591
2017-01-02 0.220969
2017-01-03 0.397006
2017-01-04 0.392343
Freq: D, dtype: float64
2017-01-01 NaN
2017-01-02 NaN
2017-01-03 0.887591
2017-01-04 0.220969
Freq: D, dtype: float64
2017-01-01 0.397006
2017-01-02 0.392343
2017-01-03 NaN
2017-01-04 NaN
Freq: D, dtype: float64
------
2017-01-01 NaN
2017-01-02 -0.751046
2017-01-03 0.796660
2017-01-04 -0.011747
Freq: D, dtype: float64
------
2017-01-03 0.887591
2017-01-04 0.220969
2017-01-05 0.397006
2017-01-06 0.392343
Freq: D, dtype: float64
2017-01-01 00:02:00 0.887591
2017-01-02 00:02:00 0.220969
2017-01-03 00:02:00 0.397006
2017-01-04 00:02:00 0.392343
Freq: D, dtype: float64
Period（Pandas时期）
pd.Period()创建时期
p = pd.Period('2017', freq = 'M')
print(p, type(p))
# 生成一个以2017-01开始，月为频率的时间构造器
# pd.Period()参数：一个时间戳 + freq 参数 → freq 用于指明该 period 的长度，时间戳则说明该 period 在时间轴上的位置
print(p + 1)
print(p - 2)
print(pd.Period('2012', freq = 'A-DEC') - 1)
# 通过加减整数，将周期整体移动
# 这里是按照 月、年 移动
2017-01 <class 'pandas._libs.tslibs.period.Period'>
2017-02
2016-11
2011
pd.period_range()创建时期范围
prng = pd.period_range('1/1/2011', '1/1/2012', freq='M')
print(prng,type(prng))
print(prng[0],type(prng[0]))
# 数据格式为PeriodIndex，单个数值为Period
ts = pd.Series(np.random.rand(len(prng)), index = prng)
print(ts,type(ts))
print(ts.index)
PeriodIndex(['2011-01', '2011-02', '2011-03', '2011-04', '2011-05', '2011-06',
'2011-07', '2011-08', '2011-09', '2011-10', '2011-11', '2011-12',
'2012-01'],
dtype='period[M]', freq='M') <class 'pandas.core.indexes.period.PeriodIndex'>
2011-01 <class 'pandas._libs.tslibs.period.Period'>
2011-01 0.824645
2011-02 0.903224
2011-03 0.531356
2011-04 0.232478
2011-05 0.410850
2011-06 0.604149
2011-07 0.680435
2011-08 0.039116
2011-09 0.600320
2011-10 0.503596
2011-11 0.995603
2011-12 0.810485
2012-01 0.985762
Freq: M, dtype: float64 <class 'pandas.core.series.Series'>
PeriodIndex(['2011-01', '2011-02', '2011-03', '2011-04', '2011-05', '2011-06',
'2011-07', '2011-08', '2011-09', '2011-10', '2011-11', '2011-12',
'2012-01'],
dtype='period[M]', freq='M')
时间序列
Period('2011', freq = 'A-DEC')可以看成多个时间期的时间段中的游标
Timestamp表示一个时间戳，是一个时间截面；Period是一个时期，是一个时间段！！但两者作为index时区别不大
asfreq：频率转换
p = pd.Period('2017','A-DEC')
print(p)
print(p.asfreq('M', how = 'start')) # 也可写 how = 's'
print(p.asfreq('D', how = 'end')) # 也可写 how = 'e'
# 通过.asfreq(freq, method=None, how=None)方法转换成别的频率
prng = pd.period_range('2017','2018',freq = 'M')
ts1 = pd.Series(np.random.rand(len(prng)), index = prng)
ts2 = pd.Series(np.random.rand(len(prng)), index = prng.asfreq('D', how = 'start'))
print(ts1.head(),len(ts1))
print(ts2.head(),len(ts2))
# asfreq也可以转换TIMESeries的index
2017
2017-01
2017-12-31
2017-01 0.803362
2017-02 0.680803
2017-03 0.503056
2017-04 0.885740
2017-05 0.983818
Freq: M, dtype: float64 13
2017-01-01 0.544769
2017-02-01 0.378608
2017-03-01 0.119503
2017-04-01 0.238595
2017-05-01 0.497255
Freq: D, dtype: float64 13
时间戳与时期之间的转换
rng = pd.date_range('2017/1/1', periods = 10, freq = 'M')
prng = pd.period_range('2017','2018', freq = 'M')
ts1 = pd.Series(np.random.rand(len(rng)), index = rng)
print(ts1.head())
print(ts1.to_period().head())
# 每月最后一日，转化为每月
ts2 = pd.Series(np.random.rand(len(prng)), index = prng)
print(ts2.head())
print(ts2.to_timestamp().head())
# 每月，转化为每月第一天
2017-01-31 0.867821
2017-02-28 0.543612
2017-03-31 0.893282
2017-04-30 0.809654
2017-05-31 0.464549
Freq: M, dtype: float64
2017-01 0.867821
2017-02 0.543612
2017-03 0.893282
2017-04 0.809654
2017-05 0.464549
Freq: M, dtype: float64
2017-01 0.731718
2017-02 0.517833
2017-03 0.473853
2017-04 0.009430
2017-05 0.998953
Freq: M, dtype: float64
2017-01-01 0.731718
2017-02-01 0.517833
2017-03-01 0.473853
2017-04-01 0.009430
2017-05-01 0.998953
Freq: MS, dtype: float64
时间序列 - 索引及切片
TimeSeries是Series的一个子类，所以Series索引及数据选取方面的方法基本一样
同时TimeSeries通过时间序列有更便捷的方法做索引和切片
索引
from datetime import datetime
rng = pd.date_range('2017/1','2017/3')
ts = pd.Series(np.random.rand(len(rng)), index = rng)
print(ts.head())
print(ts[0])
print(ts[:2])
print('-----')
# 基本下标位置索引
print(ts['2017/1/2'])
print(ts['20170103'])
print(ts['1/10/2017'])
print(ts[datetime(2017,1,20)])
print('-----')
# 时间序列标签索引，支持各种时间字符串，以及datetime.datetime
# 时间序列由于按照时间先后排序，故不用考虑顺序问题
# 索引方法同样适用于Dataframe
2017-01-01 0.790453
2017-01-02 0.252198
2017-01-03 0.360769
2017-01-04 0.525272
2017-01-05 0.452863
Freq: D, dtype: float64
0.7904527728537566
2017-01-01 0.790453
2017-01-02 0.252198
Freq: D, dtype: float64
-----
0.25219772923312767
0.3607686710038176
0.6528232574813878
0.352811458536781
-----
切片
rng = pd.date_range('2017/1','2017/3',freq = '12H')
ts = pd.Series(np.random.rand(len(rng)), index = rng)
print(ts['2017/1/5':'2017/1/10'])
print('-----')
# 和Series按照index索引原理一样，也是末端包含
print(ts['2017/2'].head())
# 传入月，直接得到一个切片
2017-01-05 00:00:00 0.042829
2017-01-05 12:00:00 0.441050
2017-01-06 00:00:00 0.400631
2017-01-06 12:00:00 0.026122
2017-01-07 00:00:00 0.493281
2017-01-07 12:00:00 0.559460
2017-01-08 00:00:00 0.673397
2017-01-08 12:00:00 0.682465
2017-01-09 00:00:00 0.160642
2017-01-09 12:00:00 0.048712
2017-01-10 00:00:00 0.668822
2017-01-10 12:00:00 0.734941
Freq: 12H, dtype: float64
-----
2017-02-01 00:00:00 0.528840
2017-02-01 12:00:00 0.692276
2017-02-02 00:00:00 0.812610
2017-02-02 12:00:00 0.620347
2017-02-03 00:00:00 0.877217
Freq: 12H, dtype: float64
重复索引的时间序列
dates = pd.DatetimeIndex(['1/1/2015','1/2/2015','1/3/2015','1/4/2015','1/1/2015','1/2/2015'])
ts = pd.Series(np.random.rand(6), index = dates)
print(ts)
print(ts.is_unique,ts.index.is_unique)
print('-----')
# index有重复，is_unique检查 → values唯一，index不唯一
print(ts['20150101'],type(ts['20150101']))
print(ts['20150104'],type(ts['20150104']))
print('-----')
# index有重复的将返回多个值
print(ts.groupby(level = 0).mean())
# 通过groupby做分组，重复的值这里用平均值处理
2015-01-01 0.987115
2015-01-02 0.331198
2015-01-03 0.900640
2015-01-04 0.906425
2015-01-01 0.090501
2015-01-02 0.841942
dtype: float64
True False
-----
2015-01-01 0.987115
2015-01-01 0.090501
dtype: float64 <class 'pandas.core.series.Series'>
2015-01-04 0.906425
dtype: float64 <class 'pandas.core.series.Series'>
-----
2015-01-01 0.538808
2015-01-02 0.586570
2015-01-03 0.900640
2015-01-04 0.906425
dtype: float64
时间序列 - 重采样
将时间序列从一个频率转换为另一个频率的过程，且会有数据的结合
降采样：高频数据 → 低频数据，eg.以天为频率的数据转为以月为频率的数据
升采样：低频数据 → 高频数据，eg.以年为频率的数据转为以月为频率的数据
重采样：.resample()
# 创建一个以天为频率的TimeSeries，重采样为按2天为频率
rng = pd.date_range('20170101', periods = 12)
ts = pd.Series(np.arange(12), index = rng)
print(ts)
ts_re = ts.resample('5D')
ts_re2 = ts.resample('5D').sum()
print(ts_re, type(ts_re))
print(ts_re2, type(ts_re2))
print('-----')
# ts.resample('5D')：得到一个重采样构建器，频率改为5天
# ts.resample('5D').sum():得到一个新的聚合后的Series，聚合方式为求和
# freq：重采样频率 → ts.resample('5D')
# .sum()：聚合方法
print(ts.resample('5D').mean(),'→ 求平均值\n')
print(ts.resample('5D').max(),'→ 求最大值\n')
print(ts.resample('5D').min(),'→ 求最小值\n')
print(ts.resample('5D').median(),'→ 求中值\n')
print(ts.resample('5D').first(),'→ 返回第一个值\n')
print(ts.resample('5D').last(),'→ 返回最后一个值\n')
print(ts.resample('5D').ohlc(),'→ OHLC重采样\n')
# OHLC:金融领域的时间序列聚合方式 → open开盘、high最大值、low最小值、close收盘
2017-01-01 0
2017-01-02 1
2017-01-03 2
2017-01-04 3
2017-01-05 4
2017-01-06 5
2017-01-07 6
2017-01-08 7
2017-01-09 8
2017-01-10 9
2017-01-11 10
2017-01-12 11
Freq: D, dtype: int32
DatetimeIndexResampler [freq=<5 * Days>, axis=0, closed=left, label=left, convention=start, base=0] <class 'pandas.core.resample.DatetimeIndexResampler'>
2017-01-01 10
2017-01-06 35
2017-01-11 21
dtype: int32 <class 'pandas.core.series.Series'>
-----
2017-01-01 2.0
2017-01-06 7.0
2017-01-11 10.5
dtype: float64 → 求平均值
2017-01-01 4
2017-01-06 9
2017-01-11 11
dtype: int32 → 求最大值
2017-01-01 0
2017-01-06 5
2017-01-11 10
dtype: int32 → 求最小值
2017-01-01 2.0
2017-01-06 7.0
2017-01-11 10.5
dtype: float64 → 求中值
2017-01-01 0
2017-01-06 5
2017-01-11 10
dtype: int32 → 返回第一个值
2017-01-01 4
2017-01-06 9
2017-01-11 11
dtype: int32 → 返回最后一个值
open high low close
2017-01-01 0 4 0 4
2017-01-06 5 9 5 9
2017-01-11 10 11 10 11 → OHLC重采样
降采样
rng = pd.date_range('20170101', periods = 12)
ts = pd.Series(np.arange(1,13), index = rng)
print(ts)
print(ts.resample('5D').sum(),'→ 默认\n')
print(ts.resample('5D', closed = 'left').sum(),'→ left\n')
print(ts.resample('5D', closed = 'right').sum(),'→ right\n')
print('-----')
# closed：各时间段哪一端是闭合（即包含）的，默认 左闭右闭
# 详解：这里values为0-11，按照5D重采样 → [1,2,3,4,5],[6,7,8,9,10],[11,12]
# left指定间隔左边为结束 → [1,2,3,4,5],[6,7,8,9,10],[11,12]
# right指定间隔右边为结束 → [1],[2,3,4,5,6],[7,8,9,10,11],[12]
print(ts.resample('5D', label = 'left').sum(),'→ leftlabel\n')
print(ts.resample('5D', label = 'right').sum(),'→ rightlabel\n')
# label：聚合值的index，默认为取左
# 值采样认为默认（这里closed默认）
2017-01-01 1
2017-01-02 2
2017-01-03 3
2017-01-04 4
2017-01-05 5
2017-01-06 6
2017-01-07 7
2017-01-08 8
2017-01-09 9
2017-01-10 10
2017-01-11 11
2017-01-12 12
Freq: D, dtype: int32
2017-01-01 15
2017-01-06 40
2017-01-11 23
dtype: int32 → 默认
2017-01-01 15
2017-01-06 40
2017-01-11 23
dtype: int32 → left
2016-12-27 1
2017-01-01 20
2017-01-06 45
2017-01-11 12
dtype: int32 → right
-----
2017-01-01 15
2017-01-06 40
2017-01-11 23
dtype: int32 → leftlabel
2017-01-06 15
2017-01-11 40
2017-01-16 23
dtype: int32 → rightlabel
升采样及插值
rng = pd.date_range('2017/1/1 0:0:0', periods = 5, freq = 'H')
ts = pd.DataFrame(np.arange(15).reshape(5,3),
index = rng,
columns = ['a','b','c'])
print(ts)
print(ts.resample('15T').asfreq())
print(ts.resample('15T').ffill())
print(ts.resample('15T').bfill())
# 低频转高频，主要是如何插值
# .asfreq()：不做填充，返回Nan
# .ffill()：向上填充
# .bfill()：向下填充
a b c
2017-01-01 00:00:00 0 1 2
2017-01-01 01:00:00 3 4 5
2017-01-01 02:00:00 6 7 8
2017-01-01 03:00:00 9 10 11
2017-01-01 04:00:00 12 13 14
a b c
2017-01-01 00:00:00 0.0 1.0 2.0
2017-01-01 00:15:00 NaN NaN NaN
2017-01-01 00:30:00 NaN NaN NaN
2017-01-01 00:45:00 NaN NaN NaN
2017-01-01 01:00:00 3.0 4.0 5.0
2017-01-01 01:15:00 NaN NaN NaN
2017-01-01 01:30:00 NaN NaN NaN
2017-01-01 01:45:00 NaN NaN NaN
2017-01-01 02:00:00 6.0 7.0 8.0
2017-01-01 02:15:00 NaN NaN NaN
2017-01-01 02:30:00 NaN NaN NaN
2017-01-01 02:45:00 NaN NaN NaN
2017-01-01 03:00:00 9.0 10.0 11.0
2017-01-01 03:15:00 NaN NaN NaN
2017-01-01 03:30:00 NaN NaN NaN
2017-01-01 03:45:00 NaN NaN NaN
2017-01-01 04:00:00 12.0 13.0 14.0
a b c
2017-01-01 00:00:00 0 1 2
2017-01-01 00:15:00 0 1 2
2017-01-01 00:30:00 0 1 2
2017-01-01 00:45:00 0 1 2
2017-01-01 01:00:00 3 4 5
2017-01-01 01:15:00 3 4 5
2017-01-01 01:30:00 3 4 5
2017-01-01 01:45:00 3 4 5
2017-01-01 02:00:00 6 7 8
2017-01-01 02:15:00 6 7 8
2017-01-01 02:30:00 6 7 8
2017-01-01 02:45:00 6 7 8
2017-01-01 03:00:00 9 10 11
2017-01-01 03:15:00 9 10 11
2017-01-01 03:30:00 9 10 11
2017-01-01 03:45:00 9 10 11
2017-01-01 04:00:00 12 13 14
a b c
2017-01-01 00:00:00 0 1 2
2017-01-01 00:15:00 3 4 5
2017-01-01 00:30:00 3 4 5
2017-01-01 00:45:00 3 4 5
2017-01-01 01:00:00 3 4 5
2017-01-01 01:15:00 6 7 8
2017-01-01 01:30:00 6 7 8
2017-01-01 01:45:00 6 7 8
2017-01-01 02:00:00 6 7 8
2017-01-01 02:15:00 9 10 11
2017-01-01 02:30:00 9 10 11
2017-01-01 02:45:00 9 10 11
2017-01-01 03:00:00 9 10 11
2017-01-01 03:15:00 12 13 14
2017-01-01 03:30:00 12 13 14
2017-01-01 03:45:00 12 13 14
2017-01-01 04:00:00 12 13 14
时期重采样 - Period
prng = pd.period_range('2016','2017',freq = 'M')
ts = pd.Series(np.arange(len(prng)), index = prng)
print(ts)
#print(ts.resample('3M').sum())
# 降采样
print(ts.resample('15D').ffill()) # 升采样
2016-01 0
2016-02 1
2016-03 2
2016-04 3
2016-05 4
2016-06 5
2016-07 6
2016-08 7
2016-09 8
2016-10 9
2016-11 10
2016-12 11
2017-01 12
Freq: M, dtype: int32
2016-01-01 0
2016-01-16 0
2016-01-31 0
2016-02-15 1
2016-03-01 2
2016-03-16 2
2016-03-31 2
2016-04-15 3
2016-04-30 3
2016-05-15 4
2016-05-30 4
2016-06-14 5
2016-06-29 5
2016-07-14 6
2016-07-29 6
2016-08-13 7
2016-08-28 7
2016-09-12 8
2016-09-27 8
2016-10-12 9
2016-10-27 9
2016-11-11 10
2016-11-26 10
2016-12-11 11
2016-12-26 11
2017-01-10 12
2017-01-25 12
Freq: 15D, dtype: int3gou
