前情提要：

• 概述
  • Julia是一门牛B、‘无耻’的语言
• 本地环境教程
  • Julia1.0.0安装指南(含 Juno IDE)
    • Windows教程
    • Mac教程
  • 目前兼容的机器学习程序包
• 在线环境教程
  • 无痛体验：几行代码识别图片内容
  • 如何进行Julia无痛体验
  • 深度定制免费无痛环境
  • 如何科学的找程序包

前段时间把环境和各种版本情况理了一遍，应该说该有的学习基础设施都有了。

笔者关注的是机器学习方面的，因此会侧重去看这些方面的资料。

说到机器学习，首先得要有数据，不然学习个啥呢。有了数据之后，那么很多语言的第一步就是处理数据。Julia也一样，有专门的数据处理程序包。

今天就说一下DataFrames这个程序包。用过其他机器学习语言的都知道，DataFrames就是数据框，中文直译。

笔者自己学习程序语言的方式是不喜欢去看书的（程序书是用来查的），程序一定是一边写一边看一边用才会掌握的好和快，尤其是看牛人写的程序。

记得前面有教程说过如何导入Github上的程序库吧。

来，先记得这个地址：https://github.com/scidom/StatsLearningByExample.jl.git

然后点开Juliabox上Git 这个按钮

会出来一个对话框：

粘贴刚才那个地址到 Git Clone URL框里，然后点后面的+号（如果需要修改一下同步到Juliabox的文件夹名字的请按+号前修改）。然后按下[OK]。

很快Juliabox就会把https://github.com/scidom/StatsLearningByExample.jl.git里的内容同步过来。

然后你的Juliabox就会多出来这个文件夹（StatsLearningByExample）：

接下来点进去02-DataFrames目录：

点开第一个02-01-DataFramesBasics.ipynb,

然后点Cell下面的Run All:

这个命令就会把所有的代码跑一遍。

这个就是今天的课程，因为已经有代码且作者已经用英文讲解了，所以接下来会倒序（按照程序代码来说是倒序）重点总结一下：

（文末有笔者运行结果，方便暂时不能操作的同学看）：

• DataFrames是由DataArrays组成的

  • DataFrames本身的基本信息获取
  • 可以对DataFrames内数据进行简单统计描述
  • 可以获得DataFrames某行/某列数据

• DataArrays可以进行矩阵运算

• DataArrays里有个特殊成员NA（缺失值）

  • 所有数值和NA进行运算结果都是NA

  • NA（缺失值）和NaN（Not a Number: 不是数字）是两个东西，数值类型也不一样

有几个注意的地方：

这个地方其实应该是文档的注释部分，作者应该没注意弄，变成代码运行报错了。

改成文档格式即可，忽略也可以。

还有这里：

第8句写了不可能通过DataArray([0.1, NA, -2.4])语句直接完成DataArray构建，这里应该是个错误示范（应该报错）。

作者希望提示大家用第9句的语法来完成构建动作。

实际上，作者写的是2年前，现在我们看到的是两种语法都得出了正确结果。也就是目前两种写法都可以。

其他的就不多解释了，大家要学习应该能看懂。

以下是笔者运行的结果，供参考：

Introduction to DataFrames

In [1]:

using DataArrays
using DataFrames

Missing values¶

• A missing value is represented by NA in Julia.
• NA is not part of Base, it is provided by the DataArrays package.
• NA poisons other values.

In [2]:

# NA poisons other values
1+NA

Out[2]:

missing

In [3]:

# Check if the evaluation of an expression results in NA
isna(1+NA)

Out[3]:

true

In [4]:

# Note the difference between NaN and NA
(isa(NaN, Float64), isa(NA, Float64))

Out[4]:

(true, false)

DataArrays

• DataArray's are used for representing arrays that contain missing values
• DataArray{T} allows storing T or NA
• In other words, DataArray{T} adds NA's to Array{T}
• PooledDataArray{T} is used for storing data efficiently.
• PooledDataArray{T} compresses DataArray{T}.

Constructing DataArrays

In [5]:

# Call the DataArray() constructor by passing a Vector to it
DataArray([0.1, 0.5, -2.4])

Out[5]:

3-element DataArrays.DataArray{Float64,1}:
  0.1
  0.5
 -2.4

In [6]:

# Construct a DataArray by calling the @data() macro with a Vector input argument
@data([0.1, 0.5, -2.4])

Out[6]:

3-element DataArrays.DataArray{Float64,1}:
  0.1
  0.5
 -2.4

In [7]:

# Convert Vector to DataArray
convert(DataArray, [0.1, 0.5, -2.4])

Out[7]:

3-element DataArrays.DataArray{Float64,1}:
  0.1
  0.5
 -2.4

In [8]:

# It is not possible to call DataArray() with NA in its input argument
DataArray([0.1, NA, -2.4])

Out[8]:

3-element DataArrays.DataArray{Float64,1}:
  0.1     
   missing
 -2.4     

In [9]:

# However, it is possible to pass NA to the @data() macro
@data([0.1, NA, -2.4])

Out[9]:

3-element DataArrays.DataArray{Float64,1}:
  0.1     
   missing
 -2.4     

In [10]:

# The DataArray() constructor can be called with a Matrix input argument
DataArray([0.4 1.2; 3.5 7.2])

Out[10]:

2×2 DataArrays.DataArray{Float64,2}:
 0.4  1.2
 3.5  7.2

In [11]:

# The @data() macro can also be called with a Matrix input argument
@data([0.4 1.2; 3.5 7.2])

Out[11]:

2×2 DataArrays.DataArray{Float64,2}:
 0.4  1.2
 3.5  7.2

In [12]:

# Convert a Matrix to DataArray
convert(DataArray, [0.4 1.2; 3.5 7.2])

Out[12]:

2×2 DataArrays.DataArray{Float64,2}:
 0.4  1.2
 3.5  7.2

Numerical computing with DataArrays

In [13]:

# Numerical computing can be done with data vectors
x = @data([0.1, NA, -2.4])
y = @data([-9.9, 0.5, 6.7])
x+y

Out[13]:

3-element DataArrays.DataArray{Float64,1}:
 -9.8     
   missing
  4.3     

In [14]:

# To remove missing values (NA), call dropna()
x = @data([0.1, NA, -2.4])
dropna(x)

Out[14]:

2-element Array{Float64,1}:
  0.1
 -2.4

In [15]:

# Numerical computing can be done with data matrices and data vectors
A = @data([0.4 1.2 4.4; NA 7.2 3.9; 5.1 1.8 4.5])
y = @data([-9.9, 0.5, 6.7])
A*y

Out[15]:

3-element DataArrays.DataArray{Float64,1}:
  26.12    
    missing
 -19.44    

DataFrames

• DataFrame's are used for representing data tables.
• A DataFrame is a list of DataArray's.
• So every DataArray of a DataFrame represents a column of the corresponding data table.
• DataFrame's accommodate heterogeneous data that might contain missing values.
• Every column (DataArray) of a DataFrame has its own type.

Example 02-01-01: NBA champions

Constructing DataFrames

In [16]:

# Call the DataFrame() constructor with keyword arguments (columns) of type Vector
DataFrame(
  player = ["Larry Bird", "Magic Johnson", "Michael Jordan", "Scottie Pippen"],
  champions = [3, 5, 6, 6]
)

Out[16]:

	player	champions
1	Larry Bird	3
2	Magic Johnson	5
3	Michael Jordan	6
4	Scottie Pippen	6

In [17]:

# Start with an empty DataFrame and populate it
ChampionsFrame = DataFrame()
ChampionsFrame[:player] = ["Larry Bird", "Magic Johnson", "Michael Jordan", "Scottie Pippen"]
ChampionsFrame[:champions] = [3, 5, 6, 6]
ChampionsFrame

Out[17]:

	player	champions
1	Larry Bird	3
2	Magic Johnson	5
3	Michael Jordan	6
4	Scottie Pippen	6

Provide CSV-like tabular data to construct a new DataFrame

In [19]:

# Call the DataFrame() constructor with keyword arguments (columns) of type DataArray
player = @data(["Larry Bird", "Magic Johnson", "Michael Jordan", "Scottie Pippen"])
champions = @data([3, 5, 6, 6])
ChampionsFrame = DataFrame(player=player, champions=champions)

Out[19]:

	player	champions
1	Larry Bird	3
2	Magic Johnson	5
3	Michael Jordan	6
4	Scottie Pippen	6

In [20]:

# Construct a DataFrame by joining two existing DataFrames
height = [2.06, 2.06, 1.98, 2.03]
HeightsFrame = DataFrame(player=player, height=height)
join(ChampionsFrame, HeightsFrame, on = :player)

Out[20]:

	player	champions	height
1	Larry Bird	3	2.06
2	Magic Johnson	5	2.06
3	Michael Jordan	6	1.98
4	Scottie Pippen	6	2.03

Quering basic information about DataFrames

In [21]:

# Get number of rows of a DataFrame
size(ChampionsFrame, 1)

Out[21]:

4

In [22]:

# Get number of columns of a DataFrame
size(ChampionsFrame, 2)

Out[22]:

2

In [23]:

# Get a numeric summary of a DataFrame
describe(ChampionsFrame)

Out[23]:

	variable	mean	min	median	max	nunique	nmissing	eltype
1	player		Larry Bird		Scottie Pippen	4	0	String
2	champions	5.0	3	5.5	6		0	Int64

Indexing DataFrames

In [24]:

# Index DataFrame by column name to get a specific column
ChampionsFrame[:player]

Out[24]:

4-element DataArrays.DataArray{String,1}:
 "Larry Bird"    
 "Magic Johnson" 
 "Michael Jordan"
 "Scottie Pippen"

In [25]:

# Index DataFrame by row numbers to get specific rows
ChampionsFrame[2:3, :]

Out[25]:

	player	champions
1	Magic Johnson	5
2	Michael Jordan	6

KevinZhang
Aug 30, 2018