K-近邻算法

原理

• 已有一系列带标签的数据，通过计算新数据与带标签数据的相似度（距离），来判定新数据是属于哪一类数据。其中，
  计算新样本与所有带标签数据之间所得到的距离，按从小到大排序，选取前K个样本，选择这K个样本出现次数最多的类别作为新数据的类别。

优点

• 原理简单，精度高，对异常数据不敏感(样本类别有K个邻居决定)。

缺点

• 时间复杂和空间复杂度高，需要消耗很高的计算时间和内存空间。

适用数据类型：

• 标称型和数值型数据。

使用k-近邻算法改进约会网站的配对效果

import pandas as pd
import numpy as np

df = pd.read_csv('../../Reference Code/datingTestSet2.txt',sep='\t', names=['kmPerYear','gameTime','iceCream','label'])
datingDateMat = df.iloc[:,:-1].as_matrix()
datingLabels = df.iloc[:,-1].as_matrix()
print(df.head())
print('datingDateMat : ',datingDateMat)
print('datingLabels : ',datingLabels[:20])

   kmPerYear   gameTime  iceCream  label
0      40920   8.326976  0.953952      3
1      14488   7.153469  1.673904      2
2      26052   1.441871  0.805124      1
3      75136  13.147394  0.428964      1
4      38344   1.669788  0.134296      1
datingDateMat :  [[4.0920000e+04 8.3269760e+00 9.5395200e-01]
 [1.4488000e+04 7.1534690e+00 1.6739040e+00]
 [2.6052000e+04 1.4418710e+00 8.0512400e-01]
 ...
 [2.6575000e+04 1.0650102e+01 8.6662700e-01]
 [4.8111000e+04 9.1345280e+00 7.2804500e-01]
 [4.3757000e+04 7.8826010e+00 1.3324460e+00]]
datingLabels :  [3 2 1 1 1 1 3 3 1 3 1 1 2 1 1 1 1 1 2 3]


/home/ubuntu/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:2: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
  
/home/ubuntu/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:3: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
  This is separate from the ipykernel package so we can avoid doing imports until

import matplotlib.pyplot as plt
fig = plt.figure()
plt.scatter(df.iloc[:,1], df.iloc[:,2],
            s=np.array(datingLabels),c=np.array(datingLabels))
plt.show()

output_3_0.png

fig = plt.figure()
plt.scatter(df.iloc[:,0], df.iloc[:,1],
            s=np.array(datingLabels),c=np.array(datingLabels))
plt.show()

output_4_0.png

'''
kNN算法原理:
1.计算新样本与每个训练样本的距离
2.选取前K个样本
3.统计前K个样本的类别
4.选取出现最多次的类别作为新样本的类别
'''
import pandas as pd
def createDate():
    group = pd.DataFrame([[1.0,1.1],[1.0,1.0],[0,0],[0,0.1]])
    labels = pd.Series(['A','A','B','B'])
    return group, labels

def file2df(path,names):
    df = pd.read_csv(path, sep='\t', names=names)
    xTrain = df.iloc[:, :-1]
    yTrain = df.iloc[:, -1]
    return xTrain,yTrain

#归一化
def autoNorm(dataSet):
    minVal = dataSet.min(0)
    maxVal = dataSet.max(0)
    ranges = maxVal - minVal
    normDataSet = (dataSet - minVal )/ranges
    return normDataSet,ranges,minVal

def classfy0(inX, dataSet, labels, k):

    #计算距离
    dfSquare = (inX - dataSet)**2
    dfDistance = dfSquare.apply(lambda x:x.sum(),axis=1)**0.5

    #选取前K个样本
    sortDisIndex = list(dfDistance.argsort())
    countLabels ={}
    labels = list(labels)
    for i in range(k):
        kLabel = labels[sortDisIndex[i]]
        countLabels[kLabel] = countLabels.get(kLabel,0) + 1
    sortK = sorted(countLabels.items(), key=lambda d: d[1],reverse=True) #[('smallDoses', 2), ('largeDoses', 1)]
    return sortK[0][0]

def datingClassTest(path, names, testRate= 0.1, k =3):
    testRate = testRate
    x, y = file2df(path, names)
    normX,ranges,minVal = autoNorm(x)
    numTest = int(testRate * normX.shape[0])
    errCount = 0
    for i in range(numTest):
        resClass = classfy0(normX.iloc[i,:], normX.iloc[numTest:,:], y[numTest:], k)
        print('the classifier came with: %s, the real answer is: %s' %(resClass,y[i]))
        if resClass != y[i]: #resClass is str, y[i] is str
            errCount += 1
    print('the total error rate is: %f' %(errCount/numTest))

def classifyPerson(path,names,k=3):
    resultList = ['not at all','in small doses','in large doses']
    kmPerYear = float(input('percentage of time spent playing video games?'))
    gameTime = float(input('frequent flier miles earned per yaer?'))
    iceCream = float(input('liters of ice cream consumed per year?'))
    diffDate = pd.DataFrame([[kmPerYear,gameTime,iceCream]])
    xTrain ,yTrain = file2df(path,names)
    normDataSet, ranges, minVal = autoNorm(xTrain)
    normDiff = (diffDate - minVal)/ranges
    resClass = classfy0(normDiff.iloc[0],normDataSet,yTrain,k)
    print('You will probably like this person:', resultList[resClass-1])
#测试
# group, labels = createDate()
# inX = pd.DataFrame([[0.1,0]])
# label = classfy0(inX,group, labels, 3)

# #可视化
# import matplotlib.pyplot as plt
# plt.figure()
# plt.scatter(inX[0],inX[1],label= label)
# plt.scatter(group.iloc[:,0],group.iloc[:,1],label = labels)
# plt.legend()
# plt.show()

# 评估模型
import time
start = time.time()
datingClassTest('../../Reference Code/datingTestSet.txt',['kmPerYear','gameTime','iceCream','label'])
runtime = time.time()-start
print(runtime,"s")

# classifyPerson('../../Reference Code/datingTestSet2.txt',['kmPerYear','gameTime','iceCream','label'])

the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: largeDoses, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: largeDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: largeDoses, the real answer is: largeDoses
the classifier came with: smallDoses, the real answer is: smallDoses
the classifier came with: didntLike, the real answer is: didntLike
the classifier came with: largeDoses, the real answer is: didntLike
the total error rate is: 0.050000
6.98456597328186 s

使用k-近邻算法识别手写数字

import pandas as pd
from os import listdir
def classfy0(inX, dataSet, labels, k):
    '''
    inX is DataFrame
    dataSet is DataFrame
    labels is Series
    '''
    #计算距离
    dfSquare = (inX - dataSet)**2 
    dfDistance = dfSquare.apply(lambda x:x.sum(),axis=1)**0.5

    #选取前K个样本
    sortDisIndex = list(dfDistance.argsort())
    countLabels ={}
    labels = list(labels)
    for i in range(k):
        kLabel = labels[sortDisIndex[i]]
        countLabels[kLabel] = countLabels.get(kLabel,0) + 1
    sortK = sorted(countLabels.items(), key=lambda d: d[1],reverse=True) #[('5', 2), ('6', 1)]
    return sortK[0][0]

#获取文件名和类别
def getFilenameAndLabels(path):
    dirName = listdir(path)
    dicFileName = {}
    for fileName in dirName:
        dicFileName[fileName] = fileName.split('_')[0]
    return dicFileName
# getFilenameAndLabels('../../Reference Code/testDigits')

#img2list
def img2Vector(path):
    file = open(path)
    returnVect = []
    for i in range(32):
        lineStr = file.readline()
        for j in range(32):
            returnVect.append(int(lineStr[j]))
    return returnVect

#imgDataFrame
def imgDataFrame(path):
    fileName = getFilenameAndLabels(path)
    vecDataFrame = []
    labels = []
    for fileName, label in fileName.items():
        vecDataFrame.append(img2Vector(path+fileName)) ##或者可以用%s: img2Vector('../digits/trainingDigits/%s' % fileName)
        labels.append(label)
    vecDataFrame = pd.DataFrame(vecDataFrame)
    labels = pd.Series(labels)
    return vecDataFrame,labels
# vecDateFrame,labels = imgDataFrame('../../Reference Code/testDigits/')
def classifyImg(trainFile, testFile, k=3):
    trainData, trainLabels = imgDataFrame(trainFile)
    testData, testLabels = imgDataFrame(testFile)
    errCount = 0
    for i in range(testData.shape[0]):
        classfyResult = classfy0(testData.iloc[i,:], trainData, trainLabels, k)
        print('the classifier came with:  %d, the real answer is: %d' %(int(classfyResult),int(testLabels[i]) ))
        if classfyResult != testLabels[i]:
            errCount += 1
    print('the total number of errors is %d:' %int(errCount))
    print('the total error rate is %f' %(errCount/testData.shape[0]))
classifyImg(trainFile='../../Reference Code/trainingDigits/',testFile='../../Reference Code/testDigits/')

the classifier came with:  0, the real answer is: 0
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  6, the real answer is: 6
the classifier came with:  0, the real answer is: 0
the classifier came with:  8, the real answer is: 8
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  6, the real answer is: 6
the classifier came with:  8, the real answer is: 8
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  4, the real answer is: 4
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  6, the real answer is: 8
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  3, the real answer is: 3
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  1, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  8, the real answer is: 8
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  6, the real answer is: 6
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  4, the real answer is: 4
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  8, the real answer is: 8
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  7, the real answer is: 7
the classifier came with:  3, the real answer is: 3
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  1, the real answer is: 1
the classifier came with:  3, the real answer is: 3
the classifier came with:  2, the real answer is: 2
the classifier came with:  4, the real answer is: 4
the classifier came with:  7, the real answer is: 7
the classifier came with:  3, the real answer is: 3
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  8, the real answer is: 8
the classifier came with:  2, the real answer is: 2
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 7
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  9, the real answer is: 9
the classifier came with:  3, the real answer is: 3
the classifier came with:  9, the real answer is: 9
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  9, the real answer is: 9
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  9, the real answer is: 9
the classifier came with:  4, the real answer is: 4
the classifier came with:  3, the real answer is: 3
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  4, the real answer is: 4
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  5, the real answer is: 5
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  5, the real answer is: 5
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  1, the real answer is: 1
the classifier came with:  4, the real answer is: 4
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  8, the real answer is: 8
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  9, the real answer is: 9
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  4, the real answer is: 4
the classifier came with:  3, the real answer is: 3
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  8, the real answer is: 8
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  2, the real answer is: 2
the classifier came with:  8, the real answer is: 8
the classifier came with:  6, the real answer is: 6
the classifier came with:  3, the real answer is: 3
the classifier came with:  6, the real answer is: 6
the classifier came with:  9, the real answer is: 9
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  3, the real answer is: 3
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 1
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  0, the real answer is: 0
the classifier came with:  8, the real answer is: 8
the classifier came with:  2, the real answer is: 2
the classifier came with:  8, the real answer is: 8
the classifier came with:  6, the real answer is: 6
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  6, the real answer is: 6
the classifier came with:  1, the real answer is: 8
the classifier came with:  2, the real answer is: 2
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  2, the real answer is: 2
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  8, the real answer is: 8
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  1, the real answer is: 1
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  3, the real answer is: 3
the classifier came with:  0, the real answer is: 0
the classifier came with:  0, the real answer is: 0
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  9, the real answer is: 9
the classifier came with:  1, the real answer is: 1
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  8, the real answer is: 8
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  3, the real answer is: 3
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  1, the real answer is: 1
the classifier came with:  8, the real answer is: 8
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  3, the real answer is: 3
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  7, the real answer is: 7
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  0, the real answer is: 0
the classifier came with:  7, the real answer is: 7
the classifier came with:  3, the real answer is: 3
the classifier came with:  9, the real answer is: 9
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  0, the real answer is: 0
the classifier came with:  8, the real answer is: 8
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 1
the classifier came with:  0, the real answer is: 0
the classifier came with:  6, the real answer is: 6
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 1
the classifier came with:  6, the real answer is: 6
the classifier came with:  8, the real answer is: 8
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 7
the classifier came with:  9, the real answer is: 9
the classifier came with:  8, the real answer is: 8
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  8, the real answer is: 8
the classifier came with:  6, the real answer is: 6
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  5, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  0, the real answer is: 0
the classifier came with:  0, the real answer is: 0
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  0, the real answer is: 0
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  3, the real answer is: 3
the classifier came with:  2, the real answer is: 2
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  5, the real answer is: 5
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  2, the real answer is: 2
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  7, the real answer is: 7
the classifier came with:  3, the real answer is: 8
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  3, the real answer is: 3
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  3, the real answer is: 3
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  8, the real answer is: 8
the classifier came with:  4, the real answer is: 4
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  8, the real answer is: 8
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  8, the real answer is: 8
the classifier came with:  3, the real answer is: 3
the classifier came with:  3, the real answer is: 3
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  3, the real answer is: 3
the classifier came with:  1, the real answer is: 8
the classifier came with:  2, the real answer is: 2
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 1
the classifier came with:  3, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  6, the real answer is: 6
the classifier came with:  3, the real answer is: 3
the classifier came with:  0, the real answer is: 0
the classifier came with:  6, the real answer is: 6
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  1, the real answer is: 1
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  8, the real answer is: 8
the classifier came with:  8, the real answer is: 8
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  0, the real answer is: 0
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  6, the real answer is: 6
the classifier came with:  1, the real answer is: 1
the classifier came with:  0, the real answer is: 0
the classifier came with:  9, the real answer is: 9
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  9, the real answer is: 9
the classifier came with:  9, the real answer is: 9
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  6, the real answer is: 6
the classifier came with:  2, the real answer is: 2
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  6, the real answer is: 6
the classifier came with:  8, the real answer is: 8
the classifier came with:  3, the real answer is: 3
the classifier came with:  7, the real answer is: 7
the classifier came with:  8, the real answer is: 8
the classifier came with:  6, the real answer is: 6
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  5, the real answer is: 9
the classifier came with:  0, the real answer is: 0
the classifier came with:  7, the real answer is: 7
the classifier came with:  9, the real answer is: 9
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  5, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  3, the real answer is: 3
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  3, the real answer is: 3
the classifier came with:  5, the real answer is: 5
the classifier came with:  1, the real answer is: 9
the classifier came with:  8, the real answer is: 8
the classifier came with:  4, the real answer is: 4
the classifier came with:  6, the real answer is: 6
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  6, the real answer is: 6
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  6, the real answer is: 6
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  9, the real answer is: 9
the classifier came with:  3, the real answer is: 3
the classifier came with:  3, the real answer is: 3
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  8, the real answer is: 8
the classifier came with:  9, the real answer is: 9
the classifier came with:  1, the real answer is: 1
the classifier came with:  2, the real answer is: 2
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 7
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  8, the real answer is: 8
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  5, the real answer is: 5
the classifier came with:  7, the real answer is: 7
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  7, the real answer is: 7
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  9, the real answer is: 9
the classifier came with:  4, the real answer is: 4
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  6, the real answer is: 6
the classifier came with:  2, the real answer is: 2
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  6, the real answer is: 6
the classifier came with:  0, the real answer is: 0
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  9, the real answer is: 9
the classifier came with:  2, the real answer is: 2
the classifier came with:  1, the real answer is: 1
the classifier came with:  7, the real answer is: 7
the classifier came with:  7, the real answer is: 7
the classifier came with:  6, the real answer is: 6
the classifier came with:  6, the real answer is: 6
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  3, the real answer is: 3
the classifier came with:  8, the real answer is: 8
the classifier came with:  3, the real answer is: 3
the classifier came with:  4, the real answer is: 4
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  9, the real answer is: 9
the classifier came with:  6, the real answer is: 6
the classifier came with:  1, the real answer is: 1
the classifier came with:  9, the real answer is: 9
the classifier came with:  4, the real answer is: 4
the classifier came with:  1, the real answer is: 1
the classifier came with:  3, the real answer is: 3
the classifier came with:  2, the real answer is: 2
the classifier came with:  2, the real answer is: 2
the classifier came with:  5, the real answer is: 5
the classifier came with:  3, the real answer is: 3
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  2, the real answer is: 2
the classifier came with:  6, the real answer is: 6
the classifier came with:  9, the real answer is: 9
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  8, the real answer is: 8
the classifier came with:  0, the real answer is: 0
the classifier came with:  1, the real answer is: 1
the classifier came with:  8, the real answer is: 8
the classifier came with:  6, the real answer is: 6
the classifier came with:  1, the real answer is: 1
the classifier came with:  1, the real answer is: 1
the classifier came with:  5, the real answer is: 5
the classifier came with:  4, the real answer is: 4
the classifier came with:  6, the real answer is: 6
the classifier came with:  3, the real answer is: 3
the classifier came with:  2, the real answer is: 2
the classifier came with:  0, the real answer is: 0
the classifier came with:  9, the real answer is: 9
the classifier came with:  7, the real answer is: 7
the classifier came with:  4, the real answer is: 4
the classifier came with:  5, the real answer is: 5
the classifier came with:  0, the real answer is: 0
the classifier came with:  5, the real answer is: 5
the classifier came with:  6, the real answer is: 6
the classifier came with:  4, the real answer is: 4
the classifier came with:  0, the real answer is: 0
the classifier came with:  7, the real answer is: 7

the total number of errors is 12:
the total error rate is 0.012685

使用KNN对鸢尾花Iris数据集分类

from sklearn.datasets import load_iris
iris = load_iris()
iris.data.shape

(150, 4)

print(iris.DESCR)

Iris Plants Database
====================

Notes
-----
Data Set Characteristics:
    :Number of Instances: 150 (50 in each of three classes)
    :Number of Attributes: 4 numeric, predictive attributes and the class
    :Attribute Information:
        - sepal length in cm
        - sepal width in cm
        - petal length in cm
        - petal width in cm
        - class:
                - Iris-Setosa
                - Iris-Versicolour
                - Iris-Virginica
    :Summary Statistics:

    ============== ==== ==== ======= ===== ====================
                    Min  Max   Mean    SD   Class Correlation
    ============== ==== ==== ======= ===== ====================
    sepal length:   4.3  7.9   5.84   0.83    0.7826
    sepal width:    2.0  4.4   3.05   0.43   -0.4194
    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)
    petal width:    0.1  2.5   1.20  0.76     0.9565  (high!)
    ============== ==== ==== ======= ===== ====================

    :Missing Attribute Values: None
    :Class Distribution: 33.3% for each of 3 classes.
    :Creator: R.A. Fisher
    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)
    :Date: July, 1988

This is a copy of UCI ML iris datasets.
http://archive.ics.uci.edu/ml/datasets/Iris

The famous Iris database, first used by Sir R.A Fisher

This is perhaps the best known database to be found in the
pattern recognition literature.  Fisher's paper is a classic in the field and
is referenced frequently to this day.  (See Duda & Hart, for example.)  The
data set contains 3 classes of 50 instances each, where each class refers to a
type of iris plant.  One class is linearly separable from the other 2; the
latter are NOT linearly separable from each other.

References
----------
   - Fisher,R.A. "The use of multiple measurements in taxonomic problems"
     Annual Eugenics, 7, Part II, 179-188 (1936); also in "Contributions to
     Mathematical Statistics" (John Wiley, NY, 1950).
   - Duda,R.O., & Hart,P.E. (1973) Pattern Classification and Scene Analysis.
     (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.
   - Dasarathy, B.V. (1980) "Nosing Around the Neighborhood: A New System
     Structure and Classification Rule for Recognition in Partially Exposed
     Environments".  IEEE Transactions on Pattern Analysis and Machine
     Intelligence, Vol. PAMI-2, No. 1, 67-71.
   - Gates, G.W. (1972) "The Reduced Nearest Neighbor Rule".  IEEE Transactions
     on Information Theory, May 1972, 431-433.
   - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al"s AUTOCLASS II
     conceptual clustering system finds 3 classes in the data.
   - Many, many more ...

from sklearn.cross_validation import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
#划分trainSet和testSet
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target,test_size=0.25,random_state=33)

#标准化
ss = StandardScaler()
X_train = ss.fit_transform(X_train)
X_test = ss.transform(X_test)

#KNN分类器
knc = KNeighborsClassifier()
knc.fit(X_train,y_train)
y_predict = knc.predict(X_test)

#性能评估：准确率，精确率，召回率，F1
print('The accuracy of K-Nearest Neighbor Classifier is:',knc.score(X_test,y_test))
from sklearn.metrics import classification_report
print(classification_report(y_test,y_predict,target_names = iris.target_names))

The accuracy of K-Nearest Neighbor Classifier is: 0.8947368421052632
             precision    recall  f1-score   support

     setosa       1.00      1.00      1.00         8
 versicolor       0.73      1.00      0.85        11
  virginica       1.00      0.79      0.88        19

avg / total       0.92      0.89      0.90        38