【COMP337 LEC 5-6】

LEC 5 Perceptron ：

Binary Classification Algorithm

8

感应器是 单个神经元的模型

突触连接的强度取决于接受外部刺激的反应

X input

W weights

a = x1*w1+x2*w2....... > / < threshold

Bias

MaxIter is a hyperparameter  超参数 which has to be chosen experimentally

• If we make many passes over the training data, then the algorithm is likely to overfit.

• If we make few passes might lead to underfitting

意思就是这个值，需要不断地手工修改枚举法，保证即不overfit，也不underfitting

LEC 6 Perceptron ：

Geometric Interpretation 几何解释

Hyperplane 超平面

Linear separability

1. When a dataset is linearly separable, there can exist more than one hyperplanes that separates the dataset into positive/negative groups.

当线性分离的时候，存在多个hyperplanes

not unique

2. 所以当不是线性分离的时候，就不会存在hyperplane

Take the average over all weight vectors during the training  (Averaged Perceptron algorithm)

Multiclass Classification

Multiclass classifiers

• k-NN

• Naive Bayes

Binary classifiers

• Perceptron

• Logistic regression

How to turn a binary classifier to a multiclass classifier?

Given binary classification algorithm A we want to design a meta-algorithm that use A to make k-class predictions.

两种方法：

One-vs.-one approach

There might be ambiguity if some classes got the same number of votes (if the binary classifier A can produce a confidence score, it can be used to break ties)

One-vs.-rest approach

Drawbacks

1. the scale of the confidence scores may differ between the binary classifiers

2. the binary classifiers are trained on unbalanced datasets: usually the set ofnegative objects will be much larger than the set of positive objects