Multi-class Classification and Neural Networks
Coursera machine learning exercise 3.pdf | |
File Size: | 293 kb |
File Type: |
Visualization of the dataset (handwritten digits 0~9)
Vectorizing regularized logistic regression
The cost function for regularized logistic regression and the partial derivative of regularized logistic regression cost can be calculated by the attached matlab file "lrcostfunction.m". It does not contain any loops.
The cost function for regularized logistic regression and the partial derivative of regularized logistic regression cost can be calculated by the attached matlab file "lrcostfunction.m". It does not contain any loops.
lrcostfunction.m | |
File Size: | 1 kb |
File Type: | m |
One-vs-all Classification
In this part, the one-vs-all classification by training multiple regularized logistic regression classifiers is implemented, one for each of
the K classes in the dataset. In the handwritten digits dataset, K = 10 (number 0 to 9). The attached code "onevsall.m" trains one classifier for each class (each number). The classifier parameters are returned in a matrix where each row of the matrix corresponds to the learned logistic regression parameters for one class.
In this part, the one-vs-all classification by training multiple regularized logistic regression classifiers is implemented, one for each of
the K classes in the dataset. In the handwritten digits dataset, K = 10 (number 0 to 9). The attached code "onevsall.m" trains one classifier for each class (each number). The classifier parameters are returned in a matrix where each row of the matrix corresponds to the learned logistic regression parameters for one class.
onevsall.m | |
File Size: | 2 kb |
File Type: | m |
One-vs-all Prediction
With the trained one-vs-all classifier, prediction of the digit contained in a given image can be made. For each input, the probability that it belongs to each class using the trained logistic regression classifiers will be calculated. The one-vs-all prediction function "predictonevsall.m" will pick the class for which the corresponding logistic regression classifier outputs the highest probability and return the class label (1, 2,..., or K) as the prediction for the input example.
The training set accuracy in this problem is about 94.9% (i.e., it classifies 94.9% of the examples in the training set correctly).
With the trained one-vs-all classifier, prediction of the digit contained in a given image can be made. For each input, the probability that it belongs to each class using the trained logistic regression classifiers will be calculated. The one-vs-all prediction function "predictonevsall.m" will pick the class for which the corresponding logistic regression classifier outputs the highest probability and return the class label (1, 2,..., or K) as the prediction for the input example.
The training set accuracy in this problem is about 94.9% (i.e., it classifies 94.9% of the examples in the training set correctly).
predictonevsall.m | |
File Size: | 1 kb |
File Type: | m |
Neural Network Feedforward Propagation and Prediction
The feedforward propagation for the neural network is implemented. The code in "predict.m" returns the neural network's prediction. The function inputs Theta1 and Theta2 are trained sets of parameters for the input of the hidden layer and output layer, respectively.
The feedforward computation computes h_theta(x^(i)) for every example i and returns the associated predictions. Similar to the one-vs-all classification strategy, the prediction from the neural network will be the label that has the largest output (h_theta(x))_k.
The neural network model is shown in the following figure.
The feedforward propagation for the neural network is implemented. The code in "predict.m" returns the neural network's prediction. The function inputs Theta1 and Theta2 are trained sets of parameters for the input of the hidden layer and output layer, respectively.
The feedforward computation computes h_theta(x^(i)) for every example i and returns the associated predictions. Similar to the one-vs-all classification strategy, the prediction from the neural network will be the label that has the largest output (h_theta(x))_k.
The neural network model is shown in the following figure.
predict.m | |
File Size: | 1 kb |
File Type: | m |