Logistic Regression
Coursera machine learning exercise 2 | |
File Size: | 233 kb |
File Type: |
Sigmoid Function
A sigmoid function is a mathematical function having an "S" shape (sigmoid curve). Often, sigmoid function refers to the special case of the logistic function and defined by the formula S(t)=1/[1+e^(-t)].
A sigmoid function is a mathematical function having an "S" shape (sigmoid curve). Often, sigmoid function refers to the special case of the logistic function and defined by the formula S(t)=1/[1+e^(-t)].
sigmoid.m | |
File Size: | 0 kb |
File Type: | m |
Cost Function and Gradient
The code in costfunction.m is used to calculate the cost function and gradient descent for logistic regression.
The code in costfunction.m is used to calculate the cost function and gradient descent for logistic regression.
costfunction.m | |
File Size: | 1 kb |
File Type: | m |
Learning Parameters Using fminunc
Instead of taking gradient descent steps, a MATLAB built-in function called fminunc is used. MATLAB's fminunc is an optimization solver that finds the minimum of an unconstrained function. For logistic regression, the cost function J(theta) with parameters theta needs to be optimized.
The main program code is all in ex2.m.
Instead of taking gradient descent steps, a MATLAB built-in function called fminunc is used. MATLAB's fminunc is an optimization solver that finds the minimum of an unconstrained function. For logistic regression, the cost function J(theta) with parameters theta needs to be optimized.
The main program code is all in ex2.m.
ex2.m | |
File Size: | 3 kb |
File Type: | m |
Figure 1. Training data with decision boundary
Regularized logistic regression
In this problem, the data set cannot be separated into positive and negative examples by a straight-line through the plot. Therefore, a straight-
forward application of logistic regression will not perform well on this data set since logistic regression will only be able to find a linear decision boundary.
In this problem, the data set cannot be separated into positive and negative examples by a straight-line through the plot. Therefore, a straight-
forward application of logistic regression will not perform well on this data set since logistic regression will only be able to find a linear decision boundary.
Feature Mapping
One way to fit the data better is to create more features from each data point. In the attached function mapFeature.m, the features are mapped into
all polynomial terms of x1 and x2 up to the sixth power.
One way to fit the data better is to create more features from each data point. In the attached function mapFeature.m, the features are mapped into
all polynomial terms of x1 and x2 up to the sixth power.
mapfeature.m | |
File Size: | 0 kb |
File Type: | m |
Regularized Cost Function and Gradient
The code in costFunctionReg.m returns the cost and gradient. The regularized cost function in regularized logistic regression is a little bit different from the previous cost function in logistic regression.
The code in costFunctionReg.m returns the cost and gradient. The regularized cost function in regularized logistic regression is a little bit different from the previous cost function in logistic regression.
costfunctionreg.m | |
File Size: | 1 kb |
File Type: | m |
Influence of Regularization Parameters (Lambda) on Decision Boundary
Figure 2. Underfitting; Lambda = 10; Train accuracy = 74.576%.
Figure 3. Lambda = 1; Train accuracy = 83.051%.
Figure 4. Overfitting; Lambda = 0; Train accuracy = 81.288%.
|
|