Table of Contents

• Overview
  • Logistic function
  • Model
  • Loss function
  • Cost function
  • Gradient descent

Overview

Logistic regression is a statistical model that in its basic form uses a logistic function to model a binary dependent variable.

• Logistic function

  Logistic function is a common example of a sigmoid function.

  

• Model

  

  

  

  where is the threshold.

• Loss function

  Cross-entropy is used as the loss function in Logistic Regression.

  

• Cost function

  Just sum up you can get the cost function.

  

• Gradient descent

  You can use gradient descent to find the optimum parameters

  1. Initialize and

  2. Compute and using chain rule

     

     

     

     

     So we can obtain

     

     Similarly,

  

  3. Update and

     

     

     where is the learning rate.

  4. Repeat step 2&3 until the improvement drops below a threshold or it reaches the maximum number of iterations.