Logistic Regression

• Identical part of Linear Regression in Logistic Regression

  = model()
  =
  

• Logistic Regression(sigmoid function)

  =
  =

• Logistic Regression and Log-Odds

  if success is p=0.8, then failure is q = 1-p = 02
  Odds are determined from probabilities and range between 0 and infinity, which are defined as the ratio of the probability of success and the probability of failure,

  therefore,

  odds(success) = p / (1-p) = p / q = 0.8 / 0.2 = 4
  that is, odds of success are 4 to 1

  odds(failure) = q / p = 0.2 / 0.8 = 0.25
  which looks a little strange but that is really saying the odds of failure are 1 to 4

  Odds of ratio = odds(success) / odss(failure) = 16.0,

  therefore, odds of success is 16.0 times as the odds of failure

• About logits
  The logarithm of the odds is calculated, specifically log base-e or the natural logarithm. This quantity is referred to as the log-odds and may be referred to as the logit(logistic unit), a unit of measure

  log-odds = logit = log(odss) = log(p/(1-p)) = log(p/q)

  The range is negative infinity to positive infinity. In regression, it is easiest to model unbounded outcomes. Logistic regression is in reality an ordinary regression using the logit as the response variable. The logit transformation allows for a linear relationship between the response variable and the coefficients

  logit = log(p/q) = a + bX

  This means that the coefficients in a simple logistic regression are in terms of the log odds, that is, the coefficient 1.694596 implies that a one-unit change in gender results in a 1.694596 unit change in the log of the odds. The equation can be expressed in odds by getting rid of the log. This is done by taking e to the power for both sides of the equation.

  or
  

• The convertion from log-odds to odds

  The log-odds of success can be converted back into an odds of success by calculating the exponential of the log-odds.
  odds = exp(log-odds)

• The convertion from logit function to logistic function
  =
  =
  =
  =
  = =
  Divide both Numberator and Denominator by
  =

  sigmoid function, i.e. logistic function

• One can derive the sigmoid function in the following way:
  

Review of the logarithm

image

In general, given base where , the logarithm base b is defined as:

= if and only if =

The domain consists of all positive real numbers (0,∞) and the range consists of all real numbers (−∞,∞).

Reference

https://hausetutorials.netlify.app/posts/2019-12-01-neural-networks-deriving-the-sigmoid-derivative/#:~:text=The%20derivative%20of%20the%20sigmoid%20function%20%CF%83(x)%20is%20the,1%E2%88%92%CF%83(x).