Interview questions for logistic regression

1. What is logistic regression?

It’s a classification algorithm, that is used where the response variable is categorical. The idea of Logistic Regression is to find arelationship between features and probability of particular outcome.
Ex. When we have to predict if a student passes or fails in an exam when the number of hours spent studying is given as a feature, the response variable has two values, pass and fail.
This type of a problem is referred to as Binomial Logistic Regression, where the response variable has two values 0 and 1 or pass and fail or true and false. Multinomial Logistic Regression deals with situations where the response variable can have three or more possible values.

2. What are the assumptions made in logistic regression?

The assumptions made in logistic regression are as follows:
1. The logistic regression assumes that there is minimal or no multicollinearity among the independent variables.
2. There should be a linear relationship between the logit of the outcome and each predictor variables. The logit function is given as logit(p) = log(p/(1-p)), where p is the probability of the outcome.
3. The logistic regression usually requires a large sample size to predict properly.
4. The Logistic regression which has two classes assumes that the dependent variable is binary and ordered logistic regression requires the dependent variable to be ordered, for example Too Little, About Right, Too Much.
5. The Logistic regression assumes the observations to be independent of each other.

3. How can the probability of a logistic regression model be expressed as conditional probability?

P(Discrete value of Target variable | X1, X2, X3….Xk). It is the probability of the target variable to take up a discrete value (either 0 or 1 in case of binary classification problems) when the values of independent variables are given. For example, the probability an employee will attrite (target variable) given his attributes such as his age, salary, KRA’s, etc.

4. What are odds?

It is the ratio of the probability of an event occurring to the probability of the event not occurring. For example, let’s assume that the probability of winning a lottery is 0.01. Then, the probability of not winning is 1- 0.01 = 0.99.
The odds of winning the lottery = (Probability of winning)/(probability of not winning)
The odds of winning the lottery = 0.01/0.99
The odds of winning the lottery is 1 to 99, and the odds of not winning the lottery is 99 to 1.

5. What are the outputs of the logistic model and the logistic function?

The logistic model outputs the logits, i.e. log odds; and the logistic function outputs the probabilities.
Logistic model = α+1X1+2X2+….+kXk. The output of the same will be logits.
Logistic function = f(z) = 1/(1+e-(α+1X1+2X2+….+kXk)). The output, in this case, will be the probabilities.

6. What is the likelihood function?

The likelihood function is the joint probability of observing the data. For example, let’s assume that a coin is tossed 100 times and we want to know the probability of getting 60 heads from the tosses. This example follows the binomial distribution formula.
p = Probability of heads from a single coin toss
n = 100 (the number of coin tosses)
x = 60 (the number of heads – success)
n-x = 30 (the number of tails)
Pr(X=60 |n = 100, p)
The likelihood function is the probability that the number of heads received is 60 in a trail of 100 coin tosses, where the probability of heads received in each coin toss is p. Here the coin toss result follows a binomial distribution.
This can be reframed as follows:
Pr(X=60|n=100,p) = c x p60x(1-p)100-60
c = constant
p = unknown parameter
The likelihood function gives the probability of observing the results using unknown parameters.

7. What can not be used in case of logistic regression?

• Which of the following evaluation metrics can not be applied in case of logistic regression output to compare with target?
• Since, Logistic Regression is a classification algorithm so it’s output can not be real time value so mean squared error can not use for evaluating it

8. Is it true that logistic regression is supervised machine learning?

Yes, the answer to this question would be TRUE because, indeed, logistic regression is a supervised machine learning algorithm. The simple reason why lies in the way this algorithm works. To get output from logistic regression, you will have to feed it with data first

9. How many people took the logistic regression skill test?

We saw the same spirit on the test we designed to assess people on Logistic Regression. More than 800 people took this test. This skill test is specially designed for you to test your knowledge on logistic regression and its nuances. If you are one of those who missed out on this skill test, here are the questionsand solutions.

10. What is the Maximum Likelihood Estimator (MLE)?

The MLE chooses those sets of unknown parameters (estimator) that maximise the likelihood function. The method to find the MLE is to use calculus and setting the derivative of the logistic function with respect to an unknown parameter to zero, and solving it will give the MLE. For a binomial model, this will be easy, but for a logistic model, the calculations are complex. Computer programs are used for deriving MLE for logistic models.

OR

MLE is a statistical approach to estimating the parameters of a mathematical model. MLE and ordinary square estimation give the same results for linear regression if the dependent variable is assumed to be normally distributed. MLE does not assume anything about independent variables.

Why is logistic regression called regression and not classification?
The main difference between regression and classification is that the output variable in regression is numerical (or continuous) while that for classification is categorical (or discrete).Logistic regression is basically a supervised classification algorithm. However, the model builds a regression model just like linear regression to predict the probability that a given data entry belongs to the category numbered as “1”.
For example, with binary classification, let ‘x’ be some feature and ‘y’ be the output which can be either 0 or 1.