Missing Values in Data Science Interview

-

If an interviewer shows you a sample dataset and asks you to tell what comes to your mind, he likely wants to ask you about MISSING VALUES in the dataset. So, check if the dataset has any missing values and answer him accordingly. The next thing the interviewer will ask you will likely be about strategies for dealing with those missing values. So, be prepared.

4 Strategies to Deal with Missing Values in a Dataset:

  1. Drop all columns with missing values (wastes a lot of valuable data, so NOT recommended)
  2. Drop all rows with missing values (if there are only a few no of rows with missing values, then you can do this)
  3. Imputation: You basically fill the missing value with some default value (like -1) or some calculated value (like mean). This is the most used strategy.
  4. Imputation with tracking: Basically, you use imputation on a column, and then you create a new column to keep track of the rows where you have applied imputation. The new column will be either TRUE or FALSE based on whether you have imputed the row. You can further extend this by doing this individually for every column. (This is not used as much because it is generally not feasible as it creates a lot of new columns).

Ending remark: You will most likely be using the third strategy  (of simple imputation) all the time.

Imputation of Missing Values: Numerical vs. Categorical feature

Imputation for numerical feature

You basically impute (fill) the missing value with some default constant value (like -1) or some calculated value like mean, median, or mode.

If you are filling it with a default value, make sure that the replacement value is outside the range of the possible values for that column. For example, you could use a value of -1 for the column PRICE because the price cannot be negative. So, -1 will denote to the algorithm that this is a missing value.

Imputation for categorical feature

In the case of a categorical feature, you impute (fill) the missing value with the most frequently occurring value (mode) or some default value like "_MISSING" (generally!)

Popular posts from this blog

ML technical definition : Learning in AI defined

-
Image
Now that you have dived deep into the waters of machine learning, you might be interested in knowing how to define it. But before you can define machine learning, you need to learn what "learning" is. So, what is "learning" in Machine Learning? Learning in ML and AI  How do human beings learn? They improve with their experiences. Right!. This is exactly what learning is. But how would you define it, then? Learning can be defined as the ability to improve with experience. In terms of AI and ML, it means making computer programs and systems which automatically improve with experience like human beings do. Experience can be thought of as "data". In ML, we try to build models which learn from data to perform some task like decision-making, prediction, etc. But this seems like an informal definition of ML. Right! How would you define it formally? Definition of ML The present standard definition of machine learning is given by Tom Mitchell, as follows:  A comput...
Read more

Batch Gradient Descent vs Stochastic Gradient Descent (SGD) vs Mini-Batch Gradient Descent

-
Image
 In this article, we will explore the basic difference between Batch Gradient Descent, Stochastic Gradient Descent (SGD) and Mini-Batch Gradient Descent.  But first let us understand what an epoch means. Meaning of the word "epoch" In Deep Learning, an epoch is one iteration over the entire sample space (training set). In other words, everytime you go over each sample of the training set, it is considered one epoch. Basic Deep Learning training process Initialize model parameters: weights and biases with some random value. Go over objects of training set. Adjust your model parameters : weights and biases w.r.t some cost/loss fuction. Repeat from step 2 till desired threshold is reached. Batch Gradient Descent or simple Gradient Descent In Batch Gradient Descent, in each epoch we go throught entire training set and then adjust parameters. In every epoch, parameters are adjusted once which makes it unsuitable for large datasets. Stochastic Gradient Descent (SGD) In Stochast...
Read more

What exactly is Artificial Intelligence in simple words ? Explain AI in simple terms.

-
Image
  AI explained in simple language: Remember, when you were a child and you saw some animal with paws. You did not know what it was and the adults told you it was a dog. Then, you saw some other animal and you called it a "dog" but the adults corrected you as it was a "cat". From then on, you started to differentiate between dogs and cats. This is what we call human intelligence i.e., the ability to learn over time from your environment.  Now, what if I told you a computer could also do what you did. Just like adults taught you, you could  teach a computer to differentiate between a dog and a cat. Would you call the computer "intelligent"? This is precisely what Artificial Intelligence (in short AI) is. It is giving computers human-like intelligence so that they could do the tasks traditionally reserved for humans. But why do we need AI? Because they are computers and can perform much more than a human could without getting tired. In fact, you are already u...
Read more