Machine Learning Quickies

Whether we build the machine learning model on sample of any population or on whole population cleansed data ? We build the ML model (especially supervised learning) on sample of any population to predict on the rest of the population because the sample is all we have. Just think about it, if we had the whole population data we won't need to make a model at all; We could just look up the population data for the desired value. What are some Machine learning projects primarily focused in the banking sector?  Machine learning projects primarily focused in the banking sector could be: Transaction Fraud detection Predicting which customers will default their loans Recommending customised policies or programs to customers Predicting which customers will buy a scheme shown to them Is math necessary for learning Machine Learning/ Data science/ Deep Learning ? Not necessarily. These days, most things in machine learning are done using prebuilt modules (sklearn, keras, pytorch, etc) wh...

 BI or business intelligence is a term used to denote all the tools, technologies and methodologies that aim at deriving actionable insights out of business data with the hope of making better business decisions. In other words, the whole point is to make intelligent decisions through the use of available business data. Power BI vs. Tableau This is a burning question that is the first thing you will have to deal with if you are trying to step into the field of business intelligence. The answer is rather simple. Pick any one (Trust me, going through endless videos and posts comparing these two megastars is a useless attempt! I have tried!) Power BI and Tableau are both BI (Business Intelligence) tools which are quite popular. Instead of getting into this futile debate of which is better, just pick any one and learn it well. Both of them offer exceptional functionality and are as business oriented as it gets. Plus, it is easy to switch if you know any one (the concept remains the sam...

 Computer Vision is a field of study that deals in providing computers with vision, i.e., the ability to understand and interpret visuals(images and videos). For example, you as a human being can visually see and differentiate between a dog and a cat. With computer vision, you try to give this ability to machines. Computer vision field includes image classification, object detection, segmentation, etc.   How will computer vision change the way our machines operate? Computer vision will change the way our machines operate, primarily,  by increasing the existing automation capabilities . For example, there are already instances of AI-driven medical applications coming up which not only display the X-ray of a patient but also analyze it to pose a recommended path for treatment (customised to each unique patient), thereby reducing the doctor's workload. Similarly, in case of self-driving autonomous cars, it also shows automation beyond human supervision. Hopefully, thing...

 M ulticollinearity simply means that there is high correlation amongst some of the predictor variables in a multiple regression model, meaning that one or more of these predictor variables can be accurately predicted (linearly) from some other predictor variable. How to deal with it? You can use decision tree based models (boosted or simple) as they are by nature immune to multicollinearity (as out of let's say 2 highly correlated features, it will use only one at any split). However, it is still good to remove any redundant features during the preprocessing phase. How to remove redundant features? One possible way could be to use the correlation value between the predictor variables and the target to flush out the less contributing features. You can easily find a function that will compute these correlation values for you for each feature and then you can set a threshold to accept the features for training your final model. Is it necessary to deal with it? Most people use tree b...

 Hyperparameter tuning is a process to find the optimal hyperparameters for an ML algorithm. The simplest manual strategy is as follows: Divide the training set into 2 parts: train_set and val_set. Set initial hyperparameters. Train your model with the train_set and evaluate on the val_set with some metric ( accuracy, AUC, etc). Change your hyperparameter values and repeat step 3. Do this for a couple of times Choose those hyperparameter values which gave the best metric value and retrain the model on the entire training set with these chosen values. Hyperparameter tuning is a very important step. You can have the best algorithms and the best data, but to get to the point of best output, you inevitable need to tune your hyperparameters. These days there are a lot of automated libraries for hyperparameter tuning as well where you don't have to lay a finger and the library will get you the optimal values. However for starters, it is best to try this manually. Later, you can switch to...

 In this article, we will explore the basic difference between Batch Gradient Descent, Stochastic Gradient Descent (SGD) and Mini-Batch Gradient Descent.  But first let us understand what an epoch means. Meaning of the word "epoch" In Deep Learning, an epoch is one iteration over the entire sample space (training set). In other words, everytime you go over each sample of the training set, it is considered one epoch. Basic Deep Learning training process Initialize model parameters: weights and biases with some random value. Go over objects of training set. Adjust your model parameters : weights and biases w.r.t some cost/loss fuction. Repeat from step 2 till desired threshold is reached. Batch Gradient Descent or simple Gradient Descent In Batch Gradient Descent, in each epoch we go throught entire training set and then adjust parameters. In every epoch, parameters are adjusted once which makes it unsuitable for large datasets. Stochastic Gradient Descent (SGD) In Stochast...