Learning systems : basic algorithm in Machine Learning design

-

 Aspects of developing a learning system

In the previous blog posts, I have already talked about the meaning of the term "learning" in machine learning. Go check out that blog (here!) if you have not read it already as it is a prerequisite for this one. This post will talk about how that "learning" is learned by machine learning systems and I will try to explain the basic algorithm that is followed in this process and the various aspects involved in the learning process.

Algorithm to design a learning system in ML workflow

The basic algorithm followed by an explanation is as follows:
  1. Choosing the training experience.
  2. Choosing the target function.
  3. Choosing the target function's representation.
  4. Choosing the learning or function approximation algorithm to infer the target function.

Explanation

Choosing the training experience

The training experience can be thought of as the training data. Basically, you need to have training data which the learning system will learn from (or train from). It is important to have properly processed data for proper future results. For example, in case of a regression problem to predict house price from features like house location, area, bedrooms, etc., your dataset regarding this will be your training experience (training data).

Choosing the target function

The target function is one we have to learn. Choosing the target function means describing it. In case of the above regression example, the target function will be a function which, given an input of features like house location, area, bedrooms, etc., will return the house price. Most of the times, it is not possible to learn the ideal target function, so we try to approximate it and this is the concept of function approximation.

Choosing the target function's representation

Choosing how to represent the target function means choosing the class of functions that you would want to use to approximate the target function : linear based, tree based, neural network, or kNN. Generally, you would want to start with a simpler class like linear-based because the functions here are easier to train and represent. However, a lot of times, you would have to go for more complex classes because the simpler ones like linear ones are simply not enough.

Choosing the learning algorithm for function approximation

Now that you have chosen how you would want to represent the target function (let us say you want to use a simple linear model), you would need a learning algorithm (also called function approximation algorithm) that would learn this target function, which basically means finding appropriate parameters (weights or coefficients in case of linear model) for the model. You might have heard of gradient descent which is the most popular learning algorithm. 

This is how a learning system comes about which can be used for further reasoning and prediction.

Through this process, we have covered the various aspects of developing a learning system. In the next post, I might talk about the different types of learning.

Popular posts from this blog

ML technical definition : Learning in AI defined

-
Image
Now that you have dived deep into the waters of machine learning, you might be interested in knowing how to define it. But before you can define machine learning, you need to learn what "learning" is. So, what is "learning" in Machine Learning? Learning in ML and AI  How do human beings learn? They improve with their experiences. Right!. This is exactly what learning is. But how would you define it, then? Learning can be defined as the ability to improve with experience. In terms of AI and ML, it means making computer programs and systems which automatically improve with experience like human beings do. Experience can be thought of as "data". In ML, we try to build models which learn from data to perform some task like decision-making, prediction, etc. But this seems like an informal definition of ML. Right! How would you define it formally? Definition of ML The present standard definition of machine learning is given by Tom Mitchell, as follows:  A comput...
Read more

Batch Gradient Descent vs Stochastic Gradient Descent (SGD) vs Mini-Batch Gradient Descent

-
Image
 In this article, we will explore the basic difference between Batch Gradient Descent, Stochastic Gradient Descent (SGD) and Mini-Batch Gradient Descent.  But first let us understand what an epoch means. Meaning of the word "epoch" In Deep Learning, an epoch is one iteration over the entire sample space (training set). In other words, everytime you go over each sample of the training set, it is considered one epoch. Basic Deep Learning training process Initialize model parameters: weights and biases with some random value. Go over objects of training set. Adjust your model parameters : weights and biases w.r.t some cost/loss fuction. Repeat from step 2 till desired threshold is reached. Batch Gradient Descent or simple Gradient Descent In Batch Gradient Descent, in each epoch we go throught entire training set and then adjust parameters. In every epoch, parameters are adjusted once which makes it unsuitable for large datasets. Stochastic Gradient Descent (SGD) In Stochast...
Read more

What exactly is Artificial Intelligence in simple words ? Explain AI in simple terms.

-
Image
  AI explained in simple language: Remember, when you were a child and you saw some animal with paws. You did not know what it was and the adults told you it was a dog. Then, you saw some other animal and you called it a "dog" but the adults corrected you as it was a "cat". From then on, you started to differentiate between dogs and cats. This is what we call human intelligence i.e., the ability to learn over time from your environment.  Now, what if I told you a computer could also do what you did. Just like adults taught you, you could  teach a computer to differentiate between a dog and a cat. Would you call the computer "intelligent"? This is precisely what Artificial Intelligence (in short AI) is. It is giving computers human-like intelligence so that they could do the tasks traditionally reserved for humans. But why do we need AI? Because they are computers and can perform much more than a human could without getting tired. In fact, you are already u...
Read more