Linear Regression From Scratch Using MXNet

Photo by Pietro Jeng on Unsplash

Linear Regression is one of the most important supervised machine learning algorithm. I think this is the one of the first machine learning algorithm people learn when they start their machine learning journey.

Apache MXNet is an open-source deep learning software framework, used to train, and deploy deep neural networks. It is scalable, allowing for fast model training, and supports a flexible programming model and multiple programming languages (including C++, Python, Java, Julia, Matlab, JavaScript, Go, R, Scala, Perl,

In linear regression, the observations (red) are assumed to be the result of random deviations (green) from an underlying relationship (blue) between a dependent variable (y) and an independent variable (x).

We will construct an artificial datasets according to a linear model with additive noise. We will keep the data low-dimensional so we can visualize it easily. In the following code snippet, we generate a datasets containing 1000 examples, each consisting of 2 features sampled from a standard normal distribution. Thus our synthetic datasets will be a matrix of (1000 x 2)

At first install all the required library:

Now import all the required library:

Now create the datasets:

Now we can see the linear relation between the label and the 2nd column of the datasets:

Training models consists of making multiple passes over the datasets, grabbing one mini batch of examples at a time, and using them to update our model. Since this process is so fundamental to training machine learning algorithms, it is worth defining a utility function to shuffle the datasets and access it in mini batches.

Initializing Model Parameters & Defining the Model:

Initializing Model Parameters & Defining the Model:

Defining the Loss Function:Since updating our model requires taking the gradient of our loss function, we ought to define the loss function first. Here we will use the squared loss function

Defining the Optimization Algorithm:

Now that we have all of the parts in place, we are ready to implement the main training loop.In each epoch, we will iterate through the entire datasets (using the data_iter function) once passing through every example in the training datasets (assuming that the number of examples is divisible by the batch size). The number of epochs num_epochs and the learning rate lr are both hyper parameters, which we set here to 3 and 0.03, respectively. Unfortunately, setting hyper parameters is tricky and requires some adjustment by trial and error.

You can also get the whole code at google colab:

Connect with me via github or blogpage

Thanks for reading & suggestion is welcome !!!