Mermaid

After a series of previous posts we hav learned the key steps of making a Machine Learning: Gradient Descent Back Propagation Adding Multiple Nodes Changing Activation Function With all of this we are in a position to construct a fully functional neural network. We combine knowledge from all of these previous posts and try to construct a fully functional multi-layer multi-node Neural Network. Setup For a fully functional network, similar to last post , we can make multiple layer multiple nodes. ...

As we saw in previous post addition of any number of layers and any number of nodes will only be able to model a linear functions. If we want our network to also be able to learn non linear functions, then we should introduce some non linearlity to the network. One way of adding non linearity is to use so called activation function. The idea is that we pass the output of each layer through a (non linear) function. Our examples so far can be considered to have been using a linear activation function. In particular, they all used identity activation function ...

In the previous two posts we have already seen how to construct a single neuron neural network with or without hidden layer. If we want to input multidimensional input and get multidimensional output from the netwro then we have to add more nodes in each layers so that each node in the layer can process independent multidimensional input. For this purpose in this post we will construct a network with a single output layer but with multiple nodes in input layer. This is useful for example if we want to train our network to learn the scalar (one number) function of $N$ dimensional vector. ...

Setup Similar to last post , where we made a basic neural network with just one neuron and no hidden layer, we will build a two neuron and a single hidden layer. %% {init {'theme':'dark'}} %% flowchart LR; a((a_0))-->|W_1| b((b_1))-->|W_2| c((b_2)) \begin{align*} a^1 = W^1 a^0 + b^1\\ a^2 = W^2 a^1 + b^2 \end{align*}So the final output of the network as a function of $x$ input becomes \begin{align*} y = W^2 \left( W^1 \cdot x + b^1 \right) + b^2 \end{align*}This is linear in $x$. So we should be able to model any linear function with this network. Like before let us try to learn a linear model through this primite network. In the last post we trained the network to learn the linear model ...