Pytorch for Neural Networks Part 2: Initializing Weights and Biases

Rijul Rajesh Posted on May 31           Pytorch for Neural Networks Part 2: Initializing Weights and Biases # machinelearning # ai In the previous article , we got started with expressing a neural network in the form of Python code. In this article, we will continue building on that. This is the neural network that we will recreate using code. You can see the weights and biases shown in the diagram above. Let us now add them to our code. We start with the basic neural network class: class MyBasicNN ( nn . Module ): def __init__ ( self ): super (). __init__ () Enter fullscreen mode Exit fullscreen mode Our first weight has the value 1.70 . We can represent it like this: class MyBasicNN ( nn . Module ): def __init__ ( self ): super (). __init__ () self . w00 = nn . Parameter ( torch . tensor ( 1.7 ), requires_grad = False ) Enter fullscreen mode Exit fullscreen mode Here, we initialize a new variable called w00 and make it a neural network parameter . When we define a weight as a parameter, PyTorch treats it as part of the neural network and gives us the option to optimize it during training. Since this value is stored as a tensor , the neural network can take advantage of features such as: automatic differentiation accelerated mathematical operations If you are unfamiliar with tensors, check out my earlier article on tensors . Since we do not need to optimize this weight, we set: requires_grad = False Enter fullscreen mode Exit fullscreen mode requires_grad is short for requires gradient . By setting it to False , we tell PyTorch that this parameter should not be updated during optimization. In a similar way, we can define the rest of the weights and biases. class MyBasicNN ( nn . Module ): def __init__ ( self ): super (). __init__ () self . w00 = nn . Parameter ( torch . tensor ( 1.7 ), requires_grad = False ) self . b00 = nn . Parameter ( torch . tensor ( - 0.85 ), requires_grad = False ) self . w01 = nn . Parameter ( torch . tensor ( - 40.