Now, in this lecture, we're going to talk about activation functions the same, that neural network
needed activation functions.
Also, the convolutional neural networks need the same activation functions.
Now, learning in convolutional neural network also is done by back propagation.
So in those particular obligation, we could be getting some negative values, which are not a very
good idea usually in image processing.
So in order to get rid of those, we could be use some activation function.
Now, back propagation is going to modify the way this is what we have learned in the neural network,
in our convolutional neural network.
However, what we are going to modify in the back of propagation is the filter values themselves and
it will be calculated that the output of the network and this error will propagate back and it will
modify the filters accordingly until we get some stable filter values that can predict any image we
have.
Now, RELU is the function that we are going to be using here and really it's a nonlinear activation
function.
Now, non-linearity proved to be very effective as an activation function because it puts your neural
network into a lot of scenarios and it eliminates linearity.
Linearity is a very bad thing when you are trying to predict a multiple using in a model.
OK, so this is why the activation function should be non-linear.
Now the non-linearity is going to throw away some information in the training phase, which proves to
be very important in order to get some accurate, powerful model, because if you throw away some information,
you are not actually you're model to learn constants stuff.
What you are forcing it to learn a variety of stuff.
This is why non-linearity is very important because.
Well, now, since you will be passing maybe the same image multiple times because, well, images,
you pass them multiple times to this data.
And every time if you throw some information in from this same image, you will get some different results
which could help you to get and predict using a better model.
It will improve your model a lot.
Right.
Because as I said, your best images to your CNN.
Right.
And let's say you passed a thousand images, but sometimes you pass this image twice or three times
so that you can enforce learning and this image that is being passed multiple times if you're getting
more information from it.
Well, this will help you create a more powerful model.
Now, the real function looks like this.
If there is a value that is less than zero or zero, it will be zero.
OK, so let's say that your convolutional network has some negative value as well.
All the negative values will be zero.
Now all the positive values will pass as they are.
OK, so if we take an example, let's say that the convolutional layer got me a result that looks like
this.
So I had an image, I had a filter.
I got all of them together and I got this result.
Now I pass this activation function, which is the real as you can see, we are changing the data we
have acquired by turning all the negative values to zero.
Now, doing that will also change the feature that you have extracted.
Right, because this is a feature and you are changing the feature and how it looks like.
Now, imagine if you press the same image multiple times and every time you pass it, you have different
rates.
So maybe you will have negative values in some other places which you will have zero at.
OK, so every time you press the same image and you are using nonlinearity, you are actually enforcing
your CNN to be more powerful.
OK, because what gives you a powerful model is having a lot of data and you can also have a lot of
data about enforcing non-linearity to your model, using activation functions like the realm.
Now there is some other activation function that are linear, which are not very useful in image recognition
or in our case here.
So Relu is the most used one.
And CNN.