Discussing the article: "Neural Networks in Practice: Activation Functions"

[MetaQuotes]

Check out the new article: Neural Networks in Practice: Activation Functions.

This is, without a doubt, the article I've enjoyed writing the most so far. In it, I showed that we don't really need anything special to achieve a certain goal. We can achieve this in various ways, provided we have the necessary knowledge and are willing to learn and dedicate ourselves to the task. I would like to express my heartfelt thanks to everyone who helped me choose these functions. The material presented here is intended solely for educational purposes. Under no circumstances should this be viewed as intended for anything other than teaching and studying the concepts presented.

In the previous article, “Neural Network in Practice: The Rectifier Function Graph”, we developed an application that allowed us to visualize an activation function graph along with its derivative. However, what we saw there is just the foundation for something a little more complex, and that is precisely what will be the main topic of this article.

I know that many people may consider it unnecessary to understand an activation function graph and its derivative, and may feel that we could dive straight into the topic, leaving such questions aside. However, it's much easier to understand why the sigmoid function can't be used in very deep networks if you understand its graph.

So here we'll focus specifically on that. We'll select a few activation functions and see how we can modify the code from the previous article to graph both the activation function and its derivative.

Author: Daniel Jose