Discussing the article: "Neural Networks in Practice: A Question of Scale"

Check out the new article: Neural Networks in Practice: A Question of Scale.

There is a common misconception among the general public that a neural network or artificial intelligence can, in a sense, comprehend the world in which we live. In some cases, this might even be true. However, if you want to achieve faster convergence of a neural network—or even a single neuron—you need to understand that the scale of the values used often affects the convergence rate. In addition, in this article I'll show you something strange that happens in neural networks. It will blow your mind and make you think about a lot of things related to this topic.

In this article, we won't focus as much on the code, and we will no longer discuss or mention the least squares method, except in very specific cases. Although they will remain in the C_Neuron class code, they will be there only because we need to explain another issue that we haven't covered yet—one that specifically involves using a loss function based on the least squares method. From this point on, the focus will be on using the gradient. In any case, I don't want you to get too attached to any particular part of the code. I want you to try to understand why you should use one or the other and when to do so.

The reason is very simple: when it comes to perceptron networks, there is no single ideal way to do everything. There are things that work and things that don't. However, in many cases, an effective approach may be quite computationally intensive, so it will need to be adapted to a more suitable model. To do this, you need to know how to modify the code. As we add more and more inputs to the perceptron and increase the number of perceptrons, these concepts will become clearer, and you'll gain a better practical understanding of them.



Author: Daniel Jose