Discussion of article "Third Generation Neural Networks: Deep Networks" - page 4
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
vlad1949:
... the development of a loner (probably a talented programmer) has not yet been brought to practical realisation.
It is. The first version is published on code.google.com. The project is called libvmr.
So far only on Java. Later I will port it to other languages, including mql5.
vlad1949:
...
Here's a suggestion: Discuss Reshetov's topic in his blog or in a separate thread (if he organises it).
Opinions and considerations on the topic of the article - "Deep Neural Networks" - are welcome here.
It is also possible on deep learning.
For example, in order to improve classification, say for images, then train autoencoders using invariance. For example, we need to distinguish between images of two classes. Let these be charts of financial instruments with images of theses indicators. The first class is a chart before the quotes growth upwards. The second class is the chart before the quotes growth downwards. In this case, when training autoencoders, it is necessary to supply unchanged images to the inputs for the first class, and inverse, i.e. negative, images for the second class.
Also, already ready autoencoders trained using the back propagation algorithm can be assembled into a cascade and a neuron already trained using VMR can be placed at the output.
Reshetov:
If you are right your "deep" intellect that passed the Turing test should already enslave the internet and rule the world.
Frankly, I'm sick of you being so opinionated.
If you're right, your "deep" intellect, which has passed the Turing test, should already enslave the Internet and rule the world.
Andryukha, don't make too much noise. Or the orderlies will hear about Turing, wake up, and then the seizure of world domination will come to an end.
Honestly, I'm sick of you being so opinionated. . .
Take a sedative and you'll be fine. After intellectual enslavement, you'll get a tenth of the universe, if you don't rage too loudly.
Already after writing this article I came across a draft of a book by one of the leading researchers in the field of "Deep Learning" Yoshua Bengio et al. http://www. iro.umontreal.ca/~bengioy/dlbook/
Lots of maths, but at the same time clear definitions and basic concepts on the topic, from basic to advanced, are given.
Below are a few excerpts of the free translation.
In section 10. " Unsupervised and Transfer Learning" the authors believe that while teacher-guided learning has been the workhorse of recent industrial successes of deep learning, a key element of future advances may be unsupervised learning of representations (images?) without a teacher .). The idea is that some representations may be more useful (e.g. in classifying objects from images or phonemes from speech) than others. As argued there, this involves learning representations in order to select the best ones in a systematic way, ie. By optimising the feature that represents the raw data their representationsиrepresentations.
In this paper, we reviewed some of the basic building blocks for unsupervisedlearning of representations ( AutoEncoder and RBM) and very successful recipes for combining them to form deep representations (Stacked AutoEncoder and Stacked RBM) trained greedy layerwise unsupervised pre-training .
Subsequently, when training deep networks with a teacher (fine tuning), we train therepresentations with the goal of selecting onethat best fits the task of predicting the target represented by the inputs.
What if instead of one task, we have many tasks that could have commonrepresentations or part ofthem? (this is multi-task learning, )
"Transfer Learning and Domain Adaptation "
Knowledge transfer and domain adaptation refer to the situation where what has been
learnt insome conditions (i.e., the P1 distribution) is used to improve generalisation in other conditions (say, the P2 distribution )
In the case of knowledge transfer, we consider that thetasks are different, but many of
thefactors that explain the P1 distribution are relevant to the changes that must be captured to learn P2. This is generally understood in thecontext ofsupervised learning , in which the inputsы are the same sameыThe inputs are the same, but the goal may be differentof nature, for example, to learn about visualcategories that aredifferent in thefirst andsecond cases. If there is much more data in the first sample ( taken from P1), it may help to learn representations that are useful for rapid generalisation when processing examples from P2 .
For example, many visual categories have common low level concepts(edges andvisualshapes), effects of geometric changes, changes in lighting, etc.
In general, knowledge transfer, multi-tasklearning anddomain adaptation can be achieved throughlearning representations when there are features that can be useful for different settings or tasks, i.e. there are common underlying factors.
In the case of domain adaptation, we consider that the task to be solved is the same but the input sets are different. For example, if we predict sentiment (positive ornegative) related to textcomments on the Internet, the first sample might be a reference toconsumer comments about books, videos, andmusic, while the second set might refer totelevision orotherproducts.
While multitask learning is generally considered in the context of learning with a teacher, the more general notion of transfer learning applies to unsupervised learning and reinforcement learning.
An extreme form of transfer learning is one-shot learning or even zero-shot learning , whereone or no examples of a new task are given.
Briefly, two important conclusions.
For deep learning of neural networks with pre-training, it is meaningless to talk about the importance of individual predictors. What is important is the internal structure in the set. This will make it possible to extract deep representations (images)at the stage of training without a teacher, and at the stage of training with a teacher to determine how well they correspond to the task at hand.
A positive side effect of pre-training is that representations are extracted that are well suited for the neural network to solve another task, not the one we taught it at the second stage.
More specifically. If you read the article carefully, you will notice that we trained the neural network to classify inputs by presenting ZigZag signals. As a result, the Accuracy results are not impressive. But the results of of the total balance obtained using the predicted signals of the trained neural network are higher than those obtained using ZigZag signals! To evaluate the efficiency of prediction, I introduced a coefficient K. This is the average increase in small points per one bar of the chart. For example, for ZZ it is equal to 17, while for network predictions it is from 25 to 35 depending on the instrument (on TF M30).
This is knowledge transfer( transfer learning ) - when learning on one set of data we get solutions for several different tasks.
This is an extremely promising and developing topic.
To finish the work started in the article, in the next post I will give an example of optimising DN SAE parameters using an evolutionary (genetic) algorithm.
I did not consider the use of DN SRBM. It has not shown stable results in my practice.
Success
Already after writing this article I came across a draft of a book by one of the leading researchers in the field of "Deep Learning" Yoshua Bengio et al. http://www. iro.umontreal.ca/~bengioy/dlbook/
Lots of maths, but at the same time clear definitions and basic concepts on the topic, from basic to advanced, are given.
I'll be honest, when I read the article, the impression I got was a bit strange and cloudy, but I wrote it off as a "pile it all up" approach. Then I read the book on the link. Alas, almost everything that is presented in this work is long-chewed ideas, presented by someone as something new and surprising. And now the mathematician and research engineer in me say only one thing: this subject is designed to squeeze money out of state budgets, alas.
By the way, there is no mathematics (some of its own, not copied from a textbook) in the book. Some spells in the form of formulas only.... I do not see in general scientific prospects, at least until some really new idea appears in this direction. But this requires, first of all, theoretical research (which no one is doing), not practical magic. So far, the guys apparently don't really understand what they want.
So you want to ban the applied use of CS and sic the theoretical nerds on it?
Even without nerds it is clear that only the clearest signal will leak through autoencoders or MB, and everything else will be smeared and become invisible.
So you want to ban CS applications and turn the theoretical nerds against it?
No, of course not. I am only fighting attempts to replace science, no matter whether applied or theoretical, with shamanism to the best of my ability. And that's exactly what's going on in the field of NS, which has actually been stuck in both applied and theoretical for about a decade now.
And what is the problem for science? Who forbids theorising about machine learning?
Another compote is that all this theorising can be disproved in no time, because there are open repositories with samples from applied areas and there are ready-made research packages like R or Weka, with the help of which anyone can easily disprove any theoretical hypothesis.
It is the lack of problems that machine learning grew out of short trousers when there were only bare theories, but practical results in the field of generalisability of algorithms were not above the plinth. As soon as computational resources became generally available, application scientists pushed the theoretical nerds out of the field.
What's the problem for science? Who forbids theorising about machine learning?
Another compote is that all this theorising can be disproved in no time, because there are open repositories with samples from applied fields and there are ready-made research packages like R or Weka, with the help of which anyone can easily disprove any theoretical hypothesis.
It is the lack of problems that machine learning grew out of short trousers when there were only bare theories, but practical results in the field of generalisability of algorithms were not above the plinth. As soon as computational resources became generally available, application scientists pushed the theory nerds out of the field.
Where do we disagree at all? Applied scientists displacing nerds is naturally a good thing. In addition, with the availability of computational resources, many problems that previously had to be thought about theoretically have become solvable. But that doesn't mean that computing resources will solve all problems. The extensive way is not eternal, and its possibilities are already coming to an end. So sooner or later we will have to return to botanism.