Where is the line between fitting and actual patterns? - page 54
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
The vast majority of traders are losing, is this a pattern? Yes. Is there a pattern? No. Fringe, though. Can you make money on this? Yes. Learn to be a chef.
Ha-ha-ha-.... It's not funny.
I accidentally made another thematic experiment, the results of which I haven't even figured out how to evaluate)
I optimized my Expert Advisor and accidentally gave it a file with completely wrong data for analysis. I.e., during optimization the Expert Advisor was trading based on the data of November 2010 but it was analyzing the data from somewhere in 2009 (there is hardly any meaningful correlation) and opening trades based on them. As a result of learning, I obtained the result of 1000 old pips of profit with pure fitting (I analyzed one thing, traded the other). Understood the error, performed optimization with correct data - the result was 2000 pips. Now I think how to learn to separate the fitting component from training results with pure fitting and fitting + regularities.
And in general, the approach: analyze one, trade another, will not allow us to determine the "persistence" of the Expert Advisor?
.........
And in general, the very approach: analyse one thing, trade another, would not allow us to determine the "persistence" of an expert?
That's what everyone always does. Analyzing history is one thing, trading in real life is another.
Or, or rather, without or, I don't understand anything.
That's what everyone always does. Analyzing history is one thing, trading in real life is another.
Or, most likely, without or, I don't understand anything.
Suppose the Expert Advisor uses the time frame from 1 to 200 and opens in the direction of its direction from the 2nd to the 1st bar, MA[1]>MA[2] - buy, MA[1]<MA[2] - sell.
The numbers were wrong and the conditions are as follows MA[1000001]>MA[1000002] - buy, MA[1000001]<MA[1000002], i.e. the wand behavior on million bars does not describe the current situation when the EA opens the trade. The Expert Advisor's condition is "empty", a placebo. However, we will almost certainly find the waving period at which we make profit on the learning period. That will be pure naked chasing. This is what I was talking about. Although, to twist such a primitive Expert Advisor with a small number of degrees of freedom in such a way is probably meaningless. But in its place, imagine a NS whose inputs also have nothing to do with reality during the training period (for example, a binary sequence of bits from a digital copy of a video about the life of penguins)? Let's make the Expert Advisor make a profit at the learning phase from nothing, deliberately fit it and try to estimate how fit it is. Somehow... Then we give the correct data describing the situation at the moment of making a decision by the Expert Advisor at the same period of learning. We also get the result. We twist these two results, trying to understand the fit of the Expert Advisor, informativeness of the inputs, ability to detect patterns, etc.
I don't know if it has become clearer)
Suppose my Expert Advisor uses a swing rule with periods from 1 to 200 and opens in the direction of its direction from the 2nd to the 1st bar, MA[1]>MA[2] - buy, MA[1]<MA[2] - sell.
The numbers were wrong and the conditions are as follows MA[1000001]>MA[1000002] - buy, MA[1000001]<MA[1000002], i.e. the wand behavior on million bars does not describe the current situation when the EA opens the trade. The Expert Advisor's condition is "empty", a placebo. However, we will almost certainly find the waving period at which we make profit on the learning period. That will be pure naked chasing. This is what I was talking about. Although, to twist such a primitive Expert Advisor with a small number of degrees of freedom in such a way is probably meaningless. But in its place, imagine a NS whose inputs also have nothing to do with reality during the training period (for example, a binary sequence of bits from a digital copy of a video about the life of penguins)? Let's make the Expert Advisor make a profit at the learning phase from nothing, deliberately fit it and try to estimate how fit it is. Somehow... Then we give the correct data describing the situation at the moment of making a decision by the Expert Advisor at the same period of learning. We also get the result. We are trying to figure out the suitability of the expert and the informativeness of the inputs, etc.
I don't know whether it has become clearer).
Anyway, the topic is cool.
Obviously, a sufficiently complex set of tunable functions can be made to map any random set to any other set (random or not) by adjusting mapping parameters. Theoretically with any degree of accuracy... :)
Traders in DC (and not only) are also trained to trade this way: they teach to convert a rather random stream of standard (read - not working) indicators into a stream of trade operations... :-) ... .. // or :-( ... ?
Recently I've had an incident: I optimized an Expert Advisor using some smart indicators that have several working modes. Modes are parameterizable (each in its own way). So I set parameters of a wrong mode for optimization. I.e., these parameters had no effect on the performance of the Expert Advisor in the current mode of indicators.
// I have not found my mistake in time, because there are a couple of parameters that are optimized in addition to those that are not working. Therefore, the optimization has been completed.
An interesting result: these (non-working) parameters have gradually converged to their specific values and have not changed until the end of optimization. That is, they behaved in a pseudo-sensible way, as if they were really optimised... (!!!) If I were not the author of the expert, I would have decided that these are the Exact-Optimal-Graal-itp values, which should be carefully preserved (preferably in secret from the public) and yout, yout, yout...... And in no case do not change it!
When I saw and realized this case, I sadly thought - this is the exact mechanism of superstition creation, with which our entire inscrutable life, and Forex in particular, is filled with... ;-)
------------------
And as for the suggestion to study systems for "adjustability" - it's not worth it, in my opinion. It is clear that a good system is well adaptable. But it is also good at abstraction. A bad system can be poorly adaptable and well adaptable. But it can't abstract. So adaptability is hardly a criterion for the quality of a system (its generalizability).
It is better to use a well-known measure with addition of a low amplitude random signal at the stage of training to reduce the fitting phenomena. This is of more practical use.
Suppose my Expert Advisor uses a swing rule with periods from 1 to 200 and opens in the direction of its direction from the 2nd to the 1st bar, MA[1]>MA[2] - buy, MA[1]<MA[2] - sell.
The numbers were wrong and the conditions are as follows MA[1000001]>MA[1000002] - buy, MA[1000001]<MA[1000002], i.e. the wand behavior on million bars does not describe the current situation when the EA opens the trade. The Expert Advisor's condition is "empty", a placebo. However, we will almost certainly find the waving period at which we make profit on the learning period. That will be pure naked chasing. This is what I was talking about. Although, to twist such a primitive Expert Advisor with a small number of degrees of freedom in such a way is probably meaningless. But in its place, imagine a NS whose inputs also have nothing to do with reality during the training period (for example, a binary sequence of bits from a digital copy of a video about the life of penguins)? Let's make the Expert Advisor make a profit at the learning phase from nothing, deliberately fit it and try to estimate how fit it is. Somehow... Then we give the correct data describing the situation at the moment of making a decision by the Expert Advisor at the same period of learning. We also get the result. We twist these two results, trying to understand the fit of the Expert Advisor, informativeness of the inputs, ability to detect patterns, etc.
I don't know if it has become clearer).
Not only there is always a MA for the optimized interval of history, at which the TS will be profitable, but attention... there may be several MAs.
Thus, the situation may well be like yours. I.e. system parameters, having nothing in common with market regularities, are profitable at different intervals of history, and may even be profitable some time in the future. And they may not.
For this reason the TSs are divided into two types. The second type will either give profit or not. It is easy to check. But the first type - dunno, how can we check it?
In my opinion, you should not look for a line between fitting and not fitting. You should concentrate on the TC, where this line does not exist at all. That is how it is. At least there will be a feeling that there is no borderline and a pattern has been found. How else can it be? There are no guarantees and will never be. There will be an eternal condition of uncertainty and an eternal right to choose. :)
I don't know if it's clear.
I don't know if I got it right this time. :)
And as for the suggestion to study systems for "fit" - it's not worth it, in my opinion. It is clear - a good system is well adaptable. But it can also abstract. A bad system can be poorly adaptable and well adaptable. But it can't abstract. So adaptability can hardly be a criterion for the quality of a system (ability to generalize).
Well, it's not an end in itself, a test of fit, it's just a wandering thought in search of a "boundary") Practical results are closer to me somehow. The thought is forming approximately the following: we have two results of learning on one piece of history, fitting (P1) and fitting+lawfulness (P2).
P1 ~= P2:
- The TS has an excessive generalisation capacity to find the patterns it is designed for (e.g. in the NS - remove 1 layer, make fewer hidden neurons, take a different activation function, etc.), otherwise it would be very difficult to isolate the result exploiting the patterns. Or it's simpler, our regularities are not regularities at all.
2*P1< P2:
- what is needed, it makes sense to mess around with TC, the best training result is likely to be capable of detecting patterns.
These are the kinds of chains for practical application that I'm trying to build. And I think for some of my TCs it might work.
And did I get it right this time. :)
I must be close to something) I was already scared of my own ability to express my thoughts.
1) Fit in order to take advantage of the after-effect (inertia of market properties)?
2) Don't adjust to take advantage of the no-false-optimisation effect?
A contradiction?
There are other options :)
1) Fit in order to take advantage of the effects of aftereffects (inertia of market properties)?
2) Don't adjust to take advantage of the no-false-optimisation effect?
Contradiction?
There are other options :)
The effect of no false optimisation... Nearly broke my brain.
1) Market inertia property. The whole TA is based on it. TA makes the assumption that a process, once started (no idea when), will continue (no idea how long). A typical example: MA200, price has crossed a slippery bar downwards, we enter the Sell position hoping that it will continue going down. But the price does not go, it does not care about МА200, it immediately turns around and goes up. We wait. The price moves upwards and crosses MA200 upwards - this is a Buy signal - the wife screws up the new coat).
Same goes for channels, levels etc.
2) The property of causality. If there is a chain of events of pre-determined sequence and length, so there is a return chain of events, as a known in advance sequence and length. The method of optimization reveals the regularities of influence of changes in the causal chain of events on the consequent chain of events. Here everything is simple - either we find regularities or not, it is easy to check, easy to use.
Suggest those very other options.
joo:
2) Causality property. If there is a chain of events of a pre-determined sequence and length, then there is also a reciprocal chain of events of a pre-determined sequence and length. The method of optimization reveals the regularities of influence of changes in the causal chain of events on the consequent chain of events. It is simple - either we find regularities or we do not, it is easy to check, easy to use.
Continuing the "mind fooled by chance" or "creative madness of Life's genetic mechanisms" cycle. ;-)
Excerpt from a book on learning with plus reinforcements. Karen Pryor "Don't growl at the dog".
// The whole book is in the trailer. Recommended.
Superstition: random reinforcements
In real life, reinforcements occur at every turn and are often just an accidental coincidence. A biologist who has studied hawks observed that if a hawk catches a mouse under a certain bush, it will check that bush daily for a week, and sometimes more; the likelihood of it flying over that particular spot is due to the strength of the reinforcement. Try walking past a rubbish bin without looking carefully at it, if the day before - you found five dollars in it. Random reinforcement is good for the hawk; generally speaking, animal behaviour can be said to have evolved such that each species has the ability to benefit from any reinforcement. However, many random reinforcements are not accompanied by a useful outcome, but can nevertheless have a strong influence on behaviour. When behaviour is not related to subsequent events but is associated in the subject's brain with them as a necessary condition for their occurrence, we speak of superstitious behaviour. An example of this is the person who chews on a pencil. If you happen to put a pencil in your mouth during an exam and immediately the right answer or a good thought occurs to you, this reinforcement could change your behaviour: good thoughts came up while you were chewing on the pencil, so the action is reinforced. When I was in college, I had not had a single pencil that was not covered in teeth marks--in particularly difficult exams I sometimes chewed the pencil in half. I was sure it helped me think. In reality, it was just random, contingent behaviour. The same could be said of wearing certain clothes, or performing a certain ritual before taking up a particular task. I saw a baseball player who performed a nine-part chain of actions every time he prepared to pitch: touching his cap, touching his glove with the ball, moving his cap forward, rubbing his ear, moving his cap back, shuffling his foot, etc. At difficult moments, he could repeat all nine actions twice without ever disturbing their order: This sequence of actions was done very quickly, commentators never dwell on it -- but it nevertheless constitutes complex superstitious behavior. "Superstitions" often arise in animal training. The animal may be guided in its responses by criteria that you did not intend to introduce, but which often happened to coincide with the reinforcement and formed a conditioned bond. For example, an animal may believe that in order to be reinforced, it must be in a certain place, turn in a certain direction, or sit in a particular way. When you want it to work in a new place or with a different orientation, suddenly the whole behaviour is mysteriously broken, and go figure out why this happened. So it is much better, as soon as the behaviour starts to form, to start diversifying the options for conditions that do not seem important to you, so that there is not some accidental conditioning that will later get in your way. Most of all, make sure that no random temporal connections are formed. Both animals and humans are very good at sensing time intervals. Once I was quite sure that I had trained two guinea pigs to jump on command (at the signal of my hand), until one of the visiting scientists proved to me with a stopwatch in his hand that they jumped every twenty-nine seconds. I was the one who had inadvertently conditioned them to give the command with very great regularity, and they took advantage of this instead of the information I assumed they were supposed to be using. Many hereditary trainers are simply captive to a superstitious way of thinking and behaving. I have met some among them who say that dolphins prefer people dressed in white, that mules should be beaten, that bears don't like women, etc. This also applies to those who work with people and believe, for example, that fifth graders should be shouted at and that punishment is necessary to gain respect. Such educators are at the mercy of tradition, they are forced to always work in the same ways because they cannot separate effective methods from what is simply superstition. This weakness, or confusion, is found in many professions -- in education, engineering, the military, but perhaps most of all in medicine. It is appalling how many things are prescribed to the patient, not because they have medicinal properties, but because it has always been done or is being done. Anyone who has ever been in hospital can think of half a dozen examples of unnecessary actions that constitute nothing more than superstitious behaviour. Interestingly, superstitious behaviour does not disappear if you simply point out its ineffectiveness; being very much rote, it is correspondingly highly guarded. Try talking to a doctor about his habit of using ineffective or even harmful treatment, and you will be rebuffed in appropriate terms; I am sure that even that baseball player with a nine-step superstitious expression of nervousness will vehemently oppose anyone who suggests that he play ball without, say, a cap which he touches four times. The only way to get rid of superstitious behaviour is to make sure it's not related to reinforcement. My son Ted loves fencing. He goes to practice two or three times a week and often goes to competitions on the weekends. One day during a duel with a strong partner, he felt depressed because he left his favourite sword at home. He lost the match. Then he realised that feeling depressed obviously had much more influence on his actions than the sword he was using, and hence having a 'favourite' sword--a superstition. Ted identified and struggled with any superstitious behaviour that might be associated with fencing. He found many such dots in himself, ranging from an attachment to certain articles of clothing to an inner belief that his fighting might be affected by a dream, an argument or even the absence of fruit juice at a competition. By systematically analysing each of these circumstances, he broke one by one his reliance on them, as he realized that they were superstitions. As a result, he now goes into each fight calm and confident, even if he had a nightmare about being late for the train, losing his equipment, battles with taxi drivers, even if he fences with a borrowed sword in a tracksuit and different socks.