Machine learning in trading: theory, models, practice and algo-trading - page 533

 
Maxim Dmitrievsky:

Yes, why not good? MO is essentially optimization until AI is invented.

Genetics is also a part of AI.

Moreover, there is a rather promising, hybrid direction - evolving NEAT neural networks.

 
Yuriy Asaulenko:
Criteria are important in "optimization". Optimization by maximal profit is not a good criterion. There is no reason to believe that it will somehow work in the future, which we often see in practice, in particular on the forum.

not necessarily by max profit, you can use R^2 as it was recently suggested in the article or something else, there are custom criteria. The point is not that, but to find stable dependencies through opt. After that you take several options hyperparameters through which entire modules are tried, say, a set of features or something else by certain rules... If it makes more sense) then best runs are chosen and analyzed why they are best and what interesting things were found, and then the system is assembled from created things and is not optimized anymore

 
Ivan Negreshniy:

Moreover, there is a rather promising, hybrid direction - NEAT evolving neural networks.


something new from you again, I will go to Google for help :)

 
Ivan Negreshniy:

Moreover, there is a rather promising, hybrid direction - NEAT evolving neural networks.

Thank you.

You have voiced what I have been thinking about, but could not formulate a thought, for the last few months.
 
Ivan Negreshniy:

NEAT

The topic in itself is interesting, but it didn't pass the forex test. There were some articles about it in the last thread, even there is a package for R -https://github.com/ahunteruk/RNeat.
NEAT a couple of words - we select neuronkey weights using genetic algorithm instead of conventional training.
Here is an example of the algorithm in action, the neuronka is trained to play a Mario gamehttps://www.youtube.com/watch?v=qv6UVOQ0F44

If with normal neural network training you can sometimes pause training and check overfit on new data to stop training in time, with NEAT it won't work, the genetics will search for weights that best fit the fitness function until it reaches its limit, resulting in a strong overfit and useless model on new data.

 
Petros Shatakhtsyan:

Testing on real ticks, that's what real time is. And it's too bad that you don't use a tester.

It's strange to hear that, "real ticks" are not in the MetaTrader tester, they are generated, mathematically it is like to open/close on (O+H+L+C)/4 of the next candle, before which there was a signal.

 
Dr. Trader:

Minus this exchange (bittrex) - their api have no functions to get ohlc values

Well no, there is. It's just they have not documented yet api 2.0, the information is somewhere in the nether regions of the Internet ((

Example of candlc request:bittrex.com/Api/v2.0/pub/market/GetTicks?marketName=BTC-ETH&tickInterval=day&_=1499127220008

Well, their candlesticks are bad, they don't make them from their order book but they aggregate them from other sources, the highs and lows may be much stronger than on their charts, you have to write ticks in any case
 
Alyosha:

Well, of course not, there is.

Indeed there is, thank you.

 
Aliosha:

It is strange to hear that, there are no "real ticks" in the MetaTrader tester, they are generated, in the mathematical sense it is something like to open/close at (O+H+L+C)/4 of the next candle, before which there was a signal.


It's strange to hear that there are no real ticks in metatrader's tether, it feels like you're an optician working with the platform

 

Hi all!!! A familiar question. HOW to make the AI last as long as possible without over-optimization???? Personally, I see two answers myself.

1. Improving the quality of inputs, i.e. to find such an input which will be the reason for the output. The task is extremely difficult and sometimes not feasible, because such inputs may not be in nature in principle.

2. Extend the period of training with the proper level of model quality. This is the approach I think we shall consider.....

Using a committee of two grids, we obtain three states "Yes", "No" and "I do not know" - when two grids look in different directions, this is the effect we will try to use.

At first I train a network on 1000 records. As a rule, about 60% of them will get "I don't know" status. Then we build a second level model, where we train it only on "I don't know" states. Besides, in the second level about 300 out of 600 entries will be undefined. That's right, brothers, we're using boosting. That is, it performs multiple retraining of the network. I managed to get to the third level. It allowed us to train the model for about three months on TF M15. I admit that preparation of the models took more than two days, and no wonder, considering that I had to build 6 waveforms + 8 bounce models, but all that time is spent only to increase the capacity of the TS up to one month without human intervention and overoptimization.

This picture shows the training period of the TS. The test was performed with one lot without the possibility of trailing positions. That is, without the SCAN model!

Pay attention to the "Profitability". It should not be too great, from 2 to 5... within these limits. Since a high level of this indicator is a sign of overtraining IMHO. When the NS has learned this section.

In the next picture I have connected the pullback model. I.e. when the signal appears we analyze whether the pullback will be, and if "Yes", we place a pending order, if "No", we enter the market. Here we use about four attachments of the model to the model and the result as we can see is much better in terms of "Profitability". Actually, this method is intended for the balance curve to have a smoother increasing type.

As a rule, the number of trades decreases, the profit decreases and the profitability increases.

But we know that the figures in the training area do not mean anything and this is true. But I've decided and answered one very important question: "How can I assess the quality of training? How do I know how well your model generalizes the market and in general, can it do this? The answer was on the surface and very simple. We need another test interval!!!!!! But what is it? ..... about it in the continuation....

Reason: