Discussion of article "Grokking market "memory" through differentiation and entropy analysis"

 

New article Grokking market "memory" through differentiation and entropy analysis has been published:

The scope of use of fractional differentiation is wide enough. For example, a differentiated series is usually input into machine learning algorithms. The problem is that it is necessary to display new data in accordance with the available history, which the machine learning model can recognize. In this article we will consider an original approach to time series differentiation. The article additionally contains an example of a self optimizing trading system based on a received differentiated series.

The Expert Advisor was run with the specified hyperparameters without genetic optimization, i.e. almost at random, on the EURUSD pair with the 15-minute timeframe, at Open prices.

Fig. 5. Tested Expert Advisor settings

Fig. 6. Results of testing with the specified settings

Fig. 7. Virtual tester results in the training sample

In this interval, the implementation showed a stable growth, which means that the approach can be interesting for further analysis.

Author: Maxim Dmitrievsky

Grokking market "memory" through differentiation and entropy analysis
Grokking market "memory" through differentiation and entropy analysis
  • www.mql5.com
A large number of participants in liquid markets, who operate with different investment horizons, produce a lot of market noise. Thus markets have a low signal-to-noise ratio. The situation can be worsened by attempts of integer time series differentiation, which erases the remaining memory and converts quotes to a series characterized by...
 

Thank you for your work in this article.  I would like to learn more about this topic.  I wish I was better at statistics!


--Dave

 

private:

//||||||||||||||||||||||||||||||||||||||||

   int               number_of_samples, relearn_timout, relearnCounter;

   virtual void      virtual_optimizer();


There is an error in Auto Optimizer file 


'virtual_optimizer' - function already defined and has different type in Auto_optimizer.mqh


 
developeralgo:

private:

//||||||||||||||||||||||||||||||||||||||||

   int               number_of_samples, relearn_timout, relearnCounter;

   virtual void      virtual_optimizer();


There is an error in Auto Optimizer file 


'virtual_optimizer' - function already defined and has different type Auto_optimizer.mqh


hi, pls add the "void" modifier. In new compiler it must be like there

//+------------------------------------------------------------------+
//|Virtual tester                                                    |
//+------------------------------------------------------------------+
void CAuto_optimizer::virtual_optimizer(void) {
 
Maxim Dmitrievsky:

hi, pls add the "void" modifier. In new compiler it must be like there

Thanks Maxim, do you have set file that you have found from your testing to be great that i can start using for initial testing and then build or improve 

 
developeralgo:

Thanks Maxim, do you have set file that you have found from your testing to be great that i can start using for initial testing and then build or improve 

you can try settings from screenshot in article ("Value" column)

 

Thank you.


I am very excited about this article and looking forward to implementing!!!

 
For those having difficulties compiling.
https://www.mql5.com/en/code/16006 <- location of MT4orders

#include <Auto optimizer.mqh> > #include <Auto_optimizer.mqh>

Clearly this is a masterwork. Is it possible to do this code in MT4 or it is too limiting?
MT4Orders
MT4Orders
  • www.mql5.com
This library allows to work with the orders in MQL5 (MT5-hedge) in the same way as in MQL4. That is, the order language system (OLS) becomes identical to MQL4. At the same time, it is still possible to use the MQL5 order system in parallel. In particular, the standard MQL5 library will continue to fully operate. It is not necessary to choose...
 

I am getting these errors:

1. It seems EA is attempting to sell but placing stops as though it was a buy? 
2019.07.25 04:35:35.174 Trades '666': failed  market sell 0.10 USDCHF sl: 0.98139 tp: 0.99039 [Invalid stops] 

2. The EA removes itself on the formation of a new bar or candle.

3. The EA can only place sell trades (unless this is by design during the optimization phase?)

4. EA uses a different trade volume for NZDUSD only?

update0: after further testing I can say the problem of auto-removal is related to the custom settings from the hyperparameters screenshot. I will attempt to isolate which one.
update1:It is related to adjusting the history depth to 1500.  I will attempt to see if different values create different results. I am testing on M1 and M15 and have sufficient price history.
update2: EA will remove itself if History_depth  1027 >= ||<= 956 

5. After some time the entropy window will shrink to a small size as in the screenshot.

update3: I can suggest the problem with sell-only is related to too low a timeframe used in testing (M1). This generates very large entropy values which are out of bounds? Testing on M15>= creates both buy and sell orders. 

update4: I can suggest the entropy window size issue is related to the initial size of the chart window. If it is scaled full screen the window appears OK. 

update5: After attempting to modify magic numbers now all charts remove on the next bar. I've tried restarting the terminal, using new charts. I'm not sure what else to try, there are no errors in the log. I cannot test further?
 
Congratulations for the high level article !!! Success and Prosperity!
 
clemmo:

I am getting these errors:

1. It seems EA is attempting to sell but placing stops as though it was a buy? 
2019.07.25 04:35:35.174 Trades '666': failed  market sell 0.10 USDCHF sl: 0.98139 tp: 0.99039 [Invalid stops] 

2. The EA removes itself on the formation of a new bar or candle.

3. The EA can only place sell trades (unless this is by design during the optimization phase?)

4. EA uses a different trade volume for NZDUSD only?

update0: after further testing I can say the problem of auto-removal is related to the custom settings from the hyperparameters screenshot. I will attempt to isolate which one.
update1:It is related to adjusting the history depth to 1500.  I will attempt to see if different values create different results. I am testing on M1 and M15 and have sufficient price history.
update2: EA will remove itself if History_depth  1027 >= ||<= 956 

5. After some time the entropy window will shrink to a small size as in the screenshot.

update3: I can suggest the problem with sell-only is related to too low a timeframe used in testing (M1). This generates very large entropy values which are out of bounds? Testing on M15>= creates both buy and sell orders. 

update4: I can suggest the entropy window size issue is related to the initial size of the chart window. If it is scaled full screen the window appears OK. 

update5: After attempting to modify magic numbers now all charts remove on the next bar. I've tried restarting the terminal, using new charts. I'm not sure what else to try, there are no errors in the log. I cannot test further?

Hi, I think there are some problems with normalization for entropy, that's why it change its values over time. Maybe a little later I fix this an include another indicator About another problems - I think I don't fully understand. Maybe have some errors in terminal log?

Reason: