Discussion of article "The algorithm of ticks’ generation within the strategy tester of the MetaTrader 5 terminal" - page 4

 
Renat :
But you are participating in the topic MetaTrader 5 + MQL5 + Tester and are directly competing with the developers.


Since you do not know the MetaTrader 5 terminal, you are not aware of the trading modes in Tester and have not even paid attention to my explanation of these modes with a picture on the first page of this topic.

Testing modes are specially designed to sober up traders and write robust Expert Advisors. This will dramatically and qualitatively improve the Expert Advisors.

I have repeatedly written about aggressive modes in the tester in forums (MQL4.com and MQL5.com).

This is the root of the problems - unwillingness to test in practice.

In general, our dialogue looks like a discussion between opponents and supporters of locks,

Each side does not want to hear the other (probably because one side has suffered some solutions that opponents do not see and do not understand).

It would be good that you are right and the practice will show it,

I wish you good luck and believe me it is a sincere wish as it is more important for me to use a better terminal.

(and have profit with its help) than to be right in some dispute.

 
Renat писал(а) :

...
4. Remembering your "skill" in modelling and your attitude to it in MT4. I can ask one more question. Do you have the same number of ticks in a bar ? In MT4 you used to throw away 20% of ticks. How is it now? If a real bar has 100 ticks, how many ticks will you have in modelling?

It coincides, but you don't want to check anything. Theoretical thinking is enough for you.

Okay, let's check it. I've sketched out an expert. Here's the code.

  int ticks=0; 
  double  old_bid=0, old_ask=0;
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit() {return(0);}
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason){}
//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
//---
  MqlTick last_tick;
//---
   if(SymbolInfoTick(Symbol(),last_tick)) // получаем тик по символу
     {
     // ограничение по времени, собираем смоделированные тики за 1 час. 2010.05.21   22:00
      if(last_tick.time>"2010.05.21 21.00.00" && last_tick.time<"2010.05.21 23.00.00") {
      if(old_ask!=last_tick.ask || old_bid!=last_tick.bid)  { // если нет изменения цены это не тик, пропускаем
         // это тик запоминаем цену, увеличиваем счетчик тиков и выводим принт для проверки
         old_bid=last_tick.bid;
         old_ask=last_tick.ask;
         ticks++;
         Print("ticks=", ticks," ",last_tick.time,": Bid =",last_tick.bid," Ask =",last_tick.ask);
     }      
     }
     }
   else Print("SymbolInfoTick() failed, error =",GetLastError());
//---
  }

Here's the result.

The numbers don't match. Now one of two things. Either my hands are crooked or you're lying. Anyone can download and check it for themselves. If you find an error in my code, I'd appreciate it.

Files:
111.mq5  3 kb
 

The practical test resulted in the first option. You made a mistake in the limits by specifying 2 hours instead of one.

Here is the correct variant of the time cut-off:

      if(last_tick.time>="2010.05.21 21:00" && last_tick.time<"2010.05.21 22:00") 

And the results:

2010.05.23 20:13:07     Core 1  ticks= 2315   2010.05.21 21:59:59 : Bid = 1.25721  Ask = 1.25738

2315 ticks were generated when the real 2318 ticks were generated. A loss of 3 ticks out of 2318 is a normal variant.


 

Great. You agree that this code can be checked. You just need to enter the correct time interval. Please enter this one.

      if(last_tick.time>"2010.05.19 12.00.00" && last_tick.time<"2010.05.19 13.00.00") {

Explain where 5200 ticks went ?

 

It is good that you are continuing with the practice tests.

I don't have 5869 ticks on this watch, I have 679. Here are my results of running this period - 679 ticks. 100% of the ticks were simulated.


Apparently, you have a cached clock (and the minutes were correct for a total of 679 ticks) - we turned on the ticker for a short time on the 19th, but then we rolled it back.

Just invoke the "Refresh" command on the chart and run it again.

Pay attention to the correct format of time YYYY.MM.DD HH:MM:SS.

 

an issue that was forgotten to be covered in this article

with the appearance of real volume data in mt5

how this information will be generated

 

I understand pips traders perfectly well, although Urain and Prival dismiss it. But my TSs are built on the contrary on the maximum possible noise filtering, TSs become immune to the qualities of ticks. Such TSs can be used with any quotes providers. Regarding NN, which Urain mentioned - it makes no sense to build a TS on NN sensitive to ticks quality.

I also understand the developers well.

I see a compromise in the creation by the developers of an add-on to the standard tick generator that would allow to regulate the "fluffiness" of generated ticks, with the possibility of regulating the frequency of occurrence and size of emissions in ticks. In my opinion, this is not the same as adjusting the "degree of aggressiveness" of testing. Perhaps we should consider the option of some kind of analyser for the degree of correspondence between the generated ticks in the tester and the real ones, so that then we could output the parameters available for users to adjust.

In any case, the developers need to do something to protect their interests and the interests of end traders. Compromise solutions can be found almost always.

PS Yes, and one more thing. I don't know why we need tick history (because adequate real testing is required), if DC can change the filters even every single day, making the accumulated tick history useless, because the future behaviour of the TS will be quite different from what was expected after testing even on the real tick history. That is why I consider the variant with "fluffiness" adjustment by the user to be more preferable than the possibility to have a real tick history.
 
joo писал(а) :

I understand pips traders perfectly well, although Urain and Prival dismiss it. But my TSs are built on the contrary on the maximum possible noise filtering, TSs become immune to the qualities of ticks. .....

And I don't understand people who label without understanding. Once again, here is the link from the first page. https://www.mql5.com/ru/forum/115584/page11#150512. Take a closer look at the picture !!! you can see everything there - trades are plotted on the chart.

Parameters - average trade size in pips 650, minimum time to hold a trade 1 hour. AM I A PIPS MAN ?

Why do YOU forbid me to analyse ticks? Do I forbid you to work and analyse the incoming information as YOU see fit? You think that by working on hourly ticks YOU automatically got rid of noise - this is a delusion, I suggest even better variant - work on annual candlesticks, the system will be even better, even more robust, even noiseless, the data providers have no problems, gave 5 digits of OHLC and "volume" once a year to the trader and let him trade. Everyone is happy, the trader has a robust system, there is no noise, the supplier has no problems with scaling, etc. (there is such a method in science, substitution of extreme cases to understand whether an idea is nonsense or not... draw your own conclusions).

Now I will write a reply to Renat and that's it. I will try to describe where I see discrepancies between modelling and real quotes. This is his brainchild, he decided to model ticks, and he believes that many parameters of are unimportant in modelling.

Our business is to understand where we can trust the tester and in what. And where the tester will deceive us. We cannot change each other's minds. Renat is under the delusion that it is possible to create a model that is better than the real one. I don't think so.

We are given the opportunity to study the TC only on the model and there is no possibility to run the created TC on real data (bad or good, fluffy or not, it doesn't matter). We don't have this opportunity!!!. Why don't they give us the opportunity to download the history at least independently, say from here http://www.dukascopy.com/swiss/russian/data_feed/historical/, put it into the tester and run the TS on this data? The result you all know very well, has no one encountered? In the tester everything is good, but in real life, even if you found the TS, you can get so beautifully on the face https://www.mql5.com/ru/forum . And you have to be an absolute paranoid to model such a thing (even with the help of a generator) and try to create a TS that will be resistant to such a thing and work in such conditions....

Now about comparing the number of history ticks and modelled ticks....

Yes, it is a problem with history (this is exactly what we are talking about).

I wrote to the Service Desk #14710 2010.05.18 08:42 a.m. about failures in quotes. They took a long time to sort it out, it seems that they have not solved it. On 20.05 I cleared all quotes manually, downloaded them all new, so it's not that (you were experimenting on the 19th). Quotes I have fresh by correspondence from Service Desk downloaded all new around 2010.05.20 12:08.

So that there would be no more misunderstandings. Again completely deleted the folder histories and uploaded a new one.

  1. Yes indeed, when I clicked the refresh button - the history changed. The data matched ticks= 678.
  2. From the data provider's point of view it is good, he can change the history like a wizard.
  3. From the trader's point of view it is a nightmare. Exclude such a situation. I have this data in my terminal, they participate in calculations, in indicators, they are used to build TSs and make decisions.

What else I paid attention to, this check is useless.

if(old_ask!=last_tick.ask || old_bid!=last_tick.bid)  

1. You do not have situations when only ask or bid changes (seller or buyer is dismantled). And this happens very often. Bid stands, only ask changes and vice versa.

2. Spread is equal to a constant. This also does not correspond to the realities.

3. If you want help in researching the quality of the model. Collect ticks yourself, preferably from 3-4 sources and post them. Show the results of the modelling of these ticks not in the form of a picture, where the price goes down by 30 points per 1000 ticks. But let's say for a day, where there are different situations, including large movements in a short time interval.

4. Give numerical characteristics of deviation from the true data, at least the RMS on both axes.

5. This is done all over the world, the results can be trusted if someone else with the same initial data gets the same result.

6. Otherwise, the situation will be as we have above, you have 678 ticks in the terminal, I have 5800. If we add here all the nuances of collecting ticks, we will never find the truth.

7. Sitting for 2-3 days collecting ticks to show or prove something to someone, sorry, I have already gathered in my time. It is enough for me to see how tics are generated to form an opinion.

8. It is a pity that because of the chosen format of storing and feeding history into the terminal, we traders lose information. Renat, no matter how hard you try, you will not be able to build a chart, let's say in the form of Renko, Kagi. It will be redrawn and different from what is built on real data.

My conclusions and recommendations for traders. You can take them into account, or not, it's up to you.

  1. The testing mode is useless. It is a waste of time, because the structure of ticks inside the bar is not accurately reproduced.
  2. The most correct method of testing is probably the same as in MT4 at opening prices. More precisely, you need to make sure that a new bar has started and take the prices of the previous one. At the same time, in real trading you need to use "paranoid" checks for the presence and synchronisation of data like this https://www.mql5.com/ru/forum/993 and the results may not coincide anyway.
  3. If you test the system with a small stop or trawl value, the results may differ even at the opening prices, because of a possible mismatch between the time of Low and High. (They can be rearranged).
  4. Before the Expert Advisor works in real life, when initialising it, you should somehow check the correctness of history (simulate pressing the refresh button).
MetaTrader не отражает реальности ! как с этим бороться? - MQL4 форум
  • www.mql5.com
MetaTrader не отражает реальности ! как с этим бороться? - MQL4 форум
 
Prival :

I don't understand people who label people without knowing what they are talking about. Once again, here is the link from the first page. https://www.mql5.com/ru/forum/115584/page11#150512. Take a closer look at the picture !!! you can see everything there - deals are plotted on the chart.

Parameters - average trade size in pips 650, minimum time to hold a trade 1 hour. AM I A PIPS MAN ?

Why do YOU forbid me to analyse ticks? Do I forbid you to work and analyse the incoming information as YOU see fit? You think that by working on hourly ticks YOU automatically got rid of noise - this is a misconception, I suggest an even better option - work on annual candlesticks, the system will be even better, even more robust, even noiseless, the data providers have no problems, gave 5 digits of OHLC and "volume" once a year to the trader and let him trade. Everyone is happy, the trader has a robust system, there is no noise, the supplier has no problems with scaling, etc. (there is such a method in science, substitution of extreme cases to understand whether the expressed idea is nonsense or not... you can draw your own conclusions).

................

Why are you reacting so violently, I'm a bit taken aback? I was supporting your point of view. I don't like the word "pipsqueak" - I take it back. I am sorry that I caused you a storm of negative emotions.

I understand the word "piper" in a broader sense. A piper is a TS that is sensitive to the quality of ticks. That's all. As if you have read only the first line of my post. And further in my post, I offered a compromise solution.

PS I use other methods of noise filtering than switching to a higher TF (in bold in my post). My research is focused on M1.

 
joo писал(а) :

....

If you took this information personally, I apologise. Yes, I react violently, because instead of trading and earning a piece of bread for my family, I sit and write. This was rather written to those who think that if a person analyses ticks, he is a pips player. This is not true. I try to show it as much as I can.

There is no concept of a qualitative/non-qualitative tick. Accordingly, there is no aggregate of qualitative or non-qualitative ticks. A tick is a reality given to us in the trading terminal. It comes to us, the Expert Advisor is launched and we analyse it. Whoever is present here will be able to say. Is this tick of high quality and this one is not?

Someone analyses only the closing of bars = a tick that came at the end of the hour. How does this tick differ in quality from the tick that came a second earlier? Let's say someone claims that it is more "fluffy", what is that and what is it with? I hope you see the absurdity of such claims.

Now just think about it, quote providers work at the level of ticks, yes they filter them, they have several suppliers, they give us what they consider necessary and profitable for them at a given time. But they work with ticks, not with OHLC, but with ticks. The ones that are then folded into bars. Are all the DCs there stupid and do not see that it is suicide to work at the level of ticks?

If not, are they smart and competent? Why should we work with the average temperature of the hospital and not with ticks? From ticks you can cut anything you want, any kind of chart. You can't do that with minutes, because it's a lossy form of data compression, irreversible loss.

It is like a packing algorithm, the same *.rar compressed the information, passed it to another, he unzipped it and read it, let's say it is a book. And now we change the algorithm, unpack only the first(Open) and the last(Close) letter in the sentence, well, you can add two more letters, the thickest (analogue of High) and the thinnest (analogue of Low). How's that? Who like to read such a book? It's the same here ((((