Machine learning in trading: theory, models, practice and algo-trading - page 1850

[Mihail Marchukajtes]

Aleksey Vyazmikin:

Do you agree that at the time of the opening of the minute bar OM should be taken for the previous record? For example, we take the opening at 10:00 a.m. as 23:49:55.

I think it is better to use the indicator on M1 and take all necessary information from the zero bar and make different comparisons in the Expert Advisor, taking into account the request for information from the indicator buffer with the desired offset.

What is the third value in your file - the first is the date, the second is OM and the third is OI? I thought it was a delta, but it does not work.

I want to use it as a reference indicator, it works in real time, in real time.

Yes, and reading from file should work in case of data interruption and presence of data in file, but market is closed and did not check it.

This is not really important at all. It is important that the written data is always loaded correctly in the indicator. The point is that such errors are allowed before the training file is saved. If we have specified this rule, let it be so, the indicator should fulfill this rule in real conditions.

Alexey, thank you for the indicator, but I have a question about a real-time Expert Advisor. Can we make it write by the minute and not with every change of OM within a minute?

In general, I do not use zero bar in my work. Moreover, I perform all calculations 30 seconds after the signal on the first bar. Consequently, there are no calculations between the signal. But the signal on the first bar in 30 seconds initiates all indicators that participate in the model and, in theory, these indicators should calculate the entire history from the last reference (signal) to the current one on the condition that they are not in the chart. This is the hole that the indicator should read from the file in order to get the current value.

The point is that I use quote-dependent data (I named it so myself :-)) which are sensitive to gaps and holes in history.

Please help me bring the code of indicators I use and I'll thank you with a couple of key ideas when preparing training kits. + I have 37 bucks on my service I will give it to you no problem. But only on condition that the full reliability of the data obtained in the Expert Advisor in real trading will be achieved.

The main problem I have is that when I feed my EA with AI on a chart and get a fresh signal I need to recompile it and then it will request adequate data from the indicators. But the problem is in the Inductors themselves. I attribute them to a chart, it displays the history correctly and keeps on presenting data. I recompile it after some time and tails that were drawn on the real account change their values.

There is also a cumulative delta indicator that calculates correctly relative to bars. However, when the history of this indicator is long, and it is 3 months on М5, it has no time to count for one cycle as it takes data from copyic, where it understands how many cycles it gets.

With the latest update, the new version of "Learning File" leads to memory exhaustion, because the request is made for 14 instruments during 3-6 months of history. As a result 8 gigs of RAM is simply not enough. And this tool writes zero bar in real time, which I do not need. It's enough for me to load only the first one when zero bar appears.

If you want to give a little help, write in a personal message and we'll come to a more specific agreement. Thanks!!!!

[Aleksey Vyazmikin]

Mihail Marchukajtes:

In fact, it is absolutely not important. It is important that the recorded data is always loaded correctly in the indicator. The fact that such errors are allowed before the training file is saved. If we have specified this rule, let it be so, the indicator should fulfill this rule in real conditions.

Alexey, thank you for the indicator, but I have a question about a real-time Expert Advisor. Can we make it write by the minute and not at every change of OM within a minute?

In general, I do not use zero bar in my work. Moreover, I perform all calculations 30 seconds after the signal on the first bar. Consequently, there are no calculations between the signal. But the signal on the first bar in 30 seconds initiates the call for all indicators that participate in the model and in theory these indicators should calculate the entire history from the last call (signal) to the current one on the condition that they are not in the chart. This is the hole that the indicator should read from the file in order to adequately obtain the current value.

This is the point: it will be impossible to perfectly reproduce the training and application, especially in the fast market. Now it turns out that the delay is up to 10 seconds relative to the new data (if we're talking about history), and if we only look at the bar opening, the delay will be 60 seconds or more. If we work with data obtained at the bar opening (not shifted), there will be an error in the indicator, i.e. we may get a glimpse into the future. In general, we must once again think about the ideology, now the data is more recent, but it's all theory. Maybe it is enough to take the data recorded at the appearance of a new bar, and to learn from them.

Well, you can make a script that simply removes unnecessary lines within the minute and the file size will be 10 times smaller.

Mihail Marchukajtes:

The main problem is that when I send an EA with AI to chart and get a new signal I need to recompile it and then it will request appropriate data from indicators. But the problem is in the Inductors themselves. I attribute them to a chart, it displays the history correctly and keeps on presenting data. I recompile after some time and the tail that was recorded on the real account changes its values.

Try the version of the indicator I gave. If the problem persists, it is probably in the Expert Advisor and not in the indicators. Please provide the code for getting indicator data.

[Aleksey Vyazmikin]

Mihail Marchukajtes:
If the break occurs globally from the broker, then there is nothing you can do about it. I would say that the data was recorded without any losses.

Now it is expected that the advisor for the recording history will save the data in parallel with the work of the indicator and trading advisor, so when the connection or closing the terminal, the new historical data will be taken from the file, it will allow the urgent continuation of work and then replace the file with the VPS and restart the advisor.

[Aleksey Vyazmikin]

Mihail Marchukajtes:

\Alexei, thank you for the indicator, but I have a question about the preserving advisor. Can we make it write by the minute, but not with every change of OM within a minute?

If you do not think too much, then check this variant

   MqlTick last_tick;
   if(SymbolInfoTick(Name_instrFS,last_tick))
      StartDate=last_tick.time;
   else StartDate=TimeCurrent();
   if(isNewBar(Name_instrFS,0,PERIOD_M1))
   {
      for (int i=0; i<100 && !IsStopped(); i++)
      {
         h=FileOpen("OpenI\\"+Name_instr+"_OI.csv",FILE_WRITE|FILE_READ|FILE_ANSI|FILE_CSV|FILE_COMMON|FILE_SHARE_READ,",");
         if(h!=INVALID_HANDLE)
         {

            FileSeek(h,0,SEEK_END);
            FileWrite(h,StartDate,DoubleToString(interest,0));
            FileClose(h);
            Sleep(100);
            break;
         }
      }
      //inter=interest;
      // byOR=byORD;
      // sellOR=sellORD;
   }

[Mihail Marchukajtes]

Aleksey Vyazmikin:

Especially without thinking too much, then check this option

Alexey, thank you very much. I will surely check everything and inform you with the result. As for looking into the future, it is absolutely unimportant and it will not happen if the EA will write the way the indicator will read. If there is no difference between them, it does not matter. Alternatively, when a new bar appears, we wait for the first change of OI and write this new value for the first previous bar. We can even write not a new value, but the old value, which was the last for the first bar. That is, in fact, the entry will be made at the close of the bar.

It is not important which OM we assign to the minute bar, but how the indicator will be read from the file.

[Mihail Marchukajtes]

Mihail Marchukajtes:

Alexei, thank you very much. I will definitely check everything and report the results. As for looking into the future, it is absolutely unimportant and it will not happen if the EA will write the way the indicator will read. If there is no difference between them, it does not matter. Alternatively, when a new bar appears, we wait for the first change of OI and write this new value for the first previous bar. We can even write not a new value, but the old value, which was the last for the first bar. That is, in fact, the entry will be made at the close of the bar.

The important thing is not the OI we assign to the minute bar, but how the indicator will be read from the file.

Please look at the full code and ask

interest=SymbolInfoDouble(Name_instrFS,SYMBOL_SESSION_INTEREST); 


  if ((interest!=inter))
  {

   MqlTick last_tick;
   /*if(SymbolInfoTick(Name_instrFS,last_tick)) 
    StartDate=last_tick.time;
    else StartDate=TimeCurrent();*/
    StartDate=iTime(Name_instrFS,PERIOD_M1,1);
  if(isNewBar(Name_instrFS,0,PERIOD_M1))
   {  
    for (int i=0;i<100 && !IsStopped();i++)
    { 
      h=FileOpen("OpenI\\"+Name_instr+"_OI.csv",FILE_WRITE|FILE_READ|FILE_ANSI|FILE_CSV|FILE_COMMON|FILE_SHARE_READ,",");
       if(h!=INVALID_HANDLE)                                                         
       {  
         
         FileSeek(h,0,SEEK_END);
         FileWrite(h,StartDate,DoubleToString(inter,0)); 
         FileClose(h); 
         Sleep(100);
         break; 
       }
    }   
    inter=interest;
   // byOR=byORD;
   // sellOR=sellORD;
  }
}

If a new bar opens and no OI changes within 5 ticks, will it enter the new bar condition?

I think it should because it will be the first call of the new bar function and only after that the result will be changed into a falsh? Right?

[Mihail Marchukajtes]

Simulating the work of the code in my head, I found a bottleneck. The last bar of yesterday's session will be written at the first change at the opening of the minute bar of today's session, in case of re-initialization of the EA, the value of OI will be lost, which will lead to unpleasant writing of zero in the last bar of yesterday's session. I think the solution is still not to write the previous OM value in the first bar, but precisely the current OM, that is, the OM to be the first in the new bar. All right, I wanted to synchronize them perfectly, but I think it will do as it is... So... just thinking out loud...

[Mihail Marchukajtes]

Aleksey Vyazmikin:

This is the point, it is impossible to reproduce perfectly the training and application, especially in a fast market. Now it turns out that the delay is up to 10 seconds relative to the new data (if we are talking about history), and if we use only the bar opening, the delay will be 60 seconds or more. If we work with data obtained at the bar opening (not shifted), there will be an error in the indicator, i.e. we may get a glimpse into the future. In general, we must once again think about the ideology, now the data is more recent, but it's all theory. Maybe it is enough to take the data recorded at the appearance of a new bar, and to learn from them.

Well, you can make a script that simply removes unnecessary lines inside the minute and the file size will be 10 times smaller.

Try the version of the indicator I gave. If the problem persists, it is probably caused by the Expert Advisor and not by the indicators. Please show the code for getting the data from the indicator.

I have checked the indicator on the history and I've got an error. The thing is that the first one-minute candlestick has the time 10:00 in the window of quotations, but the value for this candlestick is taken from the previous candlestick. That is, the candle opened at 10:00:00, and closed at 10:00:59. This is the value that must be written, isn't it? And the indicator during the construction takes the value of the bar from 23:59:59.

[Mihail Marchukajtes]

Mihail Marchukajtes:
I checked the indicator on the history and I saw the error. The thing is that the first minute candle has the time 10:00 in the window of quotations, but the value for this candle is taken from the previous candle. That is, the candle opened at 10:00:00 and closed at 10:00:59. This is the value we need to write, isn't it? And the indicator takes the value from 23:59:59 for this bar during the construction.

I did it. Now it doesn't matter if the file is written per minute or per tick, it will write the correct values for the candle. That is, if a candlestick has an open time, then we write the value at the close of this candlestick...

[Aleksey Vyazmikin]

Mihail Marchukajtes:
I checked the indicator on the history and I saw the error. The thing is that the first minute candle has the time 10:00 in the window of quotations, but the value for this candle is taken from the previous candle. That is, the candle opened at 10:00:00 and closed at 10:00:59. This is the value that must be written, isn't it? And the indicator takes the value from 23:59:59 when constructing the bar.

If the Expert Advisor is working on the bar opening, even if it is with a delay, is it correct to feed it with the data on the candle closing, i.e., to look into the future? That's why it uses the freshest data before the close of this candle, i.e. it has a small delay, but not advance.

Mihail Marchukajtes:

Please look at the full code and ask

If a new bar opens and there is no change of OI within 5 ticks, will it enter the condition of a new bar?

I think it should because it will be the first call of the function of a new bar and only after it the result will change into a false bar? Right?

Where is the counting of 5 ticks here? It happens that within all minutes there is no transaction, then you miss a bar, and that's why I removed the comparison for OI changes, so that the data were written immediately at the opening of a new candle. I may have missed something, I should have tested it with data, but I had none yesterday and it's weekdays :(