Machine learning in trading: theory, models, practice and algo-trading - page 1849

[Aleksey Vyazmikin]

Mihail Marchukajtes:
Standard through ikustom... But it reads data from the file with every tick, and the indicator reads it when a new bar appears and it turns out that it takes a wrong value.

To begin with, not every tick is written

  if ((interest!=inter))
  {

   MqlTick last_tick;
   if(SymbolInfoTick(Name_instrFS,last_tick)) 
    StartDate=last_tick.time;
    else StartDate=TimeCurrent();
    for (int i=0;i<100 && !IsStopped();i++)
    { 
      h=FileOpen("OpenI\\"+Name_instr+"_OI.csv",FILE_WRITE|FILE_READ|FILE_ANSI|FILE_CSV|FILE_COMMON|FILE_SHARE_READ,",");
       if(h!=INVALID_HANDLE)                                                         
       {  
         
         FileSeek(h,0,SEEK_END);
         FileWrite(h,StartDate,DoubleToString(interest,0)); 
         FileClose(h); 
         Sleep(100);
         break; 
       }
    }   
    inter=interest;
   // byOR=byORD;
   // sellOR=sellORD;
  }

The OM is written only if it has changed and if it has changed more than 10 (100 cycles of writing the same lines multiplied by 0,1 second wait after each record) seconds ago. In fact, a lot of ticks are skipped, so accurate synchronization is no longer possible during playback. Although I don't work with ticks - in the tester, check all ticks or only those in which there was buying/selling of an instrument (asset).

It's better to write a symbol based on the results of a closed minute - there is less garbage data and synchronization in the indicator is more convenient.

As for the indicator - something there is drawn on the history - let's assume correctly.

And then on the zero bar there is such a mess.

  BufOI[0]=SymbolInfoDouble(Symbol(),SYMBOL_SESSION_INTEREST);
 LastUPdate=FileGetInteger("OpenI\\"+FileName1,FILE_MODIFY_DATE,true);
  if ((LastUPdate!=UPdate))
     {
       int h=FileOpen("OpenI\\"+FileName1,FILE_READ|FILE_ANSI|FILE_CSV|FILE_COMMON,",");
       string str=FileReadString(h);                            // читаем очередную строку из файла

    BufOI[1]=StringToDouble(str);
       FileClose(h);
   BufOI[0]=BufOI[1];

 UPdate=LastUPdate;   

We take the data from the ticking OI from the server and it's not present in the tester.

Then, if there is a new date of modification of the file, which is updated every 15 minutes in live mode, but not in the tester, we read data from the file (the last line) FileName1 =_Symbol + FileNames + "_TMP.csv" and we assign the data to the first bar and close the file and assign a value to zero bar. It is clear that this variant will not work in the tester.

You need two modes of the indicator - for working on the history and for working in real time, I think.

Check the correctness of the data on the history - it looks strange when reading the data from the file...

         while(!FileIsEnding(h))
           {
            ArrayResize(oi,ct+2,1000);
            string str=FileReadString(h);                            // читаем очередную строку из файла
            if(cnt==0) oi[ct].time=StringToTime(str);                 //если запись первая, т.е. дата, то конвертируем из стринга в дататайм
            else if(cnt==2) oi[ct].oi=StringToDouble(str);   //если запсиь вторая, т.е. ОИ, то конвертируем в инт и 

            cnt++; // увеличиваем счетчик прочитанных строк
            if(FileIsLineEnding(h)) {cnt=0; ct++;}
           }

It turns out that the data was read

[Mihail Marchukajtes]

Aleksey Vyazmikin:

To begin with, not every tick is written

OM is written only if it has changed and if it has changed more than 10 (100 cycles of recording the same lines multiplied by 0.1 second wait after each recording) seconds ago. In fact, a lot of ticks are skipped, so accurate synchronization is no longer possible during playback. Although I don't work with ticks - in the tester, check all ticks or only those in which there was buying/selling of an instrument (asset).

It's better to write a symbol based on the results of a closed minute - there is less garbage data and synchronization in the indicator is more convenient.

As for the indicator - something there is drawn on the history - let's assume correctly.

And then on the zero bar there is such a mess.

We take the data from the ticking OI from the server and it's not present in the tester.

Then, if there is a new date of modification of the file, which is updated every 15 minutes in live mode, but not in the tester, we read data from the file (the last line) FileName1 =_Symbol + FileNames + "_TMP.csv" and we assign the data to the first bar and close the file and assign a value to zero bar. It is clear that this variant will not work in the tester.

You need two modes of the indicator - for working on the history and for working in real time, I think.

Check the correctness of the data on the history - it looks strange when reading the data from the file...

It turns out that the data was read

I absolutely agree with you there. And the data it writes several values within a minute, it is not every tick, but still. Then it builds any TF from this data.

That was my idea about the ТМP file. I tried to update the indicator at every new candlestick and it was kind of right on the real account. But at a certain moment it will either take the last value of the previous candle or the first value of the already open candle. I asked the author to re-do it for candlesticks, but it didn't work.

In the end I remembered that in order to get the actual results for the input of the NS it was necessary to recompile the EA for each signal so the indicator was initialized correctly and gave the right results. From this, the current signal may have changed. This is what was really bothering me....

[Aleksey Vyazmikin]

Mihail Marchukajtes:

Here I absolutely agree with you. And the data it writes several values within a minute, it is not every tick, but still. Then it builds any TF from this data

So, what's the point of writing more than one OI inside a minute if you take data from the indicator - there will be no less TF. And so yes, I'm wrong, there is an exit from the loop, if successfully written, but still 0.1 second minimum interval. Do you build the model on ticks?

Mihail March ukajtes:

About the TMP file was my writing. I was making the indicator to be updated at the arrival of a new candle and it turned out kind of right on the real. But at a certain point it will either take the last value of the previous candle or the first value of the already open candle. I asked the author to change it for minute indicators, but the result has never been achieved.

For a real account it is enough to have this line in the indicator, why read the data from the file when it can be taken from the market?

BufOI[rates_total-1]=SymbolInfoDouble(Symbol(),SYMBOL_SESSION_INTEREST);

Does it render the history correctly in the visualizer?

[Aleksey Vyazmikin]

Mihail Marchukajtes:

In the end, I remembered that in order to get actual results for the input of the NS I had to recompile the Expert Advisor for each signal so the indicator initialized correctly and gave the right results. From this, the current signal may have changed. This is what was really bothering me....

This is strange. Maybe we should refuse from the indicator for the EA and directly read from the file into the structure and search for the value in the array structure?

I think that it is difficult to judge abstractly.

[Aleksey Vyazmikin]

elibrarius:

I also had a similar idea, but for now I'm busy with something else. I hope to experiment with it soon.
The disadvantage of it, too, is that the model will learn from 10 times less data. It seems to me that in this case the generalization ability will decrease.

Alternatively, you could learn on 9/10, and cut off on the remaining 1/10 of the sample.

[Mihail Marchukajtes]

Aleksey Vyazmikin:

You can also do it another way - learn on 9/10, and cut off on the remaining 1/10 of the sample.

Aleksey Vyazmikin:

So what is the point of writing several times OI within a minute, if you take data from the indicator - less TF will not be. And so yes, I'm wrong, there is an exit from the cycle, if successfully written, but still 0.1 second minimum interval. The model is built on ticks?

Well, for a real account this line is enough in the indicator - why read the data from the file, if they can be taken from the market?

Has it correctly drawn in the visualizer in the Strategy Tester?

Yes, but in case of a connection failure, there will be a hole. There is no check for the completeness of the history. I completely agree about the minutiae.

OM archive

http://fayloobmennik.cloud/7399404

[Aleksey Vyazmikin]

Mihail Marchukajtes:

Yes, but there will be a hole in the case of a broken connection. There is no check for completeness of history. And about the minutes, I totally agree.

OI archive.

http://fayloobmennik.cloud/7399404

So, how will the data be written to the file if the connection is interrupted?

[Renat Akhtyamov]

Roman:

Fundamental data has many indicators that give numerical values.
Even here on the site, the news calendar gives statistics on events.
Yes, I agree, in speech statements, numerical values are missing.
For this reason, such data should probably be classified at 0 1.
The main thing is to teach to distinguish between positive or negative speech ))
But here, this is also an idea, to think! ))

to use the foundation, there are some numbers missing

as far as I remember, I have not found some monetary aggregates, there are several

well, some of them are not posted to the public domain since about 2010

[Aleksey Vyazmikin]

Mihail Marchukajtes:

Yes, but there will be a hole in the case of a broken connection. There is no check for completeness of history. And about the minutes, I totally agree.

OI archive.

http://fayloobmennik.cloud/7399404

Do you agree that the OM should be taken as the previous record at the moment of the minute bar opening? For example, we take the opening at 10:00 a.m. as 23:49:55.

I think it is better to use the indicator on M1 and take all necessary information from the zero bar and make different comparisons in the Expert Advisor, taking into account the request for information from the indicator buffer with the required offset.

What is the third value in your file - the first is the date, the second is OM and the third is OI? I thought it was a delta, but it does not work.

I want to use it as a reference indicator, it works in real time, in real time.

I also want to use read from file in case of data interruption but market is closed and I haven't tested it.

[Mihail Marchukajtes]

Aleksey Vyazmikin:

So, how will the data be written to the file when the connection is broken?

If the connection is cut globally by the broker, then there is nothing you can do about it. The data saving advisor is on the UPU server and I will say that the data was written without losses.