Single array calculation during optimisation - page 5

New comment
 
Yuri Evseenkov:
I do this. I run a single test. During this testing I write arrays into a file. Then I perform optimization in the same area but arrays are not recalculated at every bar or tick, they are read from the file. Optimization by opening prices on a weekly period is instantaneous. Without file operations it takes hours.
My situation is slightly different. I only need to fill the array once and use it for all passes. That is, for me, even with every pass, filling the array once in the inite is a lot.
 
-Aleks-:

I see, i.e. you can read the ready file with digits into an array (my case), then write it into a binary file, and then use it.

Well, if you need to read data of 10 arrays from the file, then you need to make a sorter, which will understand the end of the array by special digit and fill the next array... For it is not convenient to write the array size every time.

First, we write an int variable with the array size into the file, then an array, then another int variable with the size, then an array, etc.

In the same order we read: read variable with size, read array of that size, next variable, array, etc.

 
forexman77:
I have a slightly different situation. I only need to fill the array once and use it for all my passes. That is, for me, even with each pass, filling the array once in the inite is a lot.
So fill it once in a single test. Programs during testing and optimization must differ in file operations. During testing it is writing, while during optimization it is reading. Two codes must be written. What do you mean by initing to fill an array is a lot? Invoking an array from a file doesn't take any more time than, for example, opening an order or a multiplication operation with double.
 
Yuri Evseenkov:
So fill it once for a single test. Programs during testing and optimization must differ in file operations. During testing it is writing, while during optimization it is reading. Two codes must be written. What do you mean by initing to fill an array is a lot? Invoking an array from a file doesn't take any more time than, for example, opening an order or a multiplication operation with double.

A lot of why: I know the entire array in advance. And so the question arose as to how to save this array between passes. My understanding is that it's reset on the next pass, although I'm not exactly sure about that.

I don't agree that reading is faster in my case. The speed is faster, if I receive the same data through indicators, and significantly slows down, when I read this prepared data from a file without calculation of indicators. On a small timeframe with the number of lines of a thousand and in a large segment it is very expensive to read the file each time.

The idea was the following: to read the data and write them into an array and then, on all passes, only use this array, i.e. the array is exactly the same from the first pass to the last one.

 
forexman77:

A lot of why: I know the whole array in advance. And so the question arose as to how to save this array between passes. My understanding is that it's reset on the next pass, although I'm not exactly sure about that.

I don't agree that reading is faster in my case. It's faster, when I get the same data through indicators, and it gets slower, when I read this data from the file without calculation of indicators.

The idea was the following: to read data and put them into an array and then use this array for all passes, i.e. the array is exactly the same from the first pass to the last one.

Only practice will answer this question. Read the array on every pass. I have long been struggling with the same problem. I was optimizing for hours. Until Andrey Khatimlansky suggested I write into the file once and then read. Now I have to wait for the once-through test and then optimization is off the hook. File operations are well described in Kovalev's tutorial.
 
Dmitry Fedoseev:

First we write an int variable with the size of the array to the file, then an array, then another int variable with the size, then an array, etc.

Read in the same order: read variable with size, read array of that size, next variable, array, etc.

This is where I feel uneducated... How to write a variable int with the size of an array into a binary file? I.e. how would it be possible to identify this variable when reading from the file?
 
forexman77:

The idea was to read the data and put it into an array and then on all passes only use this array, i.e. the array is exactly the same from the first pass to the last one.

MT4 is really very slow in working with files, I don't know how it will be in MT5, especially with a binary file...

I solve this problem with a function that writes the whole array directly into the code, but it is not a universal solution, of course.

 

Here is an example of writing and reading an array

datetime T[]=
   {
   1262731020,1262735700,1262821920,1262903400,1262989740,1263247200,1263339060,1263420000,1263507480,1263595500,
   1265324700,1265407200,1265752980,1265926500,1265930820,1267657200,1267740300,1267826460,1268175840,1268346360,
   1270504920,1270684140,1270768500,1272924180,1273011720,1273097100,1273272240,1273528800,1273617180,1275344100,
   1275516000,1275602400,1275689160,1276034400,1276124580,1276208700,1276211640,1278027960,1278369780,1278373920,
   1278456660,1278540000,1278626400,1278712800,1280447880,1280527200,1280789220,1280959200,1281045720,1283292000,
   1283378400,1283812200,1285626300,1285887060,1286229600,1286316000,1286404740,1288133220,1288216860,1288305120,
   1288392420,1288648860,1288735200,1288741980,1288822080,1288994400,1290722460,1290809040,1291069320,1291329540,
   1293228420,1293577020,1293666300,1293746400,1295992800,1296079320,1296253680,1296514200,1296686940,1296770400,
   1298503260,1298592000,1298672280,1298931060,1300831920,1300917600,1301609160,1301696460,1303512420,1303768920
   };

datetime newT[];
int fileHandle;

/******************Expert initialization function*******************/
int OnInit()
{
   if((fileHandle = FileOpen("test", FILE_READ|FILE_BIN|FILE_COMMON)) != INVALID_HANDLE)// здесь поставь точку останова нажав клавиш F9
    {
     FileReadArray(fileHandle, newT);
      for(int i = 0; i < ArraySize(newT); i++)
       Print(i, " ", TimeToString(newT[i], TIME_DATE|TIME_SECONDS));
      FileClose(fileHandle);
    }
  else
   if(/*IsOptimization()*/IS_DEBUG_MODE && (fileHandle = FileOpen("test", FILE_WRITE|FILE_BIN|FILE_COMMON)) != INVALID_HANDLE)
    {
     FileWriteArray(fileHandle, T);
     FileClose(fileHandle);
    }
   return(INIT_SUCCEEDED);
}/*******************************************************************/

Run it twice in debug mode and see how long it takes to write and read the array. The first run will write the array to file, and the second will read it and start printing dates from the array. Then figure out the highlighted part, if IS_DEBUG_MODE should be removed, and if IsOptimization() should be inserted in the condition

 
-Aleks-:
This is where I feel uneducated... How to write an int variable of an array size into a binary file? I.e. how would it be possible to identify this variable when reading from the file?

There is a function, something like FileWriteInteger(). Identify by location, the variable must be at the beginning of the file, after reading it the pointer will move to its end, i.e. to the beginning of the array, after reading the array the pointer will be before the next variable...

An article on working with files is coming soon.

 
Dmitry Fedoseev:

There is a function, something like FileWriteInteger(). Identify by location, the variable must be at the beginning of the file, after reading it the pointer will move to its end, i.e. to the beginning of the array, after reading the array the pointer will be before the next variable...

There will be an article on working with files soon.

I understand, thank you.

But I would like to have a look at the code, so I'll wait for the article.

12345678
New comment
Reason: