MT5 Tester local agents/cores overwrites each other on CSV file when I use optimization.

New comment
 

Hi guys, im looking for a way around or an alternative to do things.

Basically I use FileOpen, FileWrite, etc on OnTester to write information during optimization on CSV File. I notice while back that somehow, the number of Optimization Passes was always missing 1 or 2, or there is a incomplete row of data in the CSV file. I debug and found that the 4 cores/agents was simultaenously overwriting on the CSV file when they are simultaneously optimizing. I work this around crudely by disabling 3 cores and only 1 core is working. This somehow solves my problem of simultaneously writing on the CSV file, the CSV file was ok since then. But now Im working on more Optimization Passes, around 40k passes and I really need the other cores.

Im looking maybe that my structure of coding is wrong or maybe I can command the 4 cores to write on different CSV Files so they wont interrupt each other? Is there an efficient way to do it?

Im using Mac, MT5 on Crossover if this helps. 

Thanks!

Francis

 

There are several solutions. The easier is as you said, write to a different file. You will have to re-assemble all after.

Or you need to use a mutex, though it could possibly slow down the optimization, depending of what your doing as optimization.

Or

 

Ah yes. I also read Frames a while back when I was searching for topics. I tried reading the MT5 docs and implement it but I struggled.

My understanding when I tried programming it is that the tester will execute it when you gave it a frame. For me its the same as executing a command in OnTester with FileWrite. The problem will still persist when Local Cores start working simultaneously on optimization and writes on one CSV file, they will interrupt or overwrite each other. I haven't understand enough of frames to see if i could somehow put the cores "in one line" so that they would write one at a time on the csv. But I think it cant do that as it is the Tester who is running the different instances of the code simulataneously.

Maybe Im missing something on frames?

my temporary solution for now was all my 4 cores is writing to different csv files. and i have to manually combine them.

Maybe I'll try reading on mutex.

Thanks Alain!

 
franzzzz:

Ah yes. I also read Frames a while back when I was searching for topics. I tried reading the MT5 docs and implement it but I struggled.

My understanding when I tried programming it is that the tester will execute it when you gave it a frame. For me its the same as executing a command in OnTester with FileWrite. The problem will still persist when Local Cores start working simultaneously on optimization and writes on one CSV file, they will interrupt or overwrite each other. I haven't understand enough of frames to see if i could somehow put the cores "in one line" so that they would write one at a time on the csv. But I think it cant do that as it is the Tester who is running the different instances of the code simulataneously.

Maybe Im missing something on frames?

my temporary solution for now was all my 4 cores is writing to different csv files. and i have to manually combine them.

Maybe I'll try reading on mutex.

Thanks Alain!

Yes you are missing something, it's completely different than what you are doing with your file. Search on the forum, I myself provided code to show how it works.

The simplest way to do a mutex with a file is to play with the access rights (don't allow share write) and loop until you get access to write.

New comment
Reason: