Synch CopyBuffer in EA using iCustom - page 2

New comment
 
antony23 #:

 The things are these:

- or copybuffer is thread safe, but that would mean that also writing data in buffers, inside Oncalculate, should be done using copybuffer and not setting single index value in a loop. And i didn't read this anywhere in documentation.

- or the deisgn is done in the way that OnCalulate and OnTicks, on the same chart, are executing sequentially, even if in its own thread. Using separate threads for EAs could be usefull to execute, instead, more OnTick events on the same symbol, from different EAs (and so from different charts) simultenously.  

I suspect the second is the most likely, otherwise this simultaneous data access problem should have been very well documented, but instead I haven't found anything about it, and I've searched really hard.


Anyway i'll wait for your news.

Thanks.

Dominik Egert #:
Though, no fundamental docs on this, CopyBuffer is for sure not thread safe, else this would have been actually documented.

EAs and OnChart thread is not synchronized, there is no sequential processing implemented.

The lack of documentation leads me to assume, my initial assumption is correct. There is zero thread safety in this process.

You (and I am quite confident this will be the outcome) need to figure a way to ensure you get what you actually are looking for in a different way.

Probably the only real approach is to use global variables from the terminal to achieve a locking mechanism to prevent false "in the middle" processing/reading of data.

Or, transfer your calculations to the EA.

That's a lot of assumptions. I prefer data and facts.

 

At least when called from an EA with CopyBuffer() running from OnTick(), it's thread safe unless I missed something.

The EA running on EURUSD/H1 is launching the indicator with iCustom on EURCAD/M1, with a 1 second delay between buffer[0] and buffer[2] update. I also run the indicator on EURUSD, just to get a clue when there is a new tick on EURUSD.

2025.11.16 18:18:24.307    ThreadIndicator (EURUSD,H1)    New tick 2025.11.17 01:18:24.084 [3] : 3892 microseconds [3809] (1000/1000) -> 1079251.446388
2025.11.16 18:18:24.547    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:23.143 [6] : 994704 microseconds [988215] (260000/1000) -> 332016843.824758
2025.11.16 18:18:24.547    ThreadEA (EURUSD,H1)    OnTick 2025.11.17 01:18:24.084: [0]=6 [1]=332016843.824758 [2]=6 
2025.11.16 18:18:25.316    ThreadIndicator (EURUSD,H1)    New tick 2025.11.17 01:18:25.091 [4] : 3875 microseconds [3826] (1000/1000) -> 1079251.446388
2025.11.16 18:18:25.517    ThreadIndicator (EURUSD,H1)    New tick 2025.11.17 01:18:25.293 [5] : 3861 microseconds [3806] (1000/1000) -> 1079251.446388
2025.11.16 18:18:25.535    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:24.148 [7] : 993825 microseconds [988554] (260000/1000) -> 332016843.824758
2025.11.16 18:18:26.525    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:25.153 [8] : 993260 microseconds [989303] (260000/1000) -> 332016843.824758
2025.11.16 18:18:27.513    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:25.908 [9] : 992673 microseconds [987973] (260000/1000) -> 332016843.824758
2025.11.16 18:18:28.501    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:25.908 [10] : 992206 microseconds [988005] (260000/1000) -> 332016843.824758
2025.11.16 18:18:28.501    ThreadEA (EURUSD,H1)    OnTick 2025.11.17 01:18:25.091: [0]=10 [1]=332016843.824758 [2]=10 
2025.11.16 18:18:29.489    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:25.908 [11] : 991832 microseconds [988091] (260000/1000) -> 332016844.842963
2025.11.16 18:18:30.477    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:25.908 [12] : 991515 microseconds [988037] (260000/1000) -> 332016844.842963
2025.11.16 18:18:31.465    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:25.908 [13] : 991264 microseconds [988252] (260000/1000) -> 332016844.842963
2025.11.16 18:18:32.467    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:25.908 [14] : 992033 microseconds [1002027] (260000/1000) -> 332016846.881251
2025.11.16 18:18:32.712    ThreadIndicator (EURUSD,H1)    New tick 2025.11.17 01:18:32.483 [6] : 3852 microseconds [3806] (1000/1000) -> 1079251.446388
2025.11.16 18:18:33.467    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:25.908 [15] : 992586 microseconds [1000321] (260000/1000) -> 332016846.881251
2025.11.16 18:18:34.461    ThreadIndicator (EURCAD,M1)    Update 2025.11.17 01:18:32.386 [16] : 992627 microseconds [993245] (260000/1000) -> 332016846.881251
2025.11.16 18:18:34.461    ThreadEA (EURUSD,H1)    OnTick 2025.11.17 01:18:32.483: [0]=16 [1]=332016846.881251 [2]=16 

It's thread safe because EA Ontick on EURUSD is triggered only after the iCustom indicator is updated.

I let, who want to do it, check other event handler like OnTimer or other working conditions.

Files:
ThreadEA.mq5  5 kb
ThreadIndicator.mq5  4 kb
 

Good test, it is like the terminal, knowing we have iCustom inside EA, synchronize OnTick and OnCalculate calls. 

Anyway, aside tests, it would be nice MQ would confirm it and maybe explain exaclty how it works. This should be documented.

Did you conctact them to have some explanations?

 

MT5 is built with efficiency in mind (as MQ envision it), so they do not provide bult-in syncing between threads, and this task is completely on the MQL5-developer responsibility. Specifically, the situation of OnCalculate and OnTick processing different ticks can occur if the timing (calculation of corresponding event handler) is larger than the other's. For example, if tick A is already processed by both indicator and EA, then arrives tick B wich launches OnCalculate and it's still being processed, any next tick C (if it arrive fast), will be handed to EA, but not to the indicator, I suppose.

I know there is the event queue of the chart, and event queues for every MQL5-program (running on it), but the algorithm how the events are dispatched from one queue to all the others are not disclosed.

 
Stanislav Korotky #:

MT5 is built with efficiency in mind (as MQ envision it), so they do not provide bult-in syncing between threads, and this task is completely on the MQL5-developer responsibility. Specifically, the situation of OnCalculate and OnTick processing different ticks can occur if the timing (calculation of corresponding event handler) is larger than the other's. For example, if tick A is already processed by both indicator and EA, then arrives tick B wich launches OnCalculate and it's still being processed, any next tick C (if it arrive fast), will be handed to EA, but not to the indicator, I suppose.

I know there is the event queue of the chart, and event queues for every MQL5-program (running on it), but the algorithm how the events are dispatched from one queue to all the others are not disclosed.


Based on what Alain posted, I guess iCustom from an EA is run in the EAs thread, and placed before the OnTick.

Not sure though. Thinking about how to measure this.

We had a discussion, some time back, where we concluded, OnCalculate is called for every tick received. - But it is unclear how this all gets "dispatched" to the different threads available.


 
Dominik Egert #:

Based on what Alain posted, I guess iCustom from an EA is run in the EAs thread, and placed before the OnTick.

Not sure though. Thinking about how to measure this.

We had a discussion, some time back, where we concluded, OnCalculate is called for every tick received. - But it is unclear how this all gets "dispatched" to the different threads available.


For developers, in any case, this is very, very crucial information to know. We can't develop an EA without understanding what exactly happen when more programs access to same memory data. Even if Alain's test code are working in thread-safe manner in these tests, we need to be sure it will always work this way without surprises. EAs works with money, it's not joke.

Are there MQ developers in this forum? Could someone of them give a look to this thread?

 
antony23 #:

For developers, in any case, this is very, very crucial information to know. We can't develop an EA without understanding what exactly happen when more programs access to same memory data. Even if Alain's test code are working in thread-safe manner in these tests, we need to be sure it will always work this way without surprises. EAs works with money, it's not joke.

Are there MQ developers in this forum? Could someone of them give a look to this thread?

Already asked. Be patient. 

And I can't guarantee there will be an answer, they are very busy.

 
Dominik Egert #:

Based on what Alain posted, I guess iCustom from an EA is run in the EAs thread, and placed before the OnTick.

Not sure though. Thinking about how to measure this.

We had a discussion, some time back, where we concluded, OnCalculate is called for every tick received. - But it is unclear how this all gets "dispatched" to the different threads available.

I don't think indicator could ever run in the EA thread. MQ cleary stated that they run in different threads. Most likely, the effect you see in the logs, is due to the queues processing.

As far as I can tell, according to other people tests in the past, indicators may also skip OnCalculate events if they process current tick too long, otherwise a single indicator could hang entire terminal, for example, during news releases.

 

Seems like the EA thread is being blocked by the indicator.

Files:
ThreadEA.mq5  7 kb
ThreadIndicator.mq5  3 kb
 
Stanislav Korotky #:

I don't think indicator could ever run in the EA thread. MQ cleary stated that they run in different threads. Most likely, the effect you see in the logs, is due to the queues processing.

As far as I can tell, according to other people tests in the past, indicators may also skip OnCalculate events if they process current tick too long, otherwise a single indicator could hang entire terminal, for example, during news releases.

Well, cant say anything on the threads being used for the indicators. - But I can say, OnCalculate is being called for every tick received. - We had prooven that. - I dont know when calls will be skipped, might be, if the event queue is full, but as it seems, its a long way until that happens.


https://www.mql5.com/en/forum/454153

Is it possible to work with live ticks in MQL5 indicators?
Is it possible to work with live ticks in MQL5 indicators?
  • 2023.09.16
  • www.mql5.com
We know that ontick() will not do anything in mt5 indicator scripts, and mqltick is a structure based on tick retrieval (historic ticks, not live). There is nothing stopping you from processing real-time or historical tick data in an mql5 indicator
123
New comment