Did Structs Change with the last MT5 Update? - page 10

New comment
 
Dominik Egert #:
Basically, you can use a transformer, copy from MqlStruct to your own. This way, you ensure the data format.

....
Basically, you can use a transformer, copy from MqlStruct to your own. This way, you ensure the data format.
I do

If I am not wrong, MT has internal arrays of prices, meaning all open prices are in a continuous memory, same for high, low and so on. So it is much more efficient to keep this around, because the CPU does this much better than jumping around in memory.
Buffered separately and send asynchronously permanent

Tick data is differently organized. It seems to me it is stored in structs of MqlTick. This can be seen when calling very frequently CopyTicks. Whenever new ticks come in, a certain amount of last ticks cannot be received via CopyTicks due to memory locking, and CopyTicks will give less ticks than in a call before, missing some of the most recent, previously run ceived ticks.
Buffered separately and send asynchronously permanent

An algorithm is required to keep all ticks up to date and have them be in order. To get away from the MqlTick structure a transformer into your own struct could be used.
I do

For ChartGetXXX, you need some clever code to do caching, because these functions can take forever to execute. Problem here is to know when to update your cache. This can be done in an asynchonus service, providing these values to your app.
Done, the weirdest thing ever, mentioned before - why do they need 5-50ms to copy all the chart-metrics data

Reading out position data is either polling very quickly and presenting the data, or your own implementation of calculating all the values. Latter option is faster, but much more work to code.
We do own implementation

All of this would now go into a shared memory area as result, with a well defined layout, such that your app can read any data at any time.
Thats where the update method comes in

For the RPC aspect, I suggest to use a message system with return result handling. Could be done with a fifo buffer for the requests, and a hash map for the return results. As one way to go. Could also be done with an RPC + CallBack method. Or with two Fifo buffers. Depending on how you want to use it.
Fifo, except the update variant 

For the case of ChartGet and ChartSet, it could be done asynchronous.
We do 

Until now, I cannot see any mutex or semaphore requirements. All can be done with atomic increments and ring buffers. (BTW, ring buffers can be dynamic in size without the need for modulo operators. Thus very efficient, just use pow2 in size.)
Ring buffers, as mentioned, permanent overflow risk with continiously updated volatile data like ticks. Better way out: Update method for such data as well as own response-buffers created by the clients themselves. 

The EA itself operates already asynchronously at almost every point and decides itself what is important next. No classical OnTimer, OnTick, OnChart, OnInit etc. Just one core which gets it all and distributes to the classses/objects based on internal priority. I´ve also implemented own event-handling, like in C#. Every object can fire an event and every object can subscribe to events of any object. Thats kinda the most efficient thing at all. An object needs to know when the chart-scale changed? It subscribes to the __Chart objects Event_ChartScaleChanged event. Just an example. Thats btw a thing which MQL should really provide itself. But ok. 
 
Doerk Hilger #:
Basically, you can use a transformer, copy from MqlStruct to your own. This way, you ensure the data format.
I do

If I am not wrong, MT has internal arrays of prices, meaning all open prices are in a continuous memory, same for high, low and so on. So it is much more efficient to keep this around, because the CPU does this much better than jumping around in memory.
Buffered separately and send asynchronously permanent

Tick data is differently organized. It seems to me it is stored in structs of MqlTick. This can be seen when calling very frequently CopyTicks. Whenever new ticks come in, a certain amount of last ticks cannot be received via CopyTicks due to memory locking, and CopyTicks will give less ticks than in a call before, missing some of the most recent, previously run ceived ticks.
Buffered separately and send asynchronously permanent

An algorithm is required to keep all ticks up to date and have them be in order. To get away from the MqlTick structure a transformer into your own struct could be used.
I do

For ChartGetXXX, you need some clever code to do caching, because these functions can take forever to execute. Problem here is to know when to update your cache. This can be done in an asynchonus service, providing these values to your app.
Done, the weirdest thing ever, mentioned before - why do they need 5-50ms to copy all the chart-metrics data

Reading out position data is either polling very quickly and presenting the data, or your own implementation of calculating all the values. Latter option is faster, but much more work to code.
We do own implementation

All of this would now go into a shared memory area as result, with a well defined layout, such that your app can read any data at any time.
Thats where the update method comes in

For the RPC aspect, I suggest to use a message system with return result handling. Could be done with a fifo buffer for the requests, and a hash map for the return results. As one way to go. Could also be done with an RPC + CallBack method. Or with two Fifo buffers. Depending on how you want to use it.
Fifo, except the update variant 

For the case of ChartGet and ChartSet, it could be done asynchronous.
We do 

Until now, I cannot see any mutex or semaphore requirements. All can be done with atomic increments and ring buffers. (BTW, ring buffers can be dynamic in size without the need for modulo operators. Thus very efficient, just use pow2 in size.)
Ring buffers, as mentioned, permanent overflow risk with continiously updated volatile data like ticks. Better way out: Update method for such data as well as own response-buffers created by the clients themselves. 

The EA itself operates already asynchronously at almost every point and decides itself what is important next. No classical OnTimer, OnTick, OnChart, OnInit etc. Just one core which gets it all and distributes to the classses/objects based on internal priority. I´ve also implemented own event-handling, like in C#. Every object can fire an event and every object can subscribe to events of any object. Thats kinda the most efficient thing at all. An object needs to know when the chart-scale changed? It subscribes to the __Chart objects Event_ChartScaleChanged event. Just an example. Thats btw a thing which MQL should really provide itself. But ok. 
Nothing to add. Sounds like you are all set and done. Everything working perfectly, I assume.
 
Dominik Egert #:
Nothing to add. Sounds like you are all set and done. Everything working perfectly, I assume.

Still only:

>>
1. The restriction within OnTimer(), while is not that big deal, since it mainly affects the ping-time. Once its reached, further messages can be handled instantly after this first access - <>5000 sync messages average. Nonetheless: MQL service, if it can provide something better than OnTimer(), maybe simply while (true) { if (server.Listen()) ....  Sleep(0); } that issue would be already be solved. 

2. Still the thing with the transferred data. At the moment it also works without problem, thats why I pointed out I´d be interested to see sth which is more safe in view of possible future changes within MQL.

<<

If someone can show me a simple example of how to define/setup a MQL service, I would appreciate it a lot. And, can I do this there?

while (!IsStopped())
        {
        if (_server.Listen())
                {
		// Read message ...
                // do stuff with it ... 
		// return response ...
                }
        ::Sleep(0);
        }
 
Doerk Hilger # :
If someone can show me a simple example of how to define/setup a MQL service, I would appreciate it a lot. And, can I do this there?
What is your desired delay ?
What is the minimum delay taken into account in ontimer?
 
Gerard William G J B M Dinh Sy #:
What is your desired delay ?
What is the minimum delay taken into account in ontimer?
Do I need to use a timer?
In case Im forced to do so and in case a service has its own thread, then 1ms. 
 
Doerk Hilger # :
Do I need to use a timer?
In case Im forced to do so and in case a service has its own thread, then 1ms. 
SO
It seems to me that you have a shared space where the threads are consolidated (semaphore)
Put a bool in place, why not one with a modulo so as not to overload the process, 1 every 10....

the 1 ms ontimer is no longer a trigger, but a limiter of 1 ms max delay, a security which allows to reset the flag, if it was not updated in the semaphore
 
Gerard William G J B M Dinh Sy #:
SO
It seems to me that you have a shared space where the threads are consolidated (semaphore)
Put a bool in place, why not one with a modulo so as not to overload the process, 1 every 10....

the 1 ms ontimer is no longer a trigger, but a limiter of 1 ms max delay, a security which allows to reset the flag, if it was not updated in the semaphore

I try to decrypt your message :D 

So you have in mind, that I use OnTimer() within the service to rather take care of that a semaphore is not blocked instead of using the OnTimer() to trigger the servers action?

And with semaphore you don´t mean a system-semaphore, rather a bool var in MQL?

---

Just to explain: The MQL side doesn´t have to take care of any blockings, thats all done in the DLL-bridge. What I have to do in MQL is kind of like:


while(!IsStopped())
{
        if (!DLL.Listen(server_id, _out message, _out need_response) return; // Nothing to do

        // Servers mutex is already released here in any case
        if (need_response)
        {
                result=ComputeResultForMessage(message);
                DLL.ReturnResult(server_id, result);
        }
        else 
        {
                ProcessMessageWhichNeedsNoResult(message);
        }
}

No need to worry bout any semaphores or mutexes or deadlocks. 
So what exactly did you have in mind?

 

I must not have understood the problem
It seemed to me that you wanted to process a volume of data that changes very often, more than 5,000 data every 1 ms.


That this processing could not be done at each change (1 ms / 5,000 is too short) but at least every 1 ms

 
Gerard William G J B M Dinh Sy #:

I must not have understood the problem
It seemed to me that you wanted to process a volume of data that changes very often, more than 5,000 data every 1 ms.


That this processing could not be done at each change (1 ms / 5,000 is too short) but at least every 1 ms

Not necessarily that much of data. I would just like to get rid of the 25ms timer which results in 12,5ms ping time. 
 
Doerk Hilger #:

Still only:

>>
1. The restriction within OnTimer(), while is not that big deal, since it mainly affects the ping-time. Once its reached, further messages can be handled instantly after this first access - <>5000 sync messages average. Nonetheless: MQL service, if it can provide something better than OnTimer(), maybe simply while (true) { if (server.Listen()) ....  Sleep(0); } that issue would be already be solved. 

2. Still the thing with the transferred data. At the moment it also works without problem, thats why I pointed out I´d be interested to see sth which is more safe in view of possible future changes within MQL.

<<

If someone can show me a simple example of how to define/setup a MQL service, I would appreciate it a lot. And, can I do this there?

Yes, its that simple.
1...345678910
New comment