How do I pass a timeseries array to a DLL? - page 2

[JC]

Ruptor:

[...] So how can this be more complicated or slower than copy th whole array every time.

... Because, in order to do calculations on the array - which must be the whole point of having a DLL - you must at the very minimum be having to do modulus operations on an array index rather than simply looping from array position 0 to array position n. And there are several classes of calculation where the overheads of the cyclical array become more complex.

The overhead of an array re-copy happens at most once per minute; the overheads of modifying the calculation to handle a cyclical buffer are potentially incurred on every single tick.

One copy of the array is 1000 reads 1000 writes and 1000 checks plus all the other house keeping the system does

I bow to your expertise in "low level embedded coding with a lot of assembler", but I'd imagine that an array copy in MQL very rapidly gets turned into a call to RtlMoveMemory(), which in turn maps pretty directly onto the optimal processor instruction for moving memory. If MQL is copying a 1000 item array by doing a thousand separate reads and a thousand separate writes, then that's... well, crazy.

how can this be more complicated or slower than copy th whole array every time

I'm not suggesting doing it "every time". I'm suggesting doing it at most once per minute, if the EA is on an M1 chart, or hourly if it's on an H1 chart etc. If you look at mjbrady's example, he's talking about the last 10 close prices. Even if the array copy is slower than using a cyclical buffer, and even if it's 1000 items rather than 10 which are being moved, then this is still one of those circumstances where I'd personally trade off simplicity of code against what I strongly suspect would be absolutely trivial performance gains. I'd obviously change this opinion if it became apparent that the array copy was a dangerous bottleneck.

EDIT: I've just checked this. My very average hardware can do something over 50,000 copies per second of the last 10,000 close prices into an array. The cost of the array copy versus a cyclical buffer is therefore a worst case of about 0.00002 seconds per minute, assuming that the EA runs on an M1 chart and there are absolutely no overheads whatsoever involved in having to circle round the array to do calculations rather than just looping from 0 to array-length. If that sort of performance penalty is a problem for you, then MT4 is absolutely the wrong trading platform for you.

[Paul]

Hi jjc

One should use pointers to index arrays for efficiency. There is no difference in code complexity between cycling a pointer from 0 to N campared to 6 to 5.

It is crazy to copy arrays all over the place that's why I suggested the cyclic buffer that only requires 1 copy. I see the initial array size was 10 but I took that as an arbitrary number for the example, like you say there is no problem with such a small array. If it was my code I would start with the most efficient method so I didn't have to do it all over again when the requirement changed, as they do, and larger arrays were needed.

[JC]

Ruptor:

 There is no difference in code complexity between cycling a pointer from 0 to N campared to 6 to 5.

Not necessarily true. Taking one of the possible examples, let's say that part of the calculation carried out by the DLL can only reasonably be done by modifying the array data in the course of producing the final calculation. In this scenario the DLL therefore has to take a copy of the relevant part of the array so as not to affect the data being held by the EA. If the array is non-cyclical, this is always a simple, single block transfer of memory. If the array is cyclical, then it potentially requires two copy operations rather than one, from each end of the array, in order to generate the subset of the array to work on.

In this scenario, an array copy every M1/M15/H1 etc plus a simple memory copy every tick is being replaced with zero operations on an occasional basis, but an increase in the average number of (smaller) copies every tick. It is far from clear to me either that this is going to have less "code complexity" or that it is definitely faster.

We clearly differ in opinion, but I personally regard this as a scenario where code complexity - with unavoidable implications for number of bugs etc - is being introduced to achieve a demonstrably trivial (and debatable) performance gain. If you were right that "one copy of the array is 1000 reads 1000 writes and 1000 checks" then I'd agree that a repeated array copy is a hostage to fortune. But this just isn't correct.

[Paul]

jjc:

We clearly differ in opinion, but I personally regard this as a scenario where code complexity - with unavoidable implications for number of bugs etc - is being introduced to achieve a demonstrably trivial (and debatable) performance gain. If you were right that "one copy of the array is 1000 reads 1000 writes and 1000 checks" then I'd agree that a repeated array copy is a hostage to fortune.

But this just isn't correct.

In your opinion.

Therefore we should agree to disagree.