Discussion of article "Decreasing Memory Consumption by Auxiliary Indicators"

[MetaQuotes]

New article Decreasing Memory Consumption by Auxiliary Indicators is published:

If an indicator uses values of many other indicators for its calculations, it consumes a lot of memory. The article describes several methods of decreasing the memory consumption when using auxiliary indicators. Saved memory allows increasing the number of simultaneously used currency pairs, indicators and strategies in the client terminal. It increases the reliability of trade portfolio. Such a simple care about technical resources of your computer can turn into money resources at your deposit.

Author: Andrew

[Mykola Demko]

The approach is understandable. But the relevance of the task is confusing.

While a 64-bit system supports huge amounts of RAM, the relevance of the task pales. Even 32-bit with its 3 Gb can quite pull such memory sizes as you are saving. After all, no matter how you look at it, memory grows linearly when loading new indicators, which means that 48MB more or less for modern computers is nothing.

Okay, let's assume that the task is relevant (I agree that there are people who care about it). But think about it, the task of memory saving is in direct conflict with the task of performance.

So you should pay attention to this point too.

As far as I can see from my vantage point, MQs are fighting for performance and that's where all efforts are concentrated.

If you need to save memory resources on non-drawn indicators, just transfer the indicator code to the Expert Advisor. Here you will have full control over the allocated memory. At the same time, you will be able to save on receiving the same type of data. Many indicators request the same data each for its own processing. Having received the data once, you will not need to address them in the following time.

Now let's combine both tasks speed and memory size. Most non-drawing indicators are engaged in the calculation of the last bar, this is indicated by the built-in mechanisms of blocking the recalculation of the entire array. It turns out that in the Expert Advisor it will be possible to allocate 154 memory cells (for example, for the calculation of the mask with the period of 153) with such memory saving the performance will be no worse than in a regular indicator.

But the possibility of parallelising calculations is lost in the Expert Advisor, and this is a direct blow below the belt to performance. Actually, indicators consume memory just because of parallelism. After all, each indicator works in its own thread and this thread must have its own instance of source data.

[[Deleted]]

I agree, performance is the main priority and there is no point in saving memory (unless of course you use more than 1Gb, which is unlikely).

[ds2]

Urain:

The approach is understandable. But the urgency of the task is confusing.

Well, actually, the idea to write this article came to my mind after one person, to whom I wrote a certain compound indicator, complained that this indicator was not installed on some pairs. When investigating, it turned out that there are very persistent indicators, and a lot of them do not fit into the terminal (that is, the problem was not in the pair, but in the fact that before this pair was opened a dozen more with the same persistent indicator). That indicator consumed 2 times more than the test indicator in the article.

Even 32-bit with its 3GB can quite pull such memory sizes

If you trade on a large number of pairs, it does not, that's the point.

By the way, the terminal cannot allocate more than 2 Gb of memory (in total: RAM + virtual memory). In my experiments it closed at this mark.

Of course, this problem should not exist in 64 bits.

OK, let's assume that the task is relevant (I agree that there are people who care about it). But think about it, the task of memory saving directly contradicts the task of performance.

Not always. In the article, here, most of the methods do not reduce performance.

If you need to save memory resources on non-drawable indicators, just move the indicator code to the Expert Advisor.

An Expert Advisor that requests only the last bar from indicators is a different class of programme than the one considered in the article. And it is not always possible/convenient to replace one with the other.

But in an Expert Advisor, the possibility of parallelising calculations is lost, and this is a direct blow to performance. Actually, indicators consume memory just because of parallelism. After all, each indicator works in its own thread and this thread must have its own instance of source data.
.

It is unlikely that each indicator on a common pair has its own instance of data - there is an interesting table in the article about parallel calculations, which can be found by the key phrase "2 indicators".

[Mykola Demko]

ds2:

The problem of indicators is that each of them works in its own thread and to work it must store all necessary data in the thread itself.

Data between threads is copied via CopyBuffer(). But here's the problem: you can get data from a thread, but you can't transfer them there. That's why you can't build mogostage indicators in which several instances of one indicator receive the same preprocessed data. But in this very plane can lie great opportunities for optimisation of calculations.

I think if MQ solve this problem, the work with indicators will become more convenient and flexible. Now the data can only be passed as external parameters and only at the start of the thread.

[muller]

Very usefull

so if i correctly understand the main idea stream, a question comes up:

why all the standard indicators found on MT5 installation are not designed as "class" ?

Then why master wizard dont care of such idea ?

[Rashid Umarov]

See Classes for working with Indicators

[Dark Ryd3r]

without limiting max bars from Tools> options, a quick way I use to reduce buffer space is

int maxbars =100;// last bars
limit = rates_total-rates_total+maxbars;