Refreshing offline charts - background, performance and solution ideas
It seems you have some problems in your design or use case. Indicators are normally calculated once (especially if they "static"), and then they can be calculated in optimized manner - only on new missing bars - unless you drop previous calculations intentionally as you described. Just don't do this. Also it's not clear why do you use offline charts. If you mean ticks and not complete refresh of an offline chart, then it's performed in MT4 by WinAPI call (you may see how it's used here).
Hi Stanislav,
first many thanks for your quick answer. And what coincidence I just today downloaded your code and read your great article "MQL's OOP notes: Singleton, Command queue, and Template method patterns in simple order manager". Havent read all but I already learned a lot from it. Thank you for that too.
Now, what I use is regression channels in many forms and many different starting points. you can reuse historical values for that only to a certain extend.If the ending point of the regression channel changes with new bars you have to calculate the whole channel.
I already optimized the calculation in saving the intermediate sums in supporting arrays.
I managed to get down to 40% of the orginal elapsed time as the profiler showed me. But anyway the calculation has its cost.
You are right, on normal charts I dont have that problem. I have the problem on FXBlue Barchanger Charts which should do not much different than periodconverter to restrict the charts to certain time ranges (market times).
But maybe - as I write - that is the problem. As I understand in these two lines of the example you gave ...
if(EmulateOnLineChart && MT4InternalMsg == 0) MT4InternalMsg = RegisterWindowMessageA("MetaTrader4_Internal_Message"); if(hwnd != 0) if(PostMessageA(hwnd, WM_COMMAND, 0x822c, 0) == 0) hwnd = 0; if(hwnd != 0 && MT4InternalMsg != 0) PostMessageA(hwnd, MT4InternalMsg, 2, 1);
I understand that the second one with the
MT4InternalMsg
simulates the tick. But the first one
WM_COMMAND
is nothing else but calling refresh from the menu?
So I would have to reinvent the BarChanger basing on the periodconverter without the first line? (because FXBlue is not open source)
Am I right with these assumptions?
Greetings
JHawk
(hope my english is OK)
You are right, on normal charts I dont have that problem. I have the problem on FXBlue Barchanger Charts which should do not much different than periodconverter to restrict the charts to certain time ranges (market times).
But maybe - as I write - that is the problem. As I understand in these two lines of the example you gave ...
I understand that the second one with the
simulates the tick. But the first one
is nothing else but calling refresh from the menu?
So I would have to reinvent the BarChanger basing on the periodconverter without the first line? (because FXBlue is not open source)
Am I right with these assumptions?
Well, this works like a charm in the renko charts - without lagging. Unfortunately I don't know how FXBlue charts work. The only one weird assumption which I could make is that the renko code updates another window, while yours maybe attepts to refresh itself.
Anyway, I don't see the reason why to refresh entire chart. If your code "sees" new bars for calculation regression channel, just do it on updated position without refresh.
PS. Can you try your indicator on different offline charts, for example on this renko? This way you can deduce what part is the bottleneck.
Well, this works like a charm in the renko charts - without lagging. Unfortunately I don't know how FXBlue charts work. The only one weird assumption which I could make is that the renko code updates another window, while yours maybe attepts to refresh itself.
Anyway, I don't see the reason why to refresh entire chart. If your code "sees" new bars for calculation regression channel, just do it on updated position without refresh.
The BarChanger is an indicator that produces an offline chart like the RenkoLiveChart does. This indicator uses refresh like RenkoLiveChart on the offline chart.
You understand my situation if you imagine putting an indicator on the RenkoChart. That indicator then gets a refresh every new bar and you would have my problem.
Yet your answers helped me think in new directions :-)
I thank you very much for your support and good will!
I try to find a solution and maybe can give something back here when I successful.
Greetings
JHawk
As I understand it with offline charts is that there is no other way but refresh to get new data into the chart, that is loading data from the history files.
The
PostMessageA(hwnd, MT4InternalMsg, 2, 1);
is just the eventdriver for the tick event with the aim to have an OnTick()-Event in offline charts and that for a expert to be able to work.
---
All the problem I have here is that my CFD provider gives me 24 hour quotes. That means for Dow Jones for example 6.5 hours real data (Eastern Standard Time 9.30 am to 4 p.m.) and 17.5 hours of junk data (anything what my provider thinks it would be worth).
Thats what the BarChanger is for. it filters out those not-market-quotes and gives me a something that is near the dow jones.
If I could restrict the time range on my quote FOR A VIEW like a chart, that would be fantastic and probably not only for me.
BUT most programs - and think Metatrader too - if they allow such a restriction, its not for a view like a chart but the instrument like the Symbol in MT. Other have this possibility too.
But that is like wearing sunlasses in the night. (could be funny though ...)
That means: its not possible to track a trade after hours.
I never understand people using moving averages on this kind of 24 Hour quotes. Or gaps for example just do not exist.
Sorry. That was off topic. But it explains my need. And I know I am not alone because thats why FXBlue made this MT-Indicator and sells it as addon to brokers.
Now I wish a good and relaxed night.
JHawk
Well, then I can sugget 2 methods to try.
If you have a sources code which initiates the refresh, you could probably "mark" the artificial refreshes with a global flag (in global variables or something other) and in the indicators omit every new tick when the flag is on, then changing a counter for this flag from every indicator. When the counter becomes equal to a number of your processing indicators - remove the flag.
Another way is to not use IndicatorCounted in your indicators at all. Instead check for number of bars and datetime of the latest bar. When they changed for more than 1 single bar - make refresh. Otherwise process ticks only.
Because (1) the DLL ensures more strict error checking and (2) the signature
is much easier to understand and to use then memmove/memset. I don't want to put system calls and potentially dangerous pointer arythmetics into MQL code. Last but not least, maintenance is an important factor, too.
Of course you are free to run a loop in MQL and skip the DLL alltogether. This is just one of many possibilities to do it.
From the maitenance (and development) point of view the excessive DLL is a minus, not a plus IMHO. Also, many people here do not have means to edit cpp sources (when required) and compile a DLL. The signature of the function will remain the same if you implement it in pure MQL using system calls inside. No loops needed. But I see your point.
I'll post my solution of the problem below.
I have to digest it.... and it will surely help.
German: Vielen vielen Dank :-) Ich muss das erst mal verdauen. Aber es hilft massiv, selbst wenn ich nicht den gleichen Weg gehe. Aber ich verstehe das Ganze jetzt viel besser. Und es hilft auch zu wissen, dass ich mit meiner Anforderung nicht alleine bin.
Hi, JHawk! I have another solution for you. I think it's more effecient, simple, and based on native features of MetaTrader. Below I'm attaching a test indicator which demonstrates this approach by showing a line of close prices on an offline chart.
As I wrote before, you can distinguish real refresh from the simulated refresh by analyzing solely number of bars and timestamp of the last bar (since IndicatorCounted is not reliable after our "soft refresh" and should not be used). As for automatic shifting of bars with indicator data, which is missing on offline charts, MT4 provides a function SetIndexShift. Using this function you can dynamically adjust indexing of bars in the indicator, so there is no need to move data on every new bar.
The only drawback of the method is that you need to reserve a predefined number of spare bars which you can use to shift data. When this limit is reached, the indicator will throw the full refresh. For example, if you reserve 100 bars, you'll get full recalculation on every 100-th bar, but on other 99 bars you'll not have any overheads.
It works for me, but it would be interested to test it in your environment.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hello everyone :-)
Now this is my first question after I read so much now I still don't understand, how indicator data are retrieved and from where. And how static indiactor retrieval can be optimized.
Maybe what I want is a bit special. I explain:
- Most of my indicators are static in the sense as they will never be optimized or changed with parameters. As simple well know sample you may look at daily pivot point levels.
- I know many people use many symbols and also test different parameters on the used indicators. I trade maybe 2 symbols and I simply use very very little "moving" indicators like maybe the standard bollingers or Keltener Channel with EMA but thats it.
So what I stumbled over is the refreshing of offline charts which - as I learned now - calls oncalculate with prev_calculated = 0.
That had the effect, that I even had problems to simply change charts with the mouse click while my gaming notebook I use started cooking....
And simply to say: All this to recalculate my - static - indicators!
And even for online charts it is a performance problem, because most of my indicators are never looking back, but are always calculated with asSeries = false.
A note: I use and trade with MT4 because my broker uses it. But I tried to solve the problems also in MT5 in making a custom symbol.
So is there any chance in any Metatrader to have quotes and indicator data ready for all charts one uses in a global manner?
So I don't have to use any Copy mechanism like CopyBuffer from MT5 or iCustom from MT4 and mainly reach the effect that the data are not recalculated?
I am one step before saving my static data out as CSV or may as new symbol history to reach this. But that is also one step before leaving MT and using any other System like C or C#.
Take the daily pivot point levels as a sample. They never change if you look back. How can I calculate them ONCE and forever and have them - in the best case - always at hand in memory?
Many thanks in advance for all answers.
JHawk