Timeseries array indexing using loops in EA without using indicators and oncalculate function - is it possible ?
Hi! I've been struggling with an assignment for a while now. What if you don't want to use oncalculate function and indicators but build EA without using indicators.
how can indexed time series as systematically rereadable arrays be created in this case ?
I know that some pre-defined functions such as iHigh, ILow and MqlRates+copyrates etc. work nicely in the onTick function by indexing the elements of arrays properly based on bars. But as soon as I define the array myself (it doesn't matter if it's static or dynamic using the arraysetasseries function or not , etc.), I can no longer set the elements properly by indexing arrays controlled by bar(time) or indexed iteration cycles.
How to create arrays that would be time series and series based by time and indexing without predefined functions?
I have puzzled over this so much. currenttime=! time, which only allows the loop to iterate at the beginning of the new bar with indexing ++, has not allowed me to create a correct order using the for loop i++ historical informative real-time and continnually forward method. do we really have to build Cclass ourselves or can we somehow manage without it? Or is it even possible to do this using mql ?
I have understood that this task is not at all as simple as it seems, but there are certainly many here who know more about it. Those who know the topic or have the same problem themselves, please answer. I don't want everything to be made too easy for me, but maybe some study material that you can recommend or general advice would be helpful. I have read a lot of this forum but have not received any clear answer on this topic. There has been silence or that it is thought that it has to be done by using the indicator, but the topic itself has been avoided and there is no answers so far. I really hope that this time we can get clarity on this question !
thank you in advance anyway!
Hi! I've been struggling with an assignment for a while now. What if you don't want to use oncalculate function and indicators but build EA without using indicators.
...
Why ?
You don't seem to be experimented coder and you are searching for trouble trying things which have no benefits.
If you insist to do it this way, please show your code to get coding help.
Why ?
You don't seem to be experimented coder and you are searching for trouble trying things which have no benefits.
If you insist to do it this way, please show your code to get coding help.
I once started this topic but then I lacked a bit of experience and I couldn't continue.
Now I'm back on topic. Isn't it faster to write custom indicators directly in EA.
It is certainly so much more convenient to transmit different signals without using icustom and buffers.
I'm just looking into whether there is any noticeable advantage in terms of performance than calling CopyRates for each bar and asking for information of, for example, 3000 bars?
(i do not need never 50000 or 10000 bars for analyzis )
If you don't need all the information, you can use iClose, iTime... etc. call struct instead.
Anyway, I decided to give it a quick try. Initially quite simple and raw, without using classes or anything difficult...
to my surprise, I got everything working very easily. I quickly integrated own custom zig-zag indicator, I ran several pieces of them side by side with different sensitivities.
I chose zig-zag because it is one of the most difficult algorithm which shows every mistake quickly...
when printing the data from the buffers everything seemed correct and I also added a line plot for verification.
It's been working almost 24 hours now and I don't see any worries. Easily can get all the timeseries data and manipulate it in any way you need...
I would still like to find out why it is recommended to stay away from such an approach and to make indicator calculations and drawings in the indicator before I start writing it in OOP?
I'll add a quick example as well:
#define RANGE 2000 MqlRates copied_rates[]; datetime last_tick = D'1971.01.01 00:00'; int debugging_counter = 0; int rates_total = RANGE; int prev_calculated = rates_total; void OnTick() { bool process_iteration = false; if (last_tick != iTime(Symbol(), PERIOD_M1, 0)){ process_iteration = true; last_tick = iTime(Symbol(), PERIOD_M1, 0);} if (process_iteration) { // you can do something before prev_calculated == rates_total prev_calculated--; // do what you need like usual if prev_calculated == rates_total-1 Print("rates_total ", rates_total ," prev_calculated ", prev_calculated); debugging_counter++; Print("debugging_counter ",debugging_counter); /* use struct and CopyRates if you need all data or simple function like iClose, iTime if not struct MqlRates { datetime time; // Period start time double open; // Open price double high; // The highest price of the period double low; // The lowest price of the period double close; // Close price long tick_volume; // Tick volume int spread; // Spread long real_volume; // Trade volume }; */ int copied=CopyRates(NULL,0,1,RANGE,copied_rates); if(copied<=0) Print("Error copying price data ",GetLastError()); else Print("Copied ",ArraySize(copied_rates)," bars"); if (ArraySize(copied_rates)==RANGE) Print("All ",RANGE," RANGE bars are copied now"); // build a custom indicator prev_calculated++; // set prev_calculated == rates_total Print("rates_total ", rates_total ," prev_calculated ", prev_calculated); // in addition, you can do anything else you need } }
I'm just looking into whether there is any noticeable advantage in terms of performance than calling CopyRates for each bar and asking for information of, for example, 3000 bars?
(i do not need never 50000 or 10000 bars for analyzis )
Let's say, initially, you have rates_total = 10k bars in the history and you need to analyze the last 3k bars for every new bar:
1) after the first 3k bars, you look back and execute your analysis;
2) repeat the analysis for the next 7k bars (for a total of 21k bars analyzed in the history).
Now, a new bar is formed:
3) you do your analysis on the previously 3k bars;
4) repeat 3) for every new bar.
If you don't need to analyze the entire history you can simply tell your indicator to start at (rates_total - 3k bars) and then continue for every new bar.
You could do this with the expert advisor but you will not benefit from the buffers. EAs are primarily intended to manage market trades.
Let's say, initially, you have rates_total = 10k bars in the history and you need to analyze the last 3k bars for every new bar:
1) after the first 3k bars, you look back and execute your analysis;
2) repeat the analysis for the next 7k bars (for a total of 21k bars analyzed in the history).
Now, a new bar is formed:
3) you do your analysis on the previously 3k bars;
4) repeat 3) for every new bar.
If you don't need to analyze the entire history you can simply tell your indicator to start at (rates_total - 3k bars) and then continue for every new bar.
You could do this with the expert advisor but you will not benefit from the buffers. EAs are primarily intended to manage market trades.
thank you very much for answering! really appreciate it !
right detailed information is very helpful indeed to design the systems properly.
Do I understand correctly that the timeseries buffers associated with the onCalculate function are the same as those in MqlRates structure ,
but if you use these buffets through the onCalculate function, they behave differently? It seems somehow not logical ,but it is probably due to my insufficient understanding.
is spoken using onCalculate we add new index and final received element to the []. further we don´t have to go through previous elements ?
for(int i = 2000; i < rates_total-1; i++)
if we loop through the last 2000 bar close[] array using onCalculate. In this case we only once in first iteration go the last 2000 index elements. but when a new bar appears and the iteration repeats,
we still have to go through the last 2000 index elements again, assuming that the size of our destination array is also 2000. Cause in this case ,we still have to remove the last element and add the newest element to our array ?
On the other hand, when we using MqlRates struct or iClose for example inside the onTick and call it once per bar. It should also be possible to copy the necessary history and then further copy only the last bar elements and add index when we use dynamic array ? Surely pro masters here know these things completely and i hope get an answer :) !
cheers !
thank you very much for answering! really appreciate it !
right detailed information is very helpful indeed to design the systems properly.
Do I understand correctly that the timeseries buffers associated with the onCalculate function are the same as those in MqlRates structure ,
but if you use these buffets through the onCalculate function, they behave differently? It seems somehow not logical ,but it is probably due to my insufficient understanding.
is spoken using onCalculate we add new index and final received element to the []. further we don´t have to go through previous elements ?
if we loop through the last 2000 bar close[] array using onCalculate. In this case we only once in first iteration go the last 2000 index elements. but when a new bar appears and the iteration repeats,
we still have to go through the last 2000 index elements again, assuming that the size of our destination array is also 2000. Cause in this case ,we still have to remove the last element and add the newest element to our array ?
On the other hand, when we using MqlRates struct or iClose for example inside the onTick and call it once per bar. It should also be possible to copy the necessary history and then further copy only the last bar elements and add index when we use dynamic array ? Surely pro masters here know these things completely and i hope get an answer :) !
cheers !
double c[]; int initial_size = 20; int counter = 0; if (ArraySize(c)<initial_size) { ArrayResize(c,initial_size); for (int i = initial_size - 1; i > 0; i--) c[i] = iClose(NULL,0, initial_size -1 -i); counter = initial_size; } else { counter++; ArrayResize(c, counter); c[counter-1] = iClose(NULL, 0, 1); } for (int i = 0; i < ArraySize(c); i++) { Print("close[] index ", i, " value is ", c[i]); }
to continue the topic I did a small test and it turned out what I thought. We get elements in exactly the same non series order in our custom close[] array.
We choose the required amount of data that we want process. We populate the array ascending order with the necessary data and then forward we add the new elements to the buffer ascending order when new bar forms and new tick arrives. The newest elements is at the back so that we don´t have to go through the entire array. In exactly the similar way as the indicator timeseries buffers provide us with data.
Using the data wisely, we need it represented as a timeseries, in which case we cannot use the ArrayResize reserve buffers or a statically made larger array for faster processing until it is not a series. In this case, after each new bar we have to increase the array to add a new element at the end. Here of course the question arises whether it doesn´t make sense to immediately use array as series and the selected static area. In this case, each element of the array must be shifted by one on but the array size remains the same and there is no need to use resize.
Firstly it seems that it is reasonable to use the array not a series method shown above and add an element at the end, especially for large data sets? Gurus who has measured which way is the resource efficient solution could explain.
Cheers !
Firstly it seems that it is reasonable to use the array not a series method shown above and add an element at the end, especially for large data sets?
These are functions for working with timeseries and indicators. A timeseries differs from the usual data array by its reverse ordering - elements of timeseries are indexed from the end of an array to its begin (from the most recent data to the oldest ones). To copy the time-series values and indicator data, it's recommended to use dynamic arrays only, because copying functions are designed to allocate the necessary size of arrays that receive values.
There is an important exception to this rule: if timeseries and indicator values need to be copied often, for example at each call of OnTick() in Expert Advisors or at each call of OnCalculate() in indicators, in this case one should better use statically distributed arrays, because operations of memory allocation for dynamic arrays require additional time, and this will have effect during testing and optimization.
- www.mql5.com
A timeseries differs from the usual data array by its reverse ordering - elements of timeseries are indexed from the end of an array to its begin (from the most recent data to the oldest ones).
Thanks for answering but now I´m afraid you missed the whole point. You are probably mix up series and timeseries. Both ways they are a timeseries arrays in mql5 indicator buffers(in reversed order or not). In mathematics, a time series is a sequence of data points indexed in time. In terms of mql5, is it in series (reversed order) or non series (as it is collected in mql5 timeseries buffers by default). there are also valid explanations in the texts you copied , but they are still very superficial in the documentation.
The question was rather what is the advantage of using indicators buffers instead of building the buffers directly inside EA (As I did in the previous code example). There was talk about different buffer filling methods, so which one is actually the best. At the same time, I understand that it is relative and depends on the amount of data being processed.
maybe , to summarize the whole question, it would be more correct to ask :
how much recourse is consumed by calling the ArrayResize function without prior reservation to make the array larger by one element VS. how many elements could be shifted, using same amount recourses in a static array.
Here we reach a very low level and documentation does not reflect this. But such measurements have been made... If we use MQL5 to build larger and more complicated data analysis algorithms, these questions are very competent, because it would be good to use this speed to the maximum, otherwise it would be easier to built them in pinescript or python.
There is no performance advantages to build all in an EA. It's an urban legend.
If the calculations/resources needed are low, like most of the standard indicators, the difference is unnoticeable.
If the calculations/resources are heavy, with indicator(s) you will benefit from multi-threading.
There is no performance advantages to build all in an EA. It's an urban legend.
If the calculations/resources needed are low, like most of the standard indicators, the difference is unnoticeable.
If the calculations/resources are heavy, with indicator(s) you will benefit from multi-threading.
thanks for answering, really appreciate it !
I´m not trying to claim that EA is significantly faster (the language is the same and the data processing capabilities are also same). Rather my logic says that they are equal in performance if they are properly built because in this case the demand for the iCustom resource is marginal. But building incorrectly and sending buffers between the EA and the indicator, as is often done, then the result is greatly in EA´s favor on terms of performance. Probably that´s where these urban legends come from?
In the case of large systems which contain a lot of data. It is definitely not wise to send data to EA via iCustom. It makes more to process locally and send the results as signals. if it is necessary to transmit many signals and the calculation are complicated, then sending them through iCustom or trying detect Objects using EA. This is annoying and makes things more complicated if indicator constantly calculates and transmits signals for different strategies. In addition, partial stops and signals for increasing the position etc. In this case , there must be a complex binding logic between the indcator and the EA. Isn´t it not a reasoned desire to use EA directly for everything ?
there are specific questions in my previous post and they still hanging in the air. Why this EA instead of Indicator thread is not recommended. Or however it is not recommended only for simple systems? Without going into details, it all sounds like an urban legend as well.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi! I've been struggling with an assignment for a while now. What if you don't want to use oncalculate function and indicators but build EA without using indicators.
how can indexed time series as systematically rereadable arrays be created in this case ?
I know that some pre-defined functions such as iHigh, ILow and MqlRates+copyrates etc. work nicely in the onTick function by indexing the elements of arrays properly based on bars. But as soon as I define the array myself (it doesn't matter if it's static or dynamic using the arraysetasseries function or not , etc.), I can no longer set the elements properly by indexing arrays controlled by bar(time) or indexed iteration cycles.
How to create arrays that would be time series and series based by time and indexing without predefined functions?
I have puzzled over this so much. currenttime=! time, which only allows the loop to iterate at the beginning of the new bar with indexing ++, has not allowed me to create a correct order using the for loop i++ historical informative real-time and continnually forward method. do we really have to build Cclass ourselves or can we somehow manage without it? Or is it even possible to do this using mql ?
I have understood that this task is not at all as simple as it seems, but there are certainly many here who know more about it. Those who know the topic or have the same problem themselves, please answer. I don't want everything to be made too easy for me, but maybe some study material that you can recommend or general advice would be helpful. I have read a lot of this forum but have not received any clear answer on this topic. There has been silence or that it is thought that it has to be done by using the indicator, but the topic itself has been avoided and there is no answers so far. I really hope that this time we can get clarity on this question !
thank you in advance anyway!