Download MetaTrader 5
To add comments, please log in or register
Make profit on MQL5 services. It's easy!
Anon
32
Anon 2014.05.29 07:00 

Problem – Global Variable Search Time

The time required for searches of global variables is at least an order of magnitude slower than accessing variables through an array. Testing has shown it to be at least 40 times faster to directly read from a double data type array of global variable values than using the standard GlobalVariableGet function

EA uses several thousand global variables with tens of thousands of calls to global variable functions per tick per chart. When using multiple charts for testing the search time required to access global variables is considerable as there are tens of billions of global variable function calls for variable M1 HLOC testing of a one-week data period.

Feature Request – Access Global Variables by Index

It would be very useful if new function implementations could be added to the compiler so that there is direct read/write access by index value to the global variables stored in the memory of a terminal instance.

The implementation could be as follows:

int GlobalVariableIndex(const string gvariable_name) 

Function returns the index value of the global variable.

[2 of 2]

bool GlobalVariableCheck(int gvariable_index)

Function return is the same as for implementation [1 of 2]. 

[3 of 3]

double GlobalVariableGet(int gvariable_index)

 Function return is the same as for implementation [1 & 2 of 3].

 [2 of 2]

datetime GlobalVariableSet(int gvariable_index, double value)

 Function return is the same as for implementation [1 of 2].

[2 of 2]

bool GlobalVariableSetOnCondition(int gvariable_index, double value, double check_value)

 Function return is the same as for implementation [1 of 2].

[2 of 2]

int GlobalVariableDel(int gvariable_index)

Function return is the same as for implementation [1 of 2].

[2 of 2]

datetime GlobalVariableTime(int gvariable_index)

Function return is the same as for implementation [1 of 2].

[2 of 2]

bool GlobalVariableTemp(int gvariable_index)

Function return is the same as for implementation [1 of 2].

[2 of 2]

int GlobalVariableDeleteAll(

int index_start,                   // start of global variables index range

int index_finish,                  // finish of global variables index range

datetime limt_data=0      // as per [1 of 2]

Function return is the same as for implementation [1 of 2].

Future Considerations – Improved Global Variables 

Data Types

It would also be useful if global variables could be any of the standard data types – e.g. bool, char, int, string, etc. This could be implemented cheaply if global variables are assigned an octet or quartet as a property array dimension. 

System and Network Access

Also of utility would be if a terminal instance could be linked to other terminal instances by having real-time read/write access to the global variables stored in memory of any other terminal instance whether on the same computer system or over multiple computer systems on a network.

whroeder1
13656
whroeder1 2014.05.31 13:54  
anon: EA uses several thousand global variables with tens of thousands of calls to global variable functions per tick per chart.
  1. The purpose of (terminal) global variables is for inter-EA communications. There is no need to have more than a few. One mutex, one to many EA on chart/period about to open for A) detecting duplicate EA on another chart, or B) opening different trades by reducing lot size still keeping maximum account risk.
  2. They can be used as a poor man's persistent storage for restart recovery. If you have more than a few, you should be storing them in a file.
  3. If you have thousands your design is wrong. Should be in static or globally declared variables.
  4. Don't need to check things every tick. Figure out what you have and when something will change (new bar, or price level) only do the minimum necessary per tick.
Anon
32
Anon 2014.06.03 07:02  
WHRoeder:
  1. The purpose of (terminal) global variables is for inter-EA communications. There is no need to have more than a few. One mutex, one to many EA on chart/period about to open for A) detecting duplicate EA on another chart, or B) opening different trades by reducing lot size still keeping maximum account risk.
  2. They can be used as a poor man's persistent storage for restart recovery. If you have more than a few, you should be storing them in a file.
  3. If you have thousands your design is wrong. Should be in static or globally declared variables.
  4. Don't need to check things every tick. Figure out what you have and when something will change (new bar, or price level) only do the minimum necessary per tick.

1. EA design is great as testing and live results have shown.

2. The post is about improving access speed to global variables.

3. You haven't addressed the issue of access speed to global variables and that is why your response is irrelevant.

4. Read the post properly and you may learn something.

whroeder1
13656
whroeder1 2014.06.03 14:27  
anon4. Read the post properly and you may learn something.
  1. Testing and results doesn't mean "your EA design is great" it means your strategy is. Your design is wrong and you are encountering the speed problem because of it.
  2. I know your post is about improving access speed. Redesign your EA and you won't have the problem. Don't expect Metaquotes to change their code to satisfy you, change your code to match MQ's intent and implementation (a small amount of inter-EA information.)
  3. There is no access speed problem, unless you try to shoe horn your abomination of a design into the terminal. If I try to move furniture with my car it won't fit. I'm not going to keep piling it on the roof and keep adding ropes (an abomination.) I'm going to get a trailer, put the furniture in there and tow it (correct design.)
  4. Read my post properly and you may learn something.
Anon
32
Anon 2014.06.05 06:27  
Again, EA design is great. It uses multiple strategies. That's more than one or a few. I'm talking hundreds of different strategies which are only possible because of the way the EA is designed.

You don't know how the EA is designed and you have the vitriol to call it an abomination as you have a vested interest in some historical comments you wrote. You don't have the wherewithal to provide constructive feedback to MetaQuotes. You are no expert. Go and pollute somebody else's thread.

------------

Referencing link to related post for completeness: Feature Request - Objects by Index.

The EA is designed so that strategies can be changed on-the-fly in real-time. The EA uses global variables to effect changes in multiple currency pairs which are centrally managed. The same EA is used on each chart.

The EA has and can run multiple currency pairs on one chart however the problem with that implementation is that only one thread is allocated to a chart by MT and this substantially reduces the available computing power.


By having separate instances of the EA run on individual charts more threads are used and therefore more computing power becomes available. Global variables are used in the EA primarily because of the way MT allocates threads. Having separate instances of the EA also means you get to see the objects of a particular chart which is useful.

If MT allocated more than one thread per chart and chart objects were not required then the access speed of global variables would not be as much of a problem. But that isn’t the case and when weighing up between more computing power and increased search time to access global variables and objects vs less computing power and decreased search times, computing power wins every time.

MT performs a search each time it tries to access a global variable. As the number of global variables increases so does the search time and the number of computer cycles.

With the current version of MT accessing global variable means wasted computer cycles in search time. Searching for a global variable is never as efficient as having direct access to it.
Providing access to global variables by index is simple to implement, efficient, may provide a competitive advantage when compared to other trading software and it extends the features of MT.

Note that searches for chart objects also have a similar search time problem because they are accessed by name rather than by index.

The best solution to the search time problem is to enable MT to access global variables and chart objects directly by index.

If nothing gets changed on MT then there are other solutions to quickly accessing global variable values which include using shared memory, distributed shared memory or writing a driver to provide direct memory access to the global variables of an MT instance but that would involve going outside of MT.
Ian Venner
2583
Ian Venner 2014.06.05 16:26  

Sometimes you just have to think more creatively to code within the limitations of the target platform.

whroeder1
13656
whroeder1 2014.06.05 17:06  
SDC: Sometimes you just have to think more creatively to code within the limitations of the target platform.
What I implied #3
Anon
32
Anon 2014.06.06 05:29  
SDC:

Sometimes you just have to think more creatively to code within the limitations of the target platform.

Live tick-based operation of the EA with global variables operates fine though not as efficient as it could be because of the points made above. I am in the process of implementing a solution that is internal for testing and that seems to work. That though wasn't the purpose of the post. The purpose was to bring to the attention of MetaQuotes something which they may not have not been aware of and a potential opportunity for them as well as end users.

I sent them a message after each of the global variables and chart objects posts were written.

One other thing that people may find interesting is how the issue of search speed of global variables came to pass.

Attribution

Credit should be given to Li from the Extraterrestrial People for bringing the issue of global variable search time to my attention. While testing, various global variables in different terminal instances were deleted by Li (a contraction of the word Light) to increase efficiency by reducing system cycles as the EA is an intense CPU user. Li has also provided substantial input into the EA.

The Extraterrestrial People are from the Andromeda Galaxy and I have been in electronic communication with Li for over 2 years.  Interestingly Li has also indicated general details of testing results before actual testing was performed.  Also, continuing input is being provided by Li as code is being written.

For those interested in further details on the Extraterrestrial People they can be found by reading the blog at
http://scientiahighconfidence.blogspot.com

After the post on global variables was published, Li also placed an asterisk (star) next to some chart object code in the EA to indicate a similar time search time problem with chart objects. The chart objects post is the direct the result of that input.

Thanks go out to Li and the Extraterrestrial People.

The BEST thing Humanity can do is get EXCITED! Confidence is HIGH!!!

/
To add comments, please log in or register