building trading system - a unified smart solution for simultaneous trading with different strategies
Search in Articles. There is several good discussions there. Also check out Libraries section of Codebase. I often look up codes there for the same subject you have described as I am always looking for better ways to do multiple strategies on multiple charts, timeframes, and symbols -- at the same time.
I also check out many links that appear below users comments. Such as the ones below your comment that could relate to this same subject.Search in Articles. There is several good discussions there. Also check out Libraries section of Codebase. I often look up codes there for the same subject you have described as I am always looking for better ways to do multiple strategies on multiple charts, timeframes, and symbols -- at the same time.
I also check out many links that appear below users comments. Such as the ones below your comment that could relate to this same subject.Multiple charts, timeframes, and symbols are not problem here as they are just used to generate a signal and that part is easy and fine...
So. I'm not asking for help directly, but rather to discuss the different options to find the best solutions. Since the topic is so broad, I just wanted to know if anyone is currently doing this to exchange ideas. What's to hide, a link to a seriously good and interesting solution is also very welcome.
To generate interest, I will describe my current solution that I have come up with...
I have signals from different modules. On top of these modules are layers of logic that combine the signals, giving them greater meaning and simplifying further work. This means that a signal is not just a signal but is already a combination of signals. At the moment there is a signal container containing pointers to all the pre-processed signals that are accessible without copying signals themselves...
The strategies are driven by the pure virtual strategy base class and they are subclasses of this class. They are loaded into the container in the OnInit section. Strategies look at pointers from a container of signals, aggregating them into a strategy. For more efficient handling, each strategy has only one output (true/false) at the beginning which is controlled and the pointer is now sent to the strategy container. The strategy container is used to simplify strategy management and here i going to make a small extra expense and copy the strategy pointers into array to avoid having to deal with each strategy code separately.
Now, When a strategy signal (true) appears in the strategy container, a trading class asks, via the base class of a pure virtual strategy, which strategy subclass its pointer belongs to.. From there we get additional information from the specific strategy such as strategy ID, trade direction, SL and TP levels etc. Each strategy also has an (executed) flag which is set to true to avoid multiple signals of same strategy, if a trade has already been successfully executed or if the risk management module does not allow to execute this strategy.
So far this logic seems very efficient and quite a clever and nice thing can come out of it.... here's where i am at the moment but most of the work is still ahead...
Multiple charts, timeframes, and symbols are not problem here as they are just used to generate a signal and that part is easy and fine...
So. I'm not asking for help directly, but rather to discuss the different options to find the best solutions. Since the topic is so broad, I just wanted to know if anyone is currently doing this to exchange ideas. What's to hide, a link to a seriously good and interesting solution is also very welcome.
...and I was just making suggestions, not advice or help -- since as they say -- there's no free help around here.
If you create an article of your own, I would be one of your readers, for sure.
The whole point is that you don't have to run separate EAs that trade specific strategies. A centralized solution has many advantages such a system is better risk management between strategies. We can block signals if we detect multiple similar signals. We can dynamically calculate the lot size according to the number of active strategies based on the available funds in the account. etc. Moreover, the account management is easier with this system as you don't need to manage different EAs. In the end, we should even be able to develop smarter strategies with less effort that build on the components of existing strategies...
Would you mind to perform a context search on the site? I admit that the built-in context search works not good, but you can ask a help from Google (site:mql5.com/en/articles/ ...).
You ideas sounds very similar to what have been discussed here many times. Here are some examples: https://www.mql5.com/en/articles/217, https://www.mql5.com/en/articles/770, https://www.mql5.com/en/articles/2166... (very very long list)
The links above are probably a bit outdated (and require revision), but there are a lot of more recent works.
Even the standard MQL5 Wizard is based on the idea of combining trading signals, filters and money management modules, which can be extended with your modules.
- www.mql5.com
Thanks for the links! I've already read the articles and got some new ideas. I myself have been more on top of data analysis and building maybe a bit more complex algorithms, so mql5 syntax itself is not my strongest feature and therefore it was especially useful reading. Dynamic management of orders in mql5 is pure mql5 syntax, and it's not the easiest for me to manage just one strategy which is complex enough and requires partial closures and changing orders systematically. Managing multiple strategies here doesn't make anything any easier and it's bound to be a proper week long headache. But that's ok with this topic, I just need to figure out the syntax of mql5 trading and it's fine...
But my main question boils down to how much to let the original strategy decide on closing and managing positions and how much to let the MM class interfere in this process... It seems ,good if MM took over management at certain moments for example closing all positions when they are in a very high profit and vice versa. But In this case we would not get statistics based on original strategies and that´s very bad.
Here are some thoughts , that if we leave full control of positions to the strategies but the RM module would have the right to block the strategy if it has brought too many losses and similarly the RM module would have the right to leverage good strategies.... but this approach also not ideal and may have its bad sides... So, obviously there is no answer to this question and both options should be introduced , then they can be compared. For example, in terms of safety, we could run a live MM position control system and in the demo we can run a parallel strategy based control system to analyze each strategy separately.
What is clear is that such a solution should allow us to better manage our assets rather than running separate systems. Here we would have to do complex calculations where we would calculate the size of the allowed order from the sl and tp levels requested by the corresponding strategy, taking into account the available funds and also the maximum potential losses of other strategies.
Here are some thoughts , that if we leave full control of positions to the strategies but the RM module would have the right to block the strategy if it has brought too many losses and similarly the RM module would have the right to leverage good strategies.... but this approach also not ideal and may have its bad sides... So, obviously there is no answer to this question and both options should be introduced , then they can be compared. For example, in terms of safety, we could run a live MM position control system and in the demo we can run a parallel strategy based control system to analyze each strategy separately.
What is clear is that such a solution should allow us to better manage our assets rather than running separate systems. Here we would have to do complex calculations where we would calculate the size of the allowed order from the sl and tp levels requested by the corresponding strategy, taking into account the available funds and also the maximum potential losses of other strategies.
If/when you have an EA with modular structure and engineer a set of input parameters (including blocking, filtering, etc) for each module, then you can optimize the EA in order to find proper balance of resposibility and weight of modules. This is a good case for the tester to help you and make all routine work.
On the other hand you can assemble independent EAs into dynamic baskets with best risk/reward ratio (or by other marks) - such custom tools are available in MQL5 as well. This approach is more robust since every EA is running independently, and if one of them is terminated by a critical error all the other will continue to work, whereas in your approach with a single EA with multiple systems any single error will stop trading completely (and can lead to losses due to not monitoring open positions properly).
If/when you have an EA with modular structure and engineer a set of input parameters (including blocking, filtering, etc) for each module, then you can optimize the EA in order to find proper balance of resposibility and weight of modules. This is a good case for the tester to help you and make all routine work.
On the other hand you can assemble independent EAs into dynamic baskets with best risk/reward ratio (or by other marks) - such custom tools are available in MQL5 as well. This approach is more robust since every EA is running independently, and if one of them is terminated by a critical error all the other will continue to work, whereas in your approach with a single EA with multiple systems any single error will stop trading completely (and can lead to losses due to not monitoring open positions properly).
I understand this, therefore all aspects must be carefully considered. This time i try to make everything as efficient as possible so that there is no unnecessary waste of resources. For example, when optimizing, I often consciously use static polymorphism instead of dynamic one. I used only pointers of abstract classes with true / false flag stuffed into regular array to identify strategies, where the strategy eliminates further observation of the object and immediately gives mostly a false message, so we only check the additional data of the object itself when it has given a true signal that it should be analyzed. And of course we only process each timeframe only at the moment of a new candle formation. (Here, what makes things more complicated is that the transaction executor itself has to work in real time and monitor ontick data, for example, to avoid transactions with too large spread. What makes this particularly complex is that, even at the close of a transaction, the data should be processed in real time to avoid high spread...but the rest of the time, I try to avoid real-time data processing. So, this topic is currently under construction and I haven't fully thought out the full solution yet.)
Since interaction with strategy objects occurs so rarely, giving up abstractions would be suicide here, so therefore the strategies themselves are completely virtual. Nevertheless, I have still tried to build the system resource-efficiently and so far I have managed to avoid linked lists and other similar slightly more expensive methods, getting an almost equally flexible dynamic solution. (but cheaper).
I hope you don't mind, but I stole your idea that each strategy class only needs to store its own ticket numbers to process the transaction data later. I can immediately see if the ticket number is greater than 0 and get a much better evaluation instead of the bool is_executed variable. I appreciate that idea and it makes things much better.
Oh well, coming back to multiple linked EAs versus one EA, the main argument for using one is resource saving. One EA is much more efficient because we don't have to process the data separately and load the memory with multiple loaded similar data. This is especially important when we are optimizing parameters while processing a large datasets at once in runtime. It would be pointless to do it separately, plus the costs of running the metatrader program itself. The idea here is that if strategies for different symbols were to be combined under different linked EAs, each EA would then only load and process data for its own instrument. But again, there is a big NO if we also want to measure the strength of a currency against other currencies...
Back on topic, I've been experimenting with different solutions and getting more into the mql5 trading syntax. I now think that to manage multi strategies, the strategies should each be as self-managing as possible , so that the whole system management does not become a mess. I'll add a few examples here if anyone is interested to contribute... For example, to track the active positions of each strategy separately , instead of using HistorySelect and constantly loading the entries into the buffer and checking DEAL_ENTRY_OUT, it makes more sense to use for example such a solution:
After execution we first immediately catch the data of result.order using corresponding strategy setter method via virtual base class, after result.retcode==TRADE_RETCODE_DONE. This transforms during the deal cycle to POSITION_IDENTIFIER, and we can easily track each active deals throughout their lifecycle. For example I use this method to also add corresponding price and time, so that we can activate the gate filter to exclude entries at the same time and price by the same strategy:
void Set_Deal_Data(double price, datetime time, ulong ticket) override { pos_counter++; m_price = price; m_time = time; Add_Position(ticket); m_has_executed = true; // Enter cooldown gate }
Now on we can use separate management based on each strategy:
void Add_Position(ulong ticket) { ArrayResize(m_pos_id, pos_counter, 4); m_pos_id[pos_counter - 1] = ticket; }
We only activate position tracking when we have an active transaction and we can also simply add the maximum number of allowed position to the strategy.
if (pos_counter > 0 ) { Check_Positions(); if (pos_counter >= max_pos ) m_pos_limit = true; else m_pos_limit = false; }
and then finally such a simple and effective algorithm for tracking active positions which removes closed positions.
void Check_Positions() { for(int i = pos_counter - 1; i >= 0; i--) { Print(" Strategy 1 Check_Positions() m_pos_id[i] ",m_pos_id[i]); if(!PositionSelectByTicket(m_pos_id[i])) { Print(" Strategy 1 pos_counter",pos_counter," Check_Positions() remove closed ticket at index ", i, ": ", m_pos_id[i]); if(i != pos_counter - 1) { m_pos_id[i] = m_pos_id[pos_counter - 1]; } pos_counter--; } } ArrayResize(m_pos_id, pos_counter, 4); }
I've been testing this solution for days and so far everything works like clockwork. As I am building a pretty serious system I would be grateful if someone who is a mql5 pro could tell me if there is anything that could cause problems in using such a simple solution with PositionSelectByTicket() ? Also this not exclude my previous idea that each strategy should accumulate a score and be managed higher up based on that , but I just don't think it makes sense to run the HistorySelect() function with other necessary functions all the time, especially if we have to do the calculations separately on a each strategy. It would be more efficient to activate it for example once a week or month if we want a historical analysis to collect scores for each strategy - or maybe it would be possible to use a some customized solution to collect strategy based data like win and loss scores in a cheap way? I haven't thought more deeply about it yet.
In any case, i have come to the conclusion that for the sake of efficiency and manageability, each strategy should manage its own logic itself as much as possible, and therefore the scores should be collected locally by each strategy itself, and the higher logic could ask for them if necessary. Of course, there must be a way to intervene from above, and for that you only need minimally corresponding setter and getter methods. In this way, it is likely to be possible to launch a very large number of strategies at the same time, hundreds or even thousands if the need should arise...
I now think that to manage multi strategies, the strategies should each be as self-managing as possible , so that the whole system management does not become a mess.
It's better to use OOP to prevent a mess in code. A lot of trading frameworks (with different complexity and capabilities) are already published on this site.
using OOP capabilities, here everyone has (or at least could have) their own vision, that's what makes it interesting. There is no best solution, because there is always a chance to make an even better solution. At this point, I think that you have to establish your own vision and there is no point in blindly copying something if it differs from it - at this point, of course, there is no point in reinventing the wheel either. One of the main rules here is that simplicity usually comes at the expense of performance, so we should find an appropriate optimum point. Therefore sometime it may be quicker to build your own solution than modify others or search for a suitable one...
So, at the moment, the OOP is used in such a way that there is the world of strategies, and then there is the trading center class.I use a simple pure virtual base class to manage the strategies where strategies transmit the trading signal and other necessary parameters:
class C_Virtual_Strategy { public: virtual ~C_Virtual_Strategy() {} virtual void Set_Deal_Data(double price, datetime time, ulong ticket) = 0; virtual short Get_Execute() = 0; // Must return -1 (sell), 1 (buy), 0 (none) virtual short Get_ID() = 0; virtual int Get_Magic() = 0; virtual string Get_Comment() = 0; virtual double Get_SL() = 0; virtual double Get_TP() = 0; virtual short Get_Pos_Counter() = 0; virtual long Get_Pos_Id(int i) = 0; };
All functions and members are public for testing at the moment. With Get_Execute()==0, we can early block the strategy from processing at a higher level.
Now the link between the two worlds, there is a lightweight container containing pointers of each strategy:
class C_Strategy_Container { public: C_Virtual_Strategy *strategies[STRATEGIES]; short strategy_count; C_Strategy_Container() { strategy_count = 0; } // during OnInit initialiazion - we add strategies into container void AddStrategy(C_Virtual_Strategy &strategy) { if (strategy_count < 10) { strategies[strategy_count] = &strategy; strategy_count++; } } void Process_Strategy(C_Trading_Center &trading_center) { for (int i = 0; i < strategy_count; i++) { if (strategies[i]) { trading_center.Process_Signals(*strategies[i]); } } } };
It makes sense to call this container Process_Strategy() once a minute, not real time, if we want to monitor or add additional higher level setter methods to manage strategies. Now if the strategies are self-managing, then in theory we can also just use the signal itself for further processing:
C_Virtual_Strategy *strategy = strategies[i]; if (strategy == NULL) continue; if (strategy.Get_Execute()!=0) trading_center.Process_Signal(*strategy);
Since the management of the transactions themselves, the calibration of the lot sizes for the transaction or its TP and SL according to the values set by the strategy and the matching instrument type with additional MM RM which include all sorts of calculations are quite complex and extensive, it would not be worth burdening them with tracking and mixing additional strategy based logic. To ensure efficiency and optimality, it seems to me that they should work separately. Now we can process the each individual strategy in the trading center via the strategy pointer, after the Get_Execute() signal has gotten permission, we ask the responsible strategy subclass hey what strategy is SL or TP using simple getters and then we do the calculations, etc:
// inside C_Trading_Center after getting permission
Live_Engine.Update_Prices();
SL_level = Money_Management.Set_Stop_Loss (strategy.Get_SL(),signal);
TP_level = Money_Management.Set_Take_Profit(strategy.Get_TP(),signal);
Deal_Vol = Money_Management.Set_Deal_Volume(strategy.Get_SL());
Trade_Engine.Trade_Executor(strategy); At least so far this seems like the most reasonable framework with workflow which is cheap but smooth enough... I really don't know if such a ready-made form is available anywhere. But again, arguments and links are welcome... :)
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
The whole point is that you don't have to run separate EAs that trade specific strategies. A centralized solution has many advantages such a system is better risk management between strategies. We can block signals if we detect multiple similar signals. We can dynamically calculate the lot size according to the number of active strategies based on the available funds in the account. etc. Moreover, the account management is easier with this system as you don't need to manage different EAs. In the end, we should even be able to develop smarter strategies with less effort that build on the components of existing strategies...
I once built such a system but due to lack of experience the whole framework became too rigid, a bit too large and inefficient which made it difficult to manage... So now I'm starting to implement such a solution again, trying to do it more intelligently. Is anyone else building or planning to further develop such a system? and does anyone have a deeper interest in this topic at all? Such a solution can be built in different ways but unfortunately, the task is not the easiest and some skills are needed to develop that system. So if anyone is interested in getting involved, it would be great to discuss the details and gather good ideas together.