Optimizing EA Performance: Include Files vs. iCustom() for Indicators
Hello everyone,
I'm a beginner in MQL5, and I've developed an indicator that calculates Fair Value Gaps (FVG) and displays them for a certain number of bars, defined by the maxbar input.
I’ve also created an EA that uses this indicator to take trades according to my strategy.
However, my backtesting is extremely slow, and I suspect it's because the EA calls the indicator on each new bar,
i think that you already know the solution: make it so that when in the tester, the ea does not have the visual parts.
Regarding iCustom, it is definitely discussed many many times how iCustom slows down a backtest; so you can assume that doing this part inside a include file or ea, would be benefitial; however, as this is a technical forum, you are expected to give much more details than you have given.
For example the typical first msg is to include your source code. It does not have to be your full code, but it needs to have what you think might be the area and functions that might be the target of your issue. We can tell you if your assumption or suspicions are the same as what we have when we read it. Please use the CODE S feature in the msg editor, or the moderators may delete your msg and tell you to do it again.
i think that you already know the solution: make it so that when in the tester, the ea does not have the visual parts.
Regarding iCustom, it is definitely discussed many many times how iCustom slows down a backtest; so you can assume that doing this part inside a include file or ea, would be benefitial; however, as this is a technical forum, you are expected to give much more details than you have given.
For example the typical first msg is to include your source code. It does not have to be your full code, but it needs to have what you think might be the area and functions that might be the target of your issue. We can tell you if your assumption or suspicions are the same as what we have when we read it. Please use the CODE S feature in the msg editor, or the moderators may delete your msg and tell you to do it again.
That's great advice. Anything drawn on the chart is a dog during backtesting.
I would only add to this the option of making the indicator a resource in the EA.
Connecting custom indicators as resources - Advanced language tools - MQL5 Programming for Traders - MetaTrader 5 algorithmic/automatic trading language manual
- www.mql5.com
Embedding the code of the indicator inside the EA is one of the most useless thing you can do. iCustom is not slow. iCustom do not produce bugs.
Check how you coded your indicator. Backtest it by itself to be sure it backtests fast. If not improove your broken code.
The most common cause in case of slowness of indicators is that all bars are constantly recalculated for each bar, or even worse, for each tick. Use prev_calculated and code indicators correctly, skipping the reloading of useless data that already been calculated.
Then use iCustom into your EA, and if you want to embed the indicator inside the EA, use the #resource command.Thanks a lot for all these interresting answers.
I apologize for posting a message without any code, here is the code of my EA.
FYI, backtesting the indicator itself is much faster than the EA, so I assume that my indicator's code is not that bad (it uses prev_calculated). However it is called by my EA every new bar with a calculation period of 240 bars and I think that is the reason why my EA runs slow.
Based on you comments I will try the following actions:
- Add the #ressource option to make the indicator a ressource in the EA.
- Comment the indicator section related to visual part so that my indicator just have to work on the last 4 bars for each new bar in my EA.
// Trade #include <Mql5Book\Trade.mqh> CTrade Trade; // Price #include <Mql5Book\Price.mqh> CBars Price; // Money management #include <Mql5Book\MoneyManagement.mqh> // Trailing stops #include <Mql5Book\TrailingStops.mqh> CTrailing Trail; // Timer #include <Mql5Book\Timer.mqh> CTimer Timer; CNewBar NewBar; // Indicators #include <Mql5Book\Indicators.mqh> //+------------------------------------------------------------------+ //| Expert information | //+------------------------------------------------------------------+ #property copyright "Bastien Gallet" #property version "1.00" #property description "FVG based EA" #property tester_indicator "FVG_v2.ex5" //+------------------------------------------------------------------+ //| Input variables | //+------------------------------------------------------------------+ input group "Section :: Main"; input ulong Slippage = 3; input bool TradeOnNewBar = true; sinput string MM; // Money Management input bool UseMoneyManagement = true; input double RiskPercent = 1; input double FixedVolume = 0.1; sinput string SL; // Stop Loss & Take Profit input int StopLoss = 50; input int TakeProfit = 100; sinput string TS; // Trailing Stop input bool UseTrailingStop = false; input int TrailingStop = 0; input int MinimumProfit = 0; input int Step = 0; sinput string BE; // Break Even input bool UseBreakEven = false; input int BreakEvenProfit = 0; input int LockProfit = 0; sinput string TI; // Timer input bool UseTimer = false; input int StartHour = 0; input int StartMinute = 0; input int EndHour = 0; input int EndMinute = 0; input bool UseLocalTime = false; input ENUM_TIMEFRAMES HTF = PERIOD_M30; input ENUM_TIMEFRAMES LTF = PERIOD_M3; // config input group "Section :: FVG"; input bool ContinueToMitigation = true; // Continue to mitigation input int MaxBar = 240; // Number of bars to observe input double MinPoint = 300.0; // Minimal points to consider the gap as a FVG input double MinBodyProportion = 0.7; //Minimal proportion of the candle body to consider the FVG as valid input double SLPoint = 100.0; // Nombre de points de marge entre la FVG et le stop loss input double TPPoint = 0.0; // Nombre de points de sécurité entre le dernier higher high et le take profit input group "Section :: Dev"; input bool DebugEnabled = true; // Enable debug (verbose logging) //+------------------------------------------------------------------+ //| Global variables | //+------------------------------------------------------------------+ bool glBuyPlaced, glSellPlaced; //+------------------------------------------------------------------+ //| Structures definition | //+------------------------------------------------------------------+ struct TradeInfos { double stopPrice; double takePrice; int stopPoints; int takePoints; }; //+------------------------------------------------------------------+ //| Expert initialization function | //+------------------------------------------------------------------+ int OnInit() { Trade.Deviation(Slippage); return(0); } //+------------------------------------------------------------------+ //| Expert tick function | //+------------------------------------------------------------------+ void OnTick() { // Check for new bar bool newBar = true; int barShift = 0; if(TradeOnNewBar == true) { newBar = NewBar.CheckNewBar(_Symbol, LTF); barShift = 1; } // Timer bool timerOn = true; if(UseTimer == true) { timerOn = Timer.DailyTimer(StartHour,StartMinute,EndHour,EndMinute,UseLocalTime); } // Update prices Price.Update(_Symbol, LTF); // Order placement if(newBar == true && timerOn == true) { //FVG indicator buffers double HTFFvgHighPrice[], HTFFvgLowPrice[], HTFFvgTrend[]; ArraySetAsSeries(HTFFvgHighPrice, true); ArraySetAsSeries(HTFFvgLowPrice, true); ArraySetAsSeries(HTFFvgTrend, true); //FVG indicator handler int HTFFVGHandle = iCustom(_Symbol, HTF, "FVG_v2 (ICT rules)\\FVG_v2", ContinueToMitigation, MaxBar, MinPoint, MinBodyProportion, DebugEnabled); CopyBuffer(HTFFVGHandle, 0, 0, 240, HTFFvgHighPrice); CopyBuffer(HTFFVGHandle, 1, 0, 240, HTFFvgLowPrice); CopyBuffer(HTFFVGHandle, 2, 0, 240, HTFFvgTrend); //Entry details buffer TradeInfos infos = {0, 0, 0, 0}; double ask = SymbolInfoDouble(_Symbol, SYMBOL_ASK); double bid = SymbolInfoDouble(_Symbol, SYMBOL_BID); //1st confirmation double trend = DefineTrend(_Symbol, HTF, 240); bool tradeSignal = FVGCondition(trend, HTFFvgHighPrice, HTFFvgLowPrice, HTFFvgTrend, infos); if(trend == 1) { infos.stopPoints = int(NormalizeDouble(StopPriceToPoints(_Symbol, infos.stopPrice, ask), _Digits)); infos.takePoints = int(NormalizeDouble(StopPriceToPoints(_Symbol, infos.takePrice, ask), _Digits)); } else if(trend == -1) { infos.stopPoints = int(NormalizeDouble(StopPriceToPoints(_Symbol, infos.stopPrice, bid), _Digits)); infos.takePoints = int(NormalizeDouble(StopPriceToPoints(_Symbol, infos.takePrice, bid), _Digits)); } // Money management double tradeSize; if(UseMoneyManagement == true) tradeSize = MoneyManagement(_Symbol,FixedVolume,RiskPercent,int(infos.stopPoints)); else tradeSize = VerifyVolume(_Symbol,FixedVolume); // Open buy order if(PositionType() != POSITION_TYPE_BUY && glBuyPlaced == false && trend == 1 && tradeSignal == true) { glBuyPlaced = Trade.Buy(_Symbol,tradeSize); if(glBuyPlaced == true) { double openPrice = PositionOpenPrice(_Symbol); double buyStop = BuyStopLoss(_Symbol, infos.stopPoints ,openPrice); if(buyStop > 0) AdjustBelowStopLevel(_Symbol,buyStop); double buyProfit = BuyTakeProfit(_Symbol, infos.takePoints,openPrice); if(buyProfit > 0) AdjustAboveStopLevel(_Symbol,buyProfit); if(buyStop > 0 || buyProfit > 0) Trade.ModifyPosition(_Symbol,buyStop,buyProfit); glSellPlaced = false; } } // Open sell order if(PositionType() != POSITION_TYPE_SELL && glSellPlaced == false && trend == -1 && tradeSignal == true) { glSellPlaced = Trade.Sell(_Symbol,tradeSize); if(glSellPlaced == true) { double openPrice = PositionOpenPrice(_Symbol); double sellStop = SellStopLoss(_Symbol, infos.stopPoints,openPrice); if(sellStop > 0) sellStop = AdjustAboveStopLevel(_Symbol,sellStop); double sellProfit = SellTakeProfit(_Symbol, infos.takePoints,openPrice); if(sellProfit > 0) sellProfit = AdjustBelowStopLevel(_Symbol,sellProfit); if(sellStop > 0 || sellProfit > 0) Trade.ModifyPosition(_Symbol,sellStop,sellProfit); glBuyPlaced = false; } } } // Order placement end // Break even if(UseBreakEven == true && PositionType(_Symbol) != -1) { Trail.BreakEven(_Symbol,BreakEvenProfit,LockProfit); } // Trailing stop if(UseTrailingStop == true && PositionType(_Symbol) != -1) { Trail.TrailingStop(_Symbol,TrailingStop,MinimumProfit,Step); } } // To define what is the current trend based on HTF [0: NO, -1: DOWN, 1: UP] double DefineTrend(string symbol, ENUM_TIMEFRAMES timeframe, int period) { double ma[]; int ma_handle = iMA(symbol, timeframe, period, 0, MODE_SMA, PRICE_CLOSE); ArraySetAsSeries(ma, true); CopyBuffer(ma_handle, 0, 0, 3, ma); //double current_price = NormalizeDouble(SymbolInfoDouble(symbol, SYMBOL_LAST), _Digits); double current_price = NormalizeDouble(SymbolInfoDouble(symbol,SYMBOL_ASK),_Digits); double ma_current = ma[0]; double ma_previous = ma[1]; double trend = 0; // Définir la zone neutre (en pourcentage) double neutral_zone = 0.001; // 0.1% if(current_price > ma_current * (1 + neutral_zone) && ma_current > ma_previous) { trend = 1; // Tendance haussière } else if(current_price < ma_current * (1 - neutral_zone) && ma_current < ma_previous) { trend = -1; // Tendance baissière } return trend; } // To check if the current price has mitigated an FVG in the same direction as the trend on HTF [FALSE: NO, TRUE: YES] bool FVGCondition(double trend, const double &HighPrice[], const double &LowPrice[], const double &Dir[], TradeInfos &infos) { double close = iClose(_Symbol,PERIOD_CURRENT,0); for(int i = 2; i < ArraySize(HighPrice); i++) { if(trend == 1 && Dir[i] == 1 && close < LowPrice[i] && !isMitigated(2, i, LowPrice[i])) { infos.stopPrice = GetLastLowerLow(_Symbol, PERIOD_CURRENT, i-1); infos.takePrice = GetLastHigherHigh(_Symbol, PERIOD_CURRENT, i); return true; } else if(trend == -1 && Dir[i] == -1 && close > HighPrice[i] && !isMitigated(2, i, HighPrice[i])) { infos.stopPrice = GetLastHigherHigh(_Symbol, PERIOD_CURRENT, i-1); infos.takePrice = GetLastLowerLow(_Symbol, PERIOD_CURRENT, i); return true; } } return false; } bool isMitigated(int startIndex, int closeIndex, double price) { for(int j = startIndex; j < closeIndex; j++) { double highPrice = iHigh(_Symbol, PERIOD_CURRENT, j); double lowPrice = iLow(_Symbol, PERIOD_CURRENT, j); if(highPrice >= price || lowPrice <= price) { return true; } } return false; } //Retourne le dernier lower low parmis les n dernières bougies à partir de l'indice s double GetLastLowerLow(string symbol, ENUM_TIMEFRAMES timeframe, int n, int s = 0) { double low[]; ArraySetAsSeries(low, true); int copied = CopyLow(symbol, timeframe, s, n, low); if(copied != n) { Print("Erreur lors de la copie des données de prix bas"); return 0; } double lastLowerLow = low[0]; int lastLowerLowIndex = 0; for(int i = 1; i < n; i++) { if(low[i] < lastLowerLow) { lastLowerLow = low[i]; lastLowerLowIndex = i; } } return lastLowerLow; } //Retourne le dernier higher high parmis les n dernières bougies à partir de l'indice s double GetLastHigherHigh(string symbol, ENUM_TIMEFRAMES timeframe, int n, int s = 0) { double high[]; ArraySetAsSeries(high, true); int copied = CopyHigh(symbol, timeframe, s, n, high); if(copied != n) { Print("Erreur lors de la copie des données de prix hauts"); return 0; } double lastHigherHigh = high[0]; int lastHigherHighIndex = 0; for(int i = 1; i < n; i++) { if(high[i] > lastHigherHigh) { lastHigherHigh = high[i]; lastHigherHighIndex = i; } } return lastHigherHigh; }
Custom(_Symbol, HTF, "FVG_v2 (ICT rules)\\FVG_v2", ContinueToMitigation, MaxBar, MinPoint, MinBodyProportion, DebugEnabled); CopyBuffer(HTFFVGHandle, 0, 0, 240, HTFFvgHighPrice);
Your performance problem is likely directly caused by the fact you are recreating the custom indicator every bar. That is what causes it to recalculate ALL the indicator history on EVERY new bar.
If you've had formal programming training you were likely taught to try & avoid globals for clean code. MQL goes against that teaching - pretty much forcing you to use globals if you want data to persist across subsequent calls of OnTick, etc.
Step 1 of your solution is simply to move your indicator handle declaration to the *global* scope, ie., do this at the global scope:
int HTFFVGHandle = INVALID_HANDLE; instead of inside the OnTick function.
Then only call iCustom ONCE in the OnInit function, checking the resulting handle is valid - & returning INIT_FAILED if it isn't. That will create a single indicator instance, rather than making one on every new bar. After that ONLY new indicator calculations will be run on new incoming data, rather than recalculating EVERYTHING every time you want to get data!
Do this for ALL indicators you are using - recreating them all the time & thus forcing full recalculation each time is simply madness, & a complete waste of CPU time! (& *your* time spent waiting for results!)
After that, your CopyBuffer calls will be fast, as is normal, since they are simply copying data from arrays in a block, rather than recalculating every time.
Any variable you want to have persistent data on basically needs to be moved to the global scope - otherwise that info needs to be recalculated every time you want it. Your formal training won't like it (like mine), the large number of global declarations will start looking very untidy (like mine), but the performance will greatly improve.
Bottom line: don't recreate & recalculate stuff that has already been calculated & stored previously (like indicator buffers!) if you can easily avoid it!
Brendan.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hello everyone,
I'm a beginner in MQL5, and I've developed an indicator that calculates Fair Value Gaps (FVG) and displays them for a certain number of bars, defined by the maxbar input.
I’ve also created an EA that uses this indicator to take trades according to my strategy.
However, my backtesting is extremely slow, and I suspect it's because the EA calls the indicator on each new bar, recalculating maxbar each time. I considered reducing the calculations to only the last four bars (the minimum required to determine an FVG), but this would remove the visual aspect of my indicator.
My question is:
Would it be more efficient to define my indicators directly within my EA using include files, rather than calling them through buffers and the iCustom() function? Is this a common practice? Are there any drawbacks to this approach?
Thanks in advance for your insights!