How to Evaluate the Expert Testing Results
Slava | 1 June, 2006
First, a few words about testing procedure. Before starting to test, the testing subsystem loads the expert, sets its parameters previously defined by the user, and calls the init() function. Then the Tester plays through the generated sequence and calls the start() function every time. When the test sequence is exhausted, the Tester calls the deinit() function. At this, the entire trading history generated during testing is available. The expert efficiency can be analyzed at this moment.
The CalculateSummary function below provides calculation of test results, i.e., the data given in the standard report of the Strategy Tester.
void CalculateSummary(double initial_deposit) { int sequence=0, profitseqs=0, lossseqs=0; double sequential=0.0, prevprofit=EMPTY_VALUE, drawdownpercent, drawdown; double maxpeak=initial_deposit, minpeak=initial_deposit, balance=initial_deposit; int trades_total=HistoryTotal(); //---- initialize summaries InitializeSummaries(initial_deposit); //---- for(int i=0; i<trades_total; i++) { if(!OrderSelect(i,SELECT_BY_POS,MODE_HISTORY)) continue; int type=OrderType(); //---- initial balance not considered if(i==0 && type==OP_BALANCE) continue; //---- calculate profit double profit=OrderProfit()+OrderCommission()+OrderSwap(); balance+=profit; //---- drawdown check if(maxpeak<balance) { drawdown=maxpeak-minpeak; if(maxpeak!=0.0) { drawdownpercent=drawdown/maxpeak*100.0; if(RelDrawdownPercent<drawdownpercent) { RelDrawdownPercent=drawdownpercent; RelDrawdown=drawdown; } } if(MaxDrawdown<drawdown) { MaxDrawdown=drawdown; if(maxpeak!=0.0) MaxDrawdownPercent=MaxDrawdown/maxpeak*100.0; else MaxDrawdownPercent=100.0; } maxpeak=balance; minpeak=balance; } if(minpeak>balance) minpeak=balance; if(MaxLoss>balance) MaxLoss=balance; //---- market orders only if(type!=OP_BUY && type!=OP_SELL) continue; SummaryProfit+=profit; SummaryTrades++; if(type==OP_BUY) LongTrades++; else ShortTrades++; //---- loss trades if(profit<0) { LossTrades++; GrossLoss+=profit; if(MinProfit>profit) MinProfit=profit; //---- fortune changed if(prevprofit!=EMPTY_VALUE && prevprofit>=0) { if(ConProfitTrades1<sequence || (ConProfitTrades1==sequence && ConProfit2<sequential)) { ConProfitTrades1=sequence; ConProfit1=sequential; } if(ConProfit2<sequential || (ConProfit2==sequential && ConProfitTrades1<sequence)) { ConProfit2=sequential; ConProfitTrades2=sequence; } profitseqs++; AvgConWinners+=sequence; sequence=0; sequential=0.0; } } //---- profit trades (profit>=0) else { ProfitTrades++; if(type==OP_BUY) WinLongTrades++; if(type==OP_SELL) WinShortTrades++; GrossProfit+=profit; if(MaxProfit<profit) MaxProfit=profit; //---- fortune changed if(prevprofit!=EMPTY_VALUE && prevprofit<0) { if(ConLossTrades1<sequence || (ConLossTrades1==sequence && ConLoss2>sequential)) { ConLossTrades1=sequence; ConLoss1=sequential; } if(ConLoss2>sequential || (ConLoss2==sequential && ConLossTrades1<sequence)) { ConLoss2=sequential; ConLossTrades2=sequence; } lossseqs++; AvgConLosers+=sequence; sequence=0; sequential=0.0; } } sequence++; sequential+=profit; //---- prevprofit=profit; } //---- final drawdown check drawdown=maxpeak-minpeak; if(maxpeak!=0.0) { drawdownpercent=drawdown/maxpeak*100.0; if(RelDrawdownPercent<drawdownpercent) { RelDrawdownPercent=drawdownpercent; RelDrawdown=drawdown; } } if(MaxDrawdown<drawdown) { MaxDrawdown=drawdown; if(maxpeak!=0) MaxDrawdownPercent=MaxDrawdown/maxpeak*100.0; else MaxDrawdownPercent=100.0; } //---- consider last trade if(prevprofit!=EMPTY_VALUE) { profit=prevprofit; if(profit<0) { if(ConLossTrades1<sequence || (ConLossTrades1==sequence && ConLoss2>sequential)) { ConLossTrades1=sequence; ConLoss1=sequential; } if(ConLoss2>sequential || (ConLoss2==sequential && ConLossTrades1<sequence)) { ConLoss2=sequential; ConLossTrades2=sequence; } lossseqs++; AvgConLosers+=sequence; } else { if(ConProfitTrades1<sequence || (ConProfitTrades1==sequence && ConProfit2<sequential)) { ConProfitTrades1=sequence; ConProfit1=sequential; } if(ConProfit2<sequential || (ConProfit2==sequential && ConProfitTrades1<sequence)) { ConProfit2=sequential; ConProfitTrades2=sequence; } profitseqs++; AvgConWinners+=sequence; } } //---- collecting done double dnum, profitkoef=0.0, losskoef=0.0, avgprofit=0.0, avgloss=0.0; //---- average consecutive wins and losses dnum=AvgConWinners; if(profitseqs>0) AvgConWinners=dnum/profitseqs+0.5; dnum=AvgConLosers; if(lossseqs>0) AvgConLosers=dnum/lossseqs+0.5; //---- absolute values if(GrossLoss<0.0) GrossLoss*=-1.0; if(MinProfit<0.0) MinProfit*=-1.0; if(ConLoss1<0.0) ConLoss1*=-1.0; if(ConLoss2<0.0) ConLoss2*=-1.0; //---- profit factor if(GrossLoss>0.0) ProfitFactor=GrossProfit/GrossLoss; //---- expected payoff if(ProfitTrades>0) avgprofit=GrossProfit/ProfitTrades; if(LossTrades>0) avgloss =GrossLoss/LossTrades; if(SummaryTrades>0) { profitkoef=1.0*ProfitTrades/SummaryTrades; losskoef=1.0*LossTrades/SummaryTrades; ExpectedPayoff=profitkoef*avgprofit-losskoef*avgloss; } //---- absolute drawdown AbsoluteDrawdown=initial_deposit-MaxLoss; }
For calculations to be correct, the value of the initial deposit must be known. For this, in the init() function, the AccountBalance() function must be called that will give the balance value at the testing start.
void init() { ExtInitialDeposit=AccountBalance(); }
In the above CalculateSummary function, as well as in a standard report, the profit is calculated in the deposit currency. Other trade results, such as the "Largest profit trade" or the "Maximal consecutive loss" that are calculated on profit basis, are also measured in money terms. It is easy to recalculate the profit in points then.
... //---- market orders only if(type!=OP_BUY && type!=OP_SELL) continue; //---- calculate profit in points profit=(OrderClosePrice()-OrderOpenPrice())/MarketInfo(OrderSymbol(),MODE_POINT); SummaryProfit+=profit; ...
The obtained results can be written into the report file using the WriteReport function.
void WriteReport(string report_name) { int handle=FileOpen(report_name,FILE_CSV|FILE_WRITE,'\t'); if(handle<1) return; //---- FileWrite(handle,"Initial deposit ",InitialDeposit); FileWrite(handle,"Total net profit ",SummaryProfit); FileWrite(handle,"Gross profit ",GrossProfit); FileWrite(handle,"Gross loss ",GrossLoss); if(GrossLoss>0.0) FileWrite(handle,"Profit factor ",ProfitFactor); FileWrite(handle,"Expected payoff ",ExpectedPayoff); FileWrite(handle,"Absolute drawdown ",AbsoluteDrawdown); FileWrite(handle,"Maximal drawdown ", MaxDrawdown, StringConcatenate("(",MaxDrawdownPercent,"%)")); FileWrite(handle,"Relative drawdown ", StringConcatenate(RelDrawdownPercent,"%"), StringConcatenate("(",RelDrawdown,")")); FileWrite(handle,"Trades total ",SummaryTrades); if(ShortTrades>0) FileWrite(handle,"Short positions(won %) ", ShortTrades, StringConcatenate("(",100.0*WinShortTrades/ShortTrades,"%)")); if(LongTrades>0) FileWrite(handle,"Long positions(won %) ", LongTrades, StringConcatenate("(",100.0*WinLongTrades/LongTrades,"%)")); if(ProfitTrades>0) FileWrite(handle,"Profit trades (% of total)", ProfitTrades, StringConcatenate("(",100.0*ProfitTrades/SummaryTrades,"%)")); if(LossTrades>0) FileWrite(handle,"Loss trades (% of total) ", LossTrades, StringConcatenate("(",100.0*LossTrades/SummaryTrades,"%)")); FileWrite(handle,"Largest profit trade ",MaxProfit); FileWrite(handle,"Largest loss trade ",-MinProfit); if(ProfitTrades>0) FileWrite(handle,"Average profit trade ",GrossProfit/ProfitTrades); if(LossTrades>0) FileWrite(handle,"Average loss trade ",-GrossLoss/LossTrades); FileWrite(handle,"Average consecutive wins ",AvgConWinners); FileWrite(handle,"Average consecutive losses",AvgConLosers); FileWrite(handle,"Maximum consecutive wins (profit in money)", ConProfitTrades1, StringConcatenate("(",ConProfit1,")")); FileWrite(handle,"Maximum consecutive losses (loss in money)", ConLossTrades1, StringConcatenate("(",-ConLoss1,")")); FileWrite(handle,"Maximal consecutive profit (count of wins)", ConProfit2, StringConcatenate("(",ConProfitTrades2,")")); FileWrite(handle,"Maximal consecutive loss (count of losses)", -ConLoss2, StringConcatenate("(",ConLossTrades2,")")); //---- FileClose(handle); }
An example of how these functions are used to generate a report is given below.
void deinit() { if(!IsOptimization()) { if(!IsTesting()) ExtInitialDeposit=CalculateInitialDeposit(); CalculateSummary(ExtInitialDeposit); WriteReport("MACD_Sample_Report.txt"); } }
You can see that the reports can be generated not only after testing, but at deinitialization of live expert advisor. You may ask how to get to know the initial deposit size if the account history was downloaded in the terminal only partially (for example, only one-month history was requested in the Account History tab). The CalculateInitialDeposit function helps to solve this problem.
double CalculateInitialDeposit() { double initial_deposit=AccountBalance(); //---- for(int i=HistoryTotal()-1; i>=0; i--) { if(!OrderSelect(i,SELECT_BY_POS,MODE_HISTORY)) continue; int type=OrderType(); //---- initial balance not considered if(i==0 && type==OP_BALANCE) break; if(type==OP_BUY || type==OP_SELL) { //---- calculate profit double profit=OrderProfit()+OrderCommission()+OrderSwap(); //---- and decrease balance initial_deposit-=profit; } if(type==OP_BALANCE || type==OP_CREDIT) initial_deposit-=OrderProfit(); } //---- return(initial_deposit); }
This is the way in which the reports are generated in MetaTrader 4 Client Terminal.
It can be compared to data calculated using the exposed program.
Initial deposit 10000 Total net profit -13.16 Gross profit 20363.32 Gross loss 20376.48 Profit factor 0.99935416 Expected payoff -0.01602923 Absolute drawdown 404.28 Maximal drawdown 1306.36 (11.5677%) Relative drawdown 11.5966% (1289.78) Trades total 821 Short positions(won %) 419 (24.821%) Long positions(won %) 402 (31.592%) Profit trades (% of total) 231 (28.1364%) Loss trades (% of total) 590 (71.8636%) Largest profit trade 678.08 Largest loss trade -250 Average profit trade 88.15290043 Average loss trade -34.53640678 Average consecutive wins 1 Average consecutive losses 4 Maximum consecutive wins (profit in money) 4 (355.58) Maximum consecutive losses (loss in money) 15 (-314.74) Maximal consecutive profit (count of wins) 679.4 (2) Maximal consecutive loss (count of losses) -617.16 (8)
The SummaryReport.mq4 file attached to this article is recommended to be placed in the experts\include directory and inserted using the #include directive.
#include <SummaryReport.mq4> double ExtInitialDeposit;