Optimizing a strategy using balance graph and comparing results with "Balance + max Sharpe Ratio" criterion

• 1. Introduction
• 2. How to calculate a trend line for a balance graph?
• 2.1. Algorithm for calculating the optimization criterion
• 2.2. Introducing the first TrendProfit variable
• 2.3. ProfitStability variable
• 3. Working with the ALGLIB library
• 3.1. Calculation steps (GetProfitStability method)
• 4. Including the CBalanceRegression class
• 4.1. Expert Advisor generated by MQL5 Wizard
• 4.2. Expert Advisors based on the standard MACD Sample and Moving Average
• 5. Optimization on EURUSD H4, no forward test
• 5.1. EA test balance regression.mq5, EURUSD H4, no forward test
• 5.1.1 Test 1
• 5.1.2. Comparing test speed in a home local network of two PCs and in the cloud
• 5.1.3. Test 2
• 5.1.4. Test 3
• 5.2. Moving Average balance regression.mq5, EURUSD H4, no forward test
• 5.2.1. Test 1 
• 5.2.2. Test 2
• 5.2.3. Test 3
• 5.3. MACD Sample balance regression.mq5, EURUSD H4, no forward test
• 5.3.1. Test 1 
• 5.3.2. Test 2
• 5.3.3. Test 3
• 6. Optimization on EURUSD M15 with forward test
• 6.1. EA test balance regression.mq5, forward test
• 6.1.1. Test 1
• 6.1.2. Test 2
• 6.1.3. Test 3
• 6.2. MACD Sample balance regression.mq5, forward test
• 6.2.1. Test 1
• 6.2.2. Test 2
• 6.2.3. Test 3
• 6.3. Moving Average balance regression.mq5, forward test
• 6.3.1. Test 1
• 6.3.2. Test 2
• 6.3.3. Test 3
• 7. Conclusion

As of the current moment (build 1645), the ALGLIB library has already long been integrated into the standard library: [date folder]\MQL5\Include\Math\Alglib\alglib.mqh

3.1. Calculation steps (GetProfitStability method)

Results of the trades with (commission + swap + profit) > 0.0 are to be saved in the arr_profits array. The volume of all trades is to be summed up in the total_volume variable.

//---
   double   arr_profits[];                            // array of results deals 
   double   total_volume=0;                           // total volume

Next, go through all trades — fill in the arr_profits array and sum up the volume of trades to the total_volume variable:

//--- request trade history 
   HistorySelect(m_from_date,to_date);
   uint total_deals=HistoryDealsTotal();
   ulong ticket_history_deal=0;
//--- for all deals 
   for(uint i=0;i<total_deals;i++)
     {
      //--- try to get deals ticket_history_deal 
      if((ticket_history_deal=HistoryDealGetTicket(i))>0)
        {
         long     deal_type         =HistoryDealGetInteger(ticket_history_deal,DEAL_TYPE);
         double   deal_volume       =HistoryDealGetDouble(ticket_history_deal,DEAL_VOLUME);
         double   deal_commission   =HistoryDealGetDouble(ticket_history_deal,DEAL_COMMISSION);
         double   deal_swap         =HistoryDealGetDouble(ticket_history_deal,DEAL_SWAP);
         double   deal_profit       =HistoryDealGetDouble(ticket_history_deal,DEAL_PROFIT);

         if(deal_type!=DEAL_TYPE_BUY && deal_type!=DEAL_TYPE_SELL)
            continue;

         if(deal_commission==0.0 && deal_swap==0.0 && deal_profit==0.0)
            continue;

         total_volume+=deal_volume;

         int arr_size=ArraySize(arr_profits);
         ArrayResize(arr_profits,arr_size+1,50);   // resize the aray

         if(arr_size==0)
            arr_profits[arr_size]=GetSetStartBalance()+deal_commission+deal_swap+deal_profit;
         else
            arr_profits[arr_size]=arr_profits[arr_size-1]+deal_commission+deal_swap+deal_profit;

         int d=0;
        }
     }

Note: When we make the first entry to the arr_profits array, we sum up the initial balance and the trade's financial result. For all subsequent entries, sum up the previous entry and the trade's financial result.

Declare the CMatrixDouble class object. In fact, this is a matrix. Let's fill it in: trade index (start in "1") financial result.

//--- CMatrixDouble object
   CMatrixDouble xy(arr_size,2);
   for(int i=0;i<arr_size;i++)
     {
      xy[i].Set(0,i+1);
      xy[i].Set(1,arr_profits[i]);
      //Print(arr_profits[i]); // for debag
     }

Declare the objects of the necessary classes (CLinReg, CLinearModel, CLRReport) and calculate the linear regression:

//--- linear regression construction
   CLinReg        linear_regression;
   CLinearModel   linear_model;
   CLRReport      linear_report;
   int retcode;
   linear_regression.LRBuild(xy,arr_size,1,retcode,linear_model,linear_report);
   if(retcode!=1)
     {
      Print("Linear regression failed, error code=",retcode);
      return(0.0);
     }
   int nvars;
   double coefficients[];
   linear_regression.LRUnpack(linear_model,coefficients,nvars);
   double coeff_a=coefficients[0];
   double coeff_b=coefficients[1];
   PrintFormat("y = %.1f x + %.1f",coeff_a,coeff_b);

Finally, call the LRUnpack method to obtain the ratios of the y = a*x + b equation. The ratios are to be located in the coefficients array.

Now that we have calculated the linear regression using the ALGLIB library, we can proceed directly to calculating the parameter, which is the focal point of the entire article.

//--- сalculation of parameters
   double TrendProfit=((double)arr_size*coeff_a+coeff_b)-(1.0*coeff_a+coeff_b);  // the projection of the regression line on the "Y" axis 
   TrendProfit/=(double)arr_size;                                                // divided by the number of trades
   double TrendMSE=linear_report.m_rmserror;                                     // root mean square error on a training set
   double ProfitStability=TrendProfit/TrendMSE;
//--- normalize the trading volume
   if(GetVolumeNormalization())
      ProfitStability/=total_volume;
//--- we multiply by the number of deals - we aren't interested in passes which have few deals
   ProfitStability*=arr_size;
//---
   return(ProfitStability*10000.0);

Depending on the initial parameter set via the SetVolumeNormalization method, we either divide the ProfitStability parameter to the summed up traded volume or not. The result is multiplied by 10 000 to better represent the results.

Let's consider the algorithm of including the CBalanceRegression class for various EAs. 

4.1. Expert Advisor generated by MQL5 Wizard

We want to check the effect of the normalization of traded volumes on the results. This means, we need an EA that dynamically calculates the volumes of opened positions. This can be done in the MQL5 Wizard: select Expert Advisor (generate) and include one of the money management modules on the "Money management" page:

• "Trading with fixed margin" — calculate lot in % of free margin
• "Trading with fixed risk" — calculate lot in % of balance
• "Trading with optimized trade volume" — calculate lot depending on trading history

For this article, I have selected the "Trading with fixed risk" money management module, while the EA itself is named "EA test balance regression.mq5". Using it as an example, I will show how to include the file of the CBalanceRegression class.

Step 1. Include the CBalanceRegression file and declare the class object (m_balance_regression).

//--- available money management
#include <Expert\Money\MoneyFixedRisk.mqh>
//--- available module Balance Regression
#include <Balance regression\BalanceRegression.mqh>
//+------------------------------------------------------------------+
//| Inputs                                                           |
//+------------------------------------------------------------------+

and

//+------------------------------------------------------------------+
//| Global expert object                                             |
//+------------------------------------------------------------------+
CExpert ExtExpert;
CBalanceRegression m_balance_regression;
//+------------------------------------------------------------------+
//| Initialization function of the expert                            |
//+------------------------------------------------------------------+

Step 2. Add the parameter for enabling/disabling the traded volume normalization to the input parameters:

input double             Money_FixRisk_Percent=10.0;                            // Percentage of risk
//--- inputs for balance normalization
input bool               InpVolumeNormalization=true;                           // Volume normalization
//+------------------------------------------------------------------+
//| Global expert object                                             |
//+------------------------------------------------------------------+

Step 3. Set the parameters for the m_balance_regression object of the CBalanceRegression class: initial data for requesting trading history and traded volume normalization mode:

//+------------------------------------------------------------------+
//| Initialization function of the expert                            |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- Set parameters of CBalanceRegression
   m_balance_regression.SetStartBalance(AccountInfoDouble(ACCOUNT_BALANCE));
   m_balance_regression.SetFromDate(TimeCurrent());
   m_balance_regression.SetVolumeNormalization(InpVolumeNormalization);
//--- Initializing expert

Step 4. Add the OnTester procedure to the very end of the file. Here we receive the custom optimization criterion (by calling GetProfitStability) and pass parameter to the tester:

//+------------------------------------------------------------------+
//| Tester function                                                  |
//+------------------------------------------------------------------+
double OnTester()
  {
//---
   double ret=m_balance_regression.GetProfitStability(TimeCurrent());
//---
   return(ret);
  }

4.2. Expert Advisors based on the standard MACD Sample and Moving Average

EA originals are located in [data folder]\MQL5\Experts\Examples\MACD\MACD Sample.mq5 and [data folder]MQL5\Experts\Examples\Moving Average\Moving Average.mq5, respectively.

Take the original EAs and save their copies under different names. I have selected the names "MACD Sample balance regression.mq5" and "Moving Average balance regression.mq5".

#include <Trade\AccountInfo.mqh>
//--- available module Balance Regression
#include <Balance regression\BalanceRegression.mqh>
CBalanceRegression m_balance_regression;
//---
input double InpLots          =0.1; // Lots

#include <Trade\Trade.mqh>
//--- available module Balance Regression
#include <Balance regression\BalanceRegression.mqh>
CBalanceRegression m_balance_regression;

input double MaximumRisk        = 0.02;    // Maximum Risk in percentage

input int    InpMATrendPeriod =26;  // MA trend period
//--- inputs for balance normalization
input bool   InpVolumeNormalization=true; // Volume normalization
//---
int ExtTimeOut=10; // time out in seconds between trade operations

input int    MovingShift        = 6;       // Moving Average shift
//--- inputs for balance normalization
input bool   InpVolumeNormalization=true;  // Volume normalization
//---
int    ExtHandle=0;

//+------------------------------------------------------------------+
//| Initialization and checking for input parameters                 |
//+------------------------------------------------------------------+
bool CSampleExpert::Init(void)
  {
//--- Set parameters of CBalanceRegression
   m_balance_regression.SetStartBalance(AccountInfoDouble(ACCOUNT_BALANCE));
   m_balance_regression.SetFromDate(TimeCurrent());
   m_balance_regression.SetVolumeNormalization(InpVolumeNormalization);
//--- initialize common information

//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit(void)
  {
//--- Set parameters of CBalanceRegression
   m_balance_regression.SetStartBalance(AccountInfoDouble(ACCOUNT_BALANCE));
   m_balance_regression.SetFromDate(TimeCurrent());
   m_balance_regression.SetVolumeNormalization(InpVolumeNormalization);
//--- prepare trade class to control positions if hedging mode is active

//+------------------------------------------------------------------+
//| Tester function                                                  |
//+------------------------------------------------------------------+
double OnTester()
  {
//---
   double ret=m_balance_regression.GetProfitStability(TimeCurrent());
//---
   return(ret);
  }

We have three EAs, on which we can check the idea of optimizing the balance graph using the regression method. The two EAs (EA test balance regression.mq5 and Moving Average balance regression.mq5) calculate the position volume dynamically during trading, while MACD Sample balance regression.mq5 uses a fixed lot.

5.1. EA test balance regression.mq5, EURUSD, H4, no forward test

In order not to miss any options and speed up the testing, I chose the two parameters for optimization - the boundary parameters for position open and close signals:

Pic. 7. Tester, Inputs tab

• Test 1: Optimizing the standard "Balance + max Sharp Ratio" parameter, 
• Test 2: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'false', 
• Test 3: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'true'.

5.1.1 Test 1: Optimizing the standard "Balance + max Sharp Ratio" parameter.

Tester settings for Test 1 (note: "Optimization" is equal to "Balance + max Sharpe Ratio"):

Pic. 8. Tester, Settings tab

5.1.2. Comparing test speed in a home local network of two PCs and in the cloud

The home network features two computers:

1. a laptop powered by the quad-core Intel Core i3-3120M @2.50GHz, 8077 MB. Only three out of four agents are used in the test, since the terminal also needs to communicate with agents on the second computer;
2. a desktop powered by the six-core AMD Phenom II X6 1075T Processor, 4058 MB. Only four agents out of six are used in the test in order not to overheat the CPU.

Pic. 9. Optimization in the home network

Test 1 in the home network passed within 21 minutes and 43 seconds.

Now, I delete the cached test results and launch Test 1 at MQL5 Cloud Europe 1. In the cloud, the test takes 1 minute and 9 seconds. The test cost is $0.06.

Test 1 results, Result column sorted in descending order, five best results:

Pic. 10. Test 1, Optimization pass 346

5.1.3. Test 2: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'false'.

Tester settings for Test 2 (note: "Optimization" is equal to "Custom max"):

Pic. 11. Tester, Settings tab

Tester settings for Test 2 (note: "Volume normalization" in the Value column is 'false'):

Pic. 12. Tester, Inputs tab

Test 2 results, Result column sorted in descending order, five best results:

Pic. 13. EURUSDH4, Test 2, Optimization pass 270

5.1.4. Test 3: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'true':

Tester settings for test 3 (settings are similar to Test 2 ones):

Pic. 14. Tester, Settings tab

Tester settings for Test 3 (note: "Volume normalization" in the Value column is 'true'):

Pic. 15. Tester, Inputs tab

In the cloud, the test takes 2 minutes and 27 seconds. The entire testing is performed at MQL5 Cloud Europe 1. The test cost is $0.08.

Test 3 results, Result column sorted in descending order, five best results:

5.2. Moving Average balance regression.mq5, EURUSD, H4, no forward test

The following parameters are selected for optimization:

Pic. 16. Tester, Inputs tab

I will conduct three tests for Moving Average balance regression EA:

• Test 1: Optimizing the standard "Balance + max Sharp Ratio" parameter, 
• Test 2: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'false', 
• Test 3: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'true'.

Tester settings (currency, ticks generation method and time interval are the same as on Pic. 5).

5.2.1. Test 1 (optimizing the standard "Balance + max Sharp Ratio" parameter) performed in the cloud. In the cloud, the test takes 1 minute and 22 seconds. The entire testing is performed at MQL5 Cloud Europe 1. The test cost is $0.03.

Test 1 results, Result column sorted in descending order, five best results:

Let's check the best pass #19:

Pic. 17. EURUSDH4, Test 1, Optimization pass 19

5.2.2. Test 2 (optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'false') is performed in the cloud. In the cloud, the test takes 0 minutes and 00 seconds because the results have been picked up from the cache, although they are reversed. The entire testing is performed at MQL5 Cloud Europe 2. The test cost is $0.00.

Test 2 results, Result column sorted in descending order, five best results:

Pass 141 is at the very top of the table (it has the maximum value in the Result column):

Pic. 18. EURUSDH4, Test 2, Optimization pass 141

5.2.3. Test 3 (optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'true') is performed in the cloud. In the cloud, the test takes 1 minute and 13 seconds. The entire testing is performed at MQL5 Cloud Europe 2. The test cost is $0.05.

Test 3 results, Result column sorted in descending order, five best results:

Tester settings (currency, ticks generation method, time interval are the same as on Pic. 5).

5.3.1. Test 1 (optimizing the standard "Balance + max Sharp Ratio" parameter) performed in the cloud. In the cloud, the test takes 3 minutes and 55 seconds (the time is comparatively large because the task was not executed on some slow agents, therefore, 10 tasks were distributed among other agents). The entire testing is performed at MQL5 Cloud Europe 1. The test cost is $0.04.

Test 1 results:

There are very few trades (from 13 to 16) at this timeframe. I hope, there will be more trades on a lower timeframes (M15). These tests are shown in another section below. For now, the best result is #163:

Pic. 20. EURUSDH4, Test 1, Optimization pass 163

5.3.2. Test 2 (optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'false') is performed in the cloud. In the cloud, the test takes 0 minutes and 00 seconds because the results have been picked up from the cache, although they are reversed. The entire testing is performed at MQL5 Cloud Europe 2. The test cost is $0.00.

Test 2 results:

5.3.3. Test 3 (optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'true') is performed in the cloud. In the cloud, the test takes 1 minute and 5 seconds. The entire testing is performed at MQL5 Cloud Europe 2. The test cost is $0.05.

Test 3 results:

We have the same three EAs: EA test balance regression.mq5, Moving Average balance regression.mq5 and MACD Sample balance regression.mq5. Unlike the previous tests, we now test on M15 with forward test enabled.

6.1. EA test balance regression.mq5, forward test. I will conduct three tests:

• Test 1: Optimizing the standard "Balance + max Sharp Ratio" parameter
  
• Test 2: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'false'
  
• Test 3: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'true'.

6.1.1. Test 1: Optimizing the standard "Balance + max Sharp Ratio" parameter. Tester settings:

Pic. 21. Tester, Settings tab

Tester parameters are identical to the ones displayed on Pic. 7.

In the cloud, the test takes 2 minutes and 14 seconds. The entire testing is performed at MQL5 Cloud Europe 2. The test cost is $0.08.

Optimization results:

Pic. 22. EURUSDM15, Test 1, Optimization pass 352

Forward results:

Here, the best options were the ones having only 10 trades on a forward section. Of course, this is not enough, but we still need to conduct a test:

Pic. 23. EURUSDM15, Test 1, Forward pass 284

6.1.2. Test 2: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'false'.

Best optimization results (not forward):

Pic. 24. EURUSDM15, Test 2, Optimization pass 284

Forward results:

6.1.3. Test 3: Optimizing the "Custom max" custom parameter, while the traded volumes optimization parameter is 'true'. In the cloud, the test takes 3 minutes and 29 seconds. The entire testing is performed at MQL5 Cloud Europe 1. The test cost is $0.07.

Best optimization results (not forward):

Pic. 25. EURUSDM15, Test 3, Optimization pass 208

Enabling the "Normalization of traded volumes" made no significant impact on the results. Also, one of the best results was found within the optimized interval.

The best forward results:

6.2. MACD Sample balance regression.mq5, forward test

6.2.1. Test 1: Optimizing the standard "Balance + max Sharp Ratio" parameter performed in the cloud. In the cloud, the test takes 1 minute and 3 seconds. The entire testing is performed at MQL5 Cloud Europe 1. The test cost is $0.06.

Optimization results:

Pic. 26. EURUSDM15, Test 1, Optimization pass 171

The results of applying the balance graph using the linear regression method have turned out to be at the "Balance + max Sharp Ratio" optimization method level. The EAs using dynamic lot has shown interesting results: in some cases, the "traded volumes normalization" setting combined with a forward test yields a more realistic picture.

I believe, the described method has the right to exist, and it can be further improved, for example by considering the duration of profitable and loss-making trades and experimenting with the obtained values.