Download MetaTrader 5

Strategy Testing Bug

To add comments, please log in or register
Travis Silvers
162
Travis Silvers  

While working with the Strategy tester to optimize an experts parameters I have run into an issue where when optimizing the Balance + Profit factor the result ends up blank.

I export to XML and see that the result appears to be the max size for a double.

I'm not sure what has caused this but it is repeatable. 

Anyone else experiencing this?

Build 478 

Slava
Moderator
7020
Slava  

DBL_MAX assumed as empty value

Documentation on MQL5: Standard Constants, Enumerations and Structures / Named Constants / Numerical Type Constants
  • www.mql5.com
Standard Constants, Enumerations and Structures / Named Constants / Numerical Type Constants - Documentation on MQL5
Travis Silvers
162
Travis Silvers  

Curious. Does that affect the GA during optimization to have a result with a value set at DBL_MAX? I hope that doesn't skew the optimization towards those results.

When exported to XML it results in a value with all of the zeros to represent DBL_MAX, which causes a problem when trying to open the XML in Excel. If it were to instead use scientific notation or another value to represent an empty value when written to XML that would prevent Excel from having a problem.

Documentation on MQL5: Standard Constants, Enumerations and Structures / Named Constants / Numerical Type Constants
  • www.mql5.com
Standard Constants, Enumerations and Structures / Named Constants / Numerical Type Constants - Documentation on MQL5
Slava
Moderator
7020
Slava  

You should know how calculated the profit_factor value

profit_factor = gross_profit / gross_loss;

If gross loss is zero then profit factor cannot be calculated and its value becomes DBL_MAX (NOT ZERO!!!) 

Documentation on MQL5: Standard Constants, Enumerations and Structures / Named Constants / Numerical Type Constants
  • www.mql5.com
Standard Constants, Enumerations and Structures / Named Constants / Numerical Type Constants - Documentation on MQL5
Travis Silvers
162
Travis Silvers  

Thank you for the information. I assumed it was to prevent some type of divide by zero issue or overflow.

I would still like to know how that effects the GA optimization process? Typically the "Result" with the largest value drives the optimization routine, so if the value is DBL_MAX it could potentially and erroneously drive optimization towards those results. It is of course possible that the optimization routine accounts for this and that is the information I am looking for.

Thanks 

Documentation on MQL5: Standard Constants, Enumerations and Structures / Environment State / Running MQL5 Program Properties
  • www.mql5.com
Standard Constants, Enumerations and Structures / Environment State / Running MQL5 Program Properties - Documentation on MQL5
Slava
Moderator
7020
Slava  
firestrand:

I would still like to know how that effects the GA optimization process? Typically the "Result" with the largest value drives the optimization routine, so if the value is DBL_MAX it could potentially and erroneously drive optimization towards those results. It is of course possible that the optimization routine accounts for this and that is the information I am looking for.

DBL_MAX is  the best result for GA. And there is no error because gross loss is 0 - You lost nothing
Travis Silvers
162
Travis Silvers  
stringo:
DBL_MAX is  the best result for GA. And there is no error because gross loss is 0 - You lost nothing
From an optimization standpoint I have to disagree. The optimization is supposed to be for max profit factor and balance, so when the gross loss has a 0 value, the profit factor is apparently set to DBL_MAX as a way to prevent a divide by zero. This appears to happen when a parameter set results in a small number of trades, however the actual amount of profit and final balance aren't necessarily inline with the best performing set of parameters. It is especially troubling when forward testing, as the forward testing routine typically takes a sample of the "top" results, which aren't necessarily the best due to the huge skew from the DBL_MAX profit factor. There are many possible ways to fix this but for now using profit factor with balance and forward testing is crippled from my viewpoint. I'll look into "Custom" and see if there is anything I can do there.
Slava
Moderator
7020
Slava  

Please more details about "many possible ways"

I see only one way. If gross profit is 0 (ie 0 trades because gross loss is 0 too) profit factor may be assumed as 0. Else profit factor will be DBL_MAX.

Documentation on MQL5: Standard Constants, Enumerations and Structures / Named Constants / Numerical Type Constants
  • www.mql5.com
Standard Constants, Enumerations and Structures / Named Constants / Numerical Type Constants - Documentation on MQL5
Travis Silvers
162
Travis Silvers  
stringo:

Please more details about "many possible ways"

I see only one way. If gross profit is 0 (ie 0 trades because gross loss is 0 too) profit factor may be assumed as 0. Else profit factor will be DBL_MAX.

Sure, here are a couple ideas off the top of my head.

1. Replace the 0 protected division and instead of returning DBL_MAX when gross_loss == 0, force gross_loss to a minimum value of 1. Though this would cause profit_factor to equal gross_profit which doesn't sound right.

2. Replace the DBL_MAX result when gross_loss == 0 with 10000 or some other non-scale busting result, as in theory profit_factor is supposed to be the "ratio of the gross profit to the gross loss in percents" and typically you don't see any 10000:1 ratios. This could be made dynamic by always making the result larger than any other found, but that seems more complex.

3. Use a squashing function such as the sigmoid** function on the results of both parameters being optimized. In my case this would force both profit factor and balance to range from 0-1. This would prevent the DBL_MAX when gross_loss = 0 to overpower all other results.

I would recommend #3 which is exactly what I plan to do when I figure out the custom option. 

Empirical analysis so far has shown that the DBL_MAX result tends to skew the results of the GA optimization towards results with 1 or 2 trades with 0 loss. Even on M1 and M5 periodicity. However when I optimize by balance only the results are much more in line with a potentially profitable system. 

 **Sigmoid or Logistic function P(t) = 1 / (1 + e^(-t)) where e is the natural number and t is the value being squashed.

Travis Silvers
162
Travis Silvers  

Here is what I have implemented in my EA for the "Custom" fitness:

double OnTester()
{
   double result = 0.0;
   result += Sigmoid(TesterStatistics(STAT_PROFIT_FACTOR));
   result += Sigmoid(TesterStatistics(STAT_SHARPE_RATIO));
   result += Sigmoid(TesterStatistics(STAT_PROFIT));
   return result * 100.0;
}

double Sigmoid(double value)
{
   return 1 / ( 1 + MathPow(M_E, -value));
}

 Enjoy!

Slava
Moderator
7020
Slava  

We will think about it

Thank you 

To add comments, please log in or register