Discussion of article "Creating Custom Criteria of Optimization of Expert Advisors"

[MetaQuotes]

New article Creating Custom Criteria of Optimization of Expert Advisors is published:

The MetaTrader 5 Client Terminal offers a wide range of opportunities for optimization of Expert Advisor parameters. In addition to the optimization criteria included in the strategy tester, developers are given the opportunity of creating their own criteria. This leads to an almost limitless number of possibilities of testing and optimizing of Expert Advisors. The article describes practical ways of creating such criteria - both complex and simple ones.

Author: Dmitriy Skub

[Jinsong Zhang]

You know, the multi pairs EA has different testing result in different symbol. I test EA in EURUSD, it don't open long trades for AUDUSD, then test it in AUDUSD, it don't open short trades for EURUSD!! How to resolve? Thanks

[yacoov]

Thanks for great  article Dmitriy,

 

Is there any way or room  to fit Pardo's   Perfect Profit criteria  http://www.breakoutfutures.com/Newsletters/Newsletter0605.htm   on top of your criterias?

[Ingvar Engelbrecht]

Excellent.

From my earlier work in the area of Neural Nets and Genetic Algorithms predicting futures I realized the importance of a reasonably straight equity curve.

and wrote some routines to take that into consideration. It really is a measure of the "robustness" of a predictive system.

[Ingvar Engelbrecht]

And now I have finally started to test and explore the possibilities. And I agree. A whole new world is open up. It is a very powerful tool. Thanks again for the article

[Ingvar Engelbrecht]

Findings doing practical work.

The straightness module is a good start but incomplete. It is possible to get a very good rating with zero profit. So profit has to come in to the equation. Just to add some kind of measure from the total profit

does not work. You could get a very straight line with good value by having some wins in the beginning, some drawdowns in the middle  and some better winnings in the end. It would produce a level straight line with good fit and show some profit. But that is really not what we are looking for.

We really want a regression line wich rises upwards with good fit. So, in order to make the idea of using a regression line with as little deviation as possible the coefficient for the upward slope has to be incorporated into the equation. That is what

we want to see. An upward sloping regression line with good fit. I will make an attempt of incorporate that. Suggestions and assistance are welcome  :-)

[Ingvar Engelbrecht]

The value of the custom criteria is shown in the result listing along with the "standard" criteria. Is it possible to create 2 values in the custom criteria? Have not figured out a good way to combine straightness and slope

[Ingvar Engelbrecht]

I am trying the CSTS code.

I find this result a bit strange:

Result      Profit   #Trades  Frofit factor  DrawDown    Expected Payoff  Recovery Factor

0.58        1237     84          1.26            12.70            14.74                0.93

0.57        1598     90          1.38             8.69             17.36                1.76

Here is another example

0.61        3175     123        1.33             21.04           25.82                 1.48

0.60        4460     145        1.49             11.32           30.77                 2.56

From all points of view the values for the second line is better!!  But the score is lower

The only thing I can deduce is that there is a penalty on many trades. I would have it the other way around

And as a last point. There seems to be exactly one person interested in this subject.  Me.

Further on this. I made a mistake in my code and it resulted in pending orders not been cleared. It also resulted in only 5 order placed for testing period over 12 month. With a good profit.

This really boosted the optimization result to over 100. Clearly the value of the "average win" was extremely high resulting in this extreme score. Technically it is correct but it is meaningless

in the context of backtesting. How probable is it that long trends like that are represenatative?. So I figured that the number of trades has to be incorporated in the equation in some way.

With some trial and error changing the code I arrived at a method that produces results that I find useful.

changes:

//    CSTS:
   double   tssf=real/teor;
   if(tssf <= 0) return 0;
  
   work = TesterStatistics( STAT_TRADES );
   if( work <= 0) work = 1;
   work = MathSqrt(work/4);
  
   tssf = tssf * work;
   if( tssf < 1 ) tssf = 0;
   if(TesterStatistics(STAT_PROFIT) <= 0) tssf = 0;

   return(tssf);

The original method is probably good to judge a running system but basically useless to base parameters on from a backtest run

[Ingvar Engelbrecht]

Well, here I am again, the lone wolf in this universe  :-)

I have been trying the straightness Custom Criteria trying to get the slope of the calculated straight line into the equation. As is it can give you a very hgh rating on a very feeble profit. Just adding the end profit

into the caculation does not make it any better  In an attempt to add the actual slope into the equation  I changed the code lilke seen below.

It is not a perfect solution but it is closer to what I want to see. Using result together wit balance or profit  works fine for me with this code

//---------------------------------------------------------------------
//      Get value of the optimization result:
//---------------------------------------------------------------------
double  TBalanceSlopeCriterion::GetCriterion()
  {
//      Let's try to calculate the slope of the balance curve:
   double   current_slope=1000.0*this.balance_Ptr.CalcSlope();

//      If it is inclined down:
   if(current_slope<0.0)
     {
      return(-1.0);
     }

   double   temp=this.balance_Ptr.GetCurrentSKO();
   if(temp>0.0)
     {
      
   //   return(this.scale/temp);        //This just returns how well the results adhere to a straight line that could be anything from a level line to a line pointing uppwards
      double temp2 = this.scale/temp;   //   added Ingvar
      
      return(temp2 * current_slope);    // added Ingvar
      
     }

   return(0.0);
  }

[Rogerio Figurelli]

ingvar_e:

Well, here I am again, the lone wolf in this universe  :-)

I have been trying the straightness Custom Criteria trying to get the slope of the calculated straight line into the equation. As is it can give you a very hgh rating on a very feeble profit. Just adding the end profit

into the caculation does not make it any better  In an attempt to add the actual slope into the equation  I changed the code lilke seen below.

It is not a perfect solution but it is closer to what I want to see. Using result together wit balance or profit  works fine for me with this code

ingvar_e, not so alone, I like too much this area and was surprised with your evolution, because I do believe in this line of thought too.

For instance, for genetic algorithms, you must have a good fitness algorithm, and for sure create an custom criteria aligned to this fitness.

I'm not sure calculate the slope of balance curve is the better way, since we have several other ways, anyway count on me to explore and debate this ideas.

[Benjamin Dixon]

With much experimenting, I have come to similar conclusions. I don't know why it took me so long to find this article and discussion.

  I started with a problem where my break out and trend riding strategies tended to fit themselves to a few super profitable trades. So much profit from these few that the optimizer only "cared" about these few trades and used wide stops and far away take profit and making sure its signal caught and milked these few trades to the maximum. this occurred even on 20 year samples with 100's of trades. Even when optimizing with a dd to profit ratio, it still went after the runs with these trades at all costs cause the runs were 90% of its profit. I wanted to limit this behavior in some way without cutting the head off of the breakout/trend ride strat. I found that optimizing for relative dd percent (NOT dd ratio to profit, just dd relative percent only and nothing much else) gives good results going forward as long as minimum trades and minimum profit factor (about 1.3) are met. 

//............... 

sinput double mint; //minimum trades for optimization purposes. Will impose a penalty on runs with fewer trades. 

sinput double minpf; //minimum profit factor for optimization purposes. Will impose a penalty on runs with less pf.

//...............

double OnTester()

{ 

	double dd=TesterStatistics(STAT_BALANCE_DDREL_PERCENT);//equity instead of balance also worked well going forward

	double pf=TesterStatistics(STAT_PROFIT_FACTOR);

	int    tt=TesterStatistics(STAT_TRADES); 

	double custom=(100-dd);

	if (mint>t) custom=custom*(tt/mint); // imposes a penalty if minimum trades not met

	if (minpf>minpf) custom=custom*((pf-1)/(minpf-1)) // this line also causes losing runs to be negative as well as imposing a penalty lesser pf.

	return(custom); 

}

 

 

It seems wrong to not have profit involved but by having a pf minimum criteria, mostly quite profitable runs are appearing at the top. Cropping out dd from profitable runs is usually increasing profits. In any case, by selecting the more profitable trades and doing the final touch-ups carefully manually by hand (such as near the end of this excellent article https://www.mql5.com/en/articles/156), I can allow the optimizer to do a lot of the initial work while avoiding a lot of curve fitting. Ive experimented and had mixed results with code such as the following. (Experiments should include changing risk and starting balance too once you begin put additional weights in on top of a percent based score)

//.............

sinput double pfw //Profit factor weight 

//.............

        double custom=(100-dd)+(pf-1)*pfw;
//...........