Optimization Solution/elimination of unwanted passes at the very beginning of the optimization/ (highly Expert Moderator-Programmer case)

 

Hello Everyone

I have a problem in optimization passes. I've searched a lot but none of questions nor the answers in the forum was the one solving this problem.

The problem is that I have some Boolean parameters in my inputs which when they are set as false there is no need to optimize some other parameters.

for example, if the "Psar_Active" parameter is set to false my EA won't use Parabolic SAR indicator in its decision-making process,

so optimizing the related input parameters of the indicator "step" and "max" is useless (only when the Activation parameter is false).

 

Unfortunately, I couldn't find a way for this. so many of the optimization results are useless

and this make the whole optimization much much less effective. (when using the genetic based algorithm)

 

Let’s assume that "Step" Parameter has 5  different steps and "Max" Parameter has 4 different   steps the total steps considering the

"Psar_Active" parameter (which has two steps) is currently resulting to 40 passes (5*4*2), which 19 of them are useless

(the ones with "Psar_Active" ==false). if this problem could be solved the total passes should be reduced to 21 (5*4*1+1).

 

The solution should be able to solve the problem for multiple parameters.

 

for example, if the Psar_Active==false Parameters Step & Max should not be optimized

but at the same time ADX_Active==true Parameters Period & Min should be optimized.

so if number of passes are as follow

 

Psar_Active 2

Step_Psar 5

max_Psar 4

 

ADX_Active 2

Period_ADX 3

Min_ADX 4

 

the total passes should be [(1*5*4+1)*(1*3*4+1)]=273 which at the current time the program uses (2*5*4*2*3*4)=960 which 687 passes (%72 of the total calculations made by the agents!!)

are totally useless and misguiding in the genetic base decision making progress.it should be noted that the ones with the false entry should not be totally eliminated. 

 

It's not really important that the total number of passes shown correctly in the input tab of the program, but if the figure is calculated correctly it will be comforting.

but when the optimization is run, there should not be any unnecessary passes in the results (slow complete method or fast genetic base)

 

I've wrote this problem later on forum but there were not much of help.

https://www.mql5.com/en/forum/298982

 

I'm not sure which way will do but i assume that writing a function in the tester sections with some parameters that

one of the is them input parameter of EA and the other one a Boolean parameter which determines the necessity of optimization

(or any solution that changes the total number of passes in optimization) can solve this problem. perhaps a remove duplicate kind of function

at the begging of the testing session should be implemented (if all the other -non relative-parameters are the same there should be only one pass

with the false input for that parameter).

 

usually relative parameter won't exceed more than 10 parameters.so the function should be something like:

void Remove_Unnecessary_Optimize(bool Main_input, Raltive_Paramer1, altive_Paramer2, . . . .,altive_Paramer10)

 

so far, I couldn't find anyway with this and I don't intend to get anything for free.

please advise me if you can help on this.

 

thank in advance for your help.

Conditional Parameter Optimization solution needed!
Conditional Parameter Optimization solution needed!
  • 2019.01.13
  • www.mql5.com
Hello Everyone I have a problem in optimization passes...
 
Mohammad Hossein Najarzadeh: I've wrote this problem later on forum but there were not much of help.

https://www.mql5.com/en/forum/298982

Don't double post! You already had that thread open.

          General rules and best pratices of the Forum. - General - MQL5 programming forum

I already answered your question there, six months ago Conditional Parameter Optimization solution needed! - Parabolic SAR - Expert Advisors and Automated Trading - MQL5 programming forum #4

 
William Roeder:

Don't double post! You already had that thread open.

          General rules and best pratices of the Forum. - General - MQL5 programming forum

I already answered your question there, six months ago Conditional Parameter Optimization solution needed! - Parabolic SAR - Expert Advisors and Automated Trading - MQL5 programming forum #4

Dear william 

thanks a lot for your time. i'm really grateful.

your solution will not work. since you really didn't understand the problem.

1- your solution returns a zero value for the pass. it will not eliminate it at the beginning.
 your solution will only make the optimization faster (not more accurate)

2- your solution is based on a false setting of parameters and not based on the passes that has bean calculated (or at the beginning, the passes that should be calculated)

3- it should be mentioned that there is no false pass. the optimizer should not recalculate unnecessary passes (since the result will be the same).
for example if the Psar is false the EA is not using it, so optimizing the related parameters will lead to nothing.

the reason that i re-posted the question is that, since that time so many new build of the program has bean released and i thought maybe something has changed or
maybe a real expert programmer could help.

thanks again for your response.

yours

 
  1. OnInit returns INIT_PARAMETERS_INCORRECT (zero) therefor the pass does not start. Accuracy is irrelevant. Your title is

    elimination of unwanted passes

    nothing about accuracy. Each pass is just as accurate as any other. You apparently just want to argue.
  2. The optimizer is only estimating the number of passes. Actual results may be less or more than double. You want to eliminate unwanted passes. That is not dependent on other passes. You apparently just want to argue.
  3. The optimizer does not "recalculate unnecessary passes" It has no idea what your parameters do. It is passing unique combinations to your code. Only you know that those combinations are essentially the same. So I showed you how I inform the optimizer. You apparently just want to argue.
 
Mohammad Hossein Najarzadeh:

Dear william 

thanks a lot for your time. i'm really grateful.

your solution will not work. since you really didn't understand the problem.

1- your solution returns a zero value for the pass. it will not eliminate it at the beginning.
 your solution will only make the optimization faster (not more accurate)

2- your solution is based on a false setting of parameters and not based on the passes that has bean calculated (or at the beginning, the passes that should be calculated)

3- it should be mentioned that there is no false pass. the optimizer should not recalculate unnecessary passes (since the result will be the same).
for example if the Psar is false the EA is not using it, so optimizing the related parameters will lead to nothing.

the reason that i re-posted the question is that, since that time so many new build of the program has bean released and i thought maybe something has changed or
maybe a real expert programmer could help.

thanks again for your response.

yours

Please don't waste people time. You are now saying that the recommended solution (as proposed by William) doesn't fit your needs, that's ok but you should have mentioned it from the start. Don't let people guess what you already have tried or envisaged.

That's not clear to me why the "INIT_PARAMETERS_INCORRECT" solution is not good for you ?

1° "solution returns zero", so what prevent you from using a custom function to return the value you want in such cases ?

2° That's how MT5 genetic optimization works, if a parameter is selected for optimization it participates to the algorithm. Obviously all optimized parameters are used to "calculate" the passes.

3° The optimizer has no way to know the internal logic of your EA. Statement like "the optimizer should not..." is not a serious way to tackle this issue. The optimizer work as it is and is documented, what you think it "should" do is totally irrelevant.

In summary :

If the parameters set you want to exclude from your optimization represent no more then 15% of the total combinations of parameters, there is no problem using INIT_PARAMETERS_INCORRECT solution.

If that's not the case you have to be creative and find a custom solution. Waiting the optimizer to do what you think it should do is not a solution, it's a wishful thinking. This topic has already discussed several times on this forum, with some possible solutions mentioned, up to you to find them. If you want "no false pass", then be creative and refactor your parameters to fit this specification.

PS: And please, please stop double posting and create new topic when several similar topics already exist.
 
Alain Verleyen:

Please don't waste people time. You are now saying that the recommended solution (as proposed by William) doesn't fit your needs, that's ok but you should have mentioned it from the start. Don't let people guess what you already have tried or envisaged.

That's not clear to me why the "INIT_PARAMETERS_INCORRECT" solution is not good for you ?

1° "solution returns zero", so what prevent you from using a custom function to return the value you want in such cases ?

2° That's how MT5 genetic optimization works, if a parameter is selected for optimization it participates to the algorithm. Obviously all optimized parameters are used to "calculate" the passes.

3° The optimizer has no way to know the internal logic of your EA. Statement like "the optimizer should not..." is not a serious way to tackle this issue. The optimizer work as it is and is documented, what you think it "should" do is totally irrelevant.

In summary :

If the parameters set you want to exclude from your optimization represent no more then 15% of the total combinations of parameters, there is no problem using INIT_PARAMETERS_INCORRECT solution.

If that's not the case you have to be creative and find a custom solution. Waiting the optimizer to do what you think it should do is not a solution, it's a wishful thinking. This topic has already discussed several times on this forum, with some possible solutions mentioned, up to you to find them. If you want "no false pass", then be creative and refactor your parameters to fit this specification.

PS: And please, please stop double posting and create new topic when several similar topics already exist.

Dear Alain

Sorry for re-posting the topic (I really am). As I've mentioned, i thought the new build of program might have changed some aspects.

but regarding your responses on "The optimizer has no way to know the internal logic of your EA" or "that's how MT5 genetic optimization works" 
or "Waiting the optimizer to do what you think it should do is not a solution, it's a wishful thinking." i have to remind you that what i'm searching for
 is a solution like the walking forward optimization. as you are aware there is no walking forward optimization in the program too.
but this has been done by Stanislav Korotky ( https://www.mql5.com/en/users/marketeer)  in the product( https://www.mql5.com/en/articles/3279 ),
this solution (programming skill) is not based on what is already in the terminal and it uses a very complicated coding to do something was once a dream (i guess).

so why can't be a solution for my case?!?! (and i really don't see it as a wishful thinking, and perhaps much much easier that the walking forward optimization)

anyway, once again i apologize if you considered your time wasted.

Custom Walk Forward optimization in MetaTrader 5
Custom Walk Forward optimization in MetaTrader 5
  • www.mql5.com
Algorithmic trading consists not only of planning and development of trading robots but also (to a greater extent) testing and verifying the survivability of ideas and algorithms implemented in them. MetaTrader 5 provides the built-in tester for optimizing Expert Advisors on historical data. This tool is often indispensable in everyday...
 
William Roeder:
  1. OnInit returns INIT_PARAMETERS_INCORRECT (zero) therefor the pass does not start. Accuracy is irrelevant. Your title is

    elimination of unwanted passes

    nothing about accuracy. Each pass is just as accurate as any other. You apparently just want to argue.
  2. The optimizer is only estimating the number of passes. Actual results may be less or more than double. You want to eliminate unwanted passes. That is not dependent on other passes. You apparently just want to argue.
  3. The optimizer does not "recalculate unnecessary passes" It has no idea what your parameters do. It is passing unique combinations to your code. Only you know that those combinations are essentially the same. So I showed you how I inform the optimizer. You apparently just want to argue.

Dear william

Sorry to bother you. my intention was not to argue (at all).

Maybe i'm mistaken. i'm quite an amature anyway.

but as i explained to Alain, what i was looking for was a solution like Walking forward optimization (something that is not in the terminal by default).
a solution that changes the way that the terminal works. (redefining the passes that should be searched within based on the a parameter being true or false)

anyway, once again i apologize if you considered your time wasted or if my writing was in a way that may you think that i like argues.

best regards

 
Mohammad Hossein Najarzadeh:

Dear Alain

Sorry for re-posting the topic (I really am). As I've mentioned, i thought the new build of program might have changed some aspects.

but regarding your responses on "The optimizer has no way to know the internal logic of your EA" or "that's how MT5 genetic optimization works" 
or "Waiting the optimizer to do what you think it should do is not a solution, it's a wishful thinking." i have to remind you that what i'm searching for
 is a solution like the walking forward optimization. as you are aware there is no walking forward optimization in the program too.
but this has been done by Stanislav Korotky ( https://www.mql5.com/en/users/marketeer)  in the product( https://www.mql5.com/en/articles/3279 ),
this solution (programming skill) is not based on what is already in the terminal and it uses a very complicated coding to do something was once a dream (i guess).

so why can't be a solution for my case?!?! (and i really don't see it as a wishful thinking, and perhaps much much easier that the walking forward optimization)

anyway, once again i apologize if you considered your time wasted.

I never said there is no solution, I said you have to create it (as Stanislav did) IF what is existing doesn't match your needs.

Now, you didn't answer why the INIT_PARAMETERS_INCORRECT solution isn't suitable for your needs.

 
Alain Verleyen:

I never said there is no solution, I said you have to create it (as Stanislav did) IF what is existing doesn't match your needs.

Now, you didn't answer why the INIT_PARAMETERS_INCORRECT solution isn't suitable for your needs.

thanks in advance for your effort and understanding

from my point of view (programming knowledge), the INIT_PARAMETERS_INCORRECT solution stops a pass after the OnInit function, based on the parameters of the very same pass.
this means that there is no way to find out which passes has already been calculated (since the pass should be killed only when all the other independent parameters are the same as a calculated pass), 
and determining whether this pass is needed to be calculated or killed.
plus even if there is a way to do so, the optimizer calculates a zero result for the pass which perhaps (correct me if i'm wrong pls) will misguide the next generation of the 
fast genetic based method.

I don't have a solid understanding on fast genetic base (FGB) method procedure, but assuming that at the beginning of the optimization the terminal generates a list of passes (does it?)
and creates the generations of optimizations based the that list (if such a list exists), then the best solution is to delete the unwanted passes from that list right after the creation of the
, so there won' be a need of determination that if the pass should be killed in every and each pass (perhaps this make the optimization much faster plus the  zero results won't be misguiding
for the remaining procedure of the FGB).

 
You keep concentrating on passes and the number of passes. What part of, it's only an estimate, was unclear? There is no "list," after the first hundred or so.
 
William Roeder:
You keep concentrating on passes and the number of passes. What part of, it's only an estimate, was unclear? There is no "list," after the first hundred or so.

Oh!

thanks for informing me on this. so there is no list. that makes things different.   :|

but there should be a list of what passes that has already been tested. maybe it can be used? right?

another question is that: would the position of my parameters (higher in the programming line or being lower) makes any difference in the procedure of fast genetic based (FGB) optimization?

is there any reference (document, article, etc) on how FGB of meta trader procedure is? (how it works, how to make it better, what are the critical lines , . . .)

yours

Reason: