Download MetaTrader 5
To add comments, please log in or register
Find out how to add a video in the messages editor
MTcrafter
57
MTcrafter 2016.01.09 09:21 

Hi,

The following EA was a sample I downloaded attempting to test Bar Open Only processing that operates 100% dependent on the platform during operation.

That is to say it should not have any dependency on sending or operating on tick files or broker sent limit orders, only market order action on bar open.

During optimize testing I began to notice the number of passes, rejections and cache file stats were not remaining stable.

This of course meant the results of the optimize table also were not reliable from pass to pass.

When I modified a copy of the native MacD sample EA to do Bar Open Only, I did not find this problem to occur.

I'm wondering if anyone can spot why this particular code assembly might be errant or otherwise triggering a hidden bug in the platform.

The code below relies on bar counting methods to trigger in and out of the market.  It does use a time based compare to operate only on bar data.

I tried multiple Mt4 installs and also tried it during market hours closed, it consistently will not repeat the same result.


Simply compile it, choose a few criteria to optimize and see if the same occurs for you or if anything points to a cause in code.  I suspect it may have to do with bar count referencing but I cannot see how or why that would alter the optimizer results unless something is causing the optimizer to not initialize consistently.  In all reality it should not even need calls to the tickwise cache file since it makes no tickwise references I can see save that the Start() command is likely tied to the tick engine thread as a dependency.  There's very few places that expose any precision variations that might stumble and even if compares are unreliable in precision it should still be consistent from pass to pass comparisons.  The spread value in 600+ versions is a fixed value and also should not have a dynamic effect during optimize, especially when testing while the market is down, so I'm fairly duped in what is causing the variation.

Thanks for any input or ideas,


extern double Lots = 0.1;
extern double Slippage = 5;
extern int BarsToLookUp = 8;
extern int MinDiff = 3;
extern int MaxDuration = 5;

extern bool TradeOnSundays = false;
extern bool TradeOnMondays = false;
extern bool TradeOnTuesdays = true;
extern bool TradeOnWednesdays = true;
extern bool TradeOnThursdays = true;
extern bool TradeOnFridays = false;
extern bool TradeOnSaturdays = false;

int Magic;
int Duration = 0;
int LastBars = 0;
int OT;

int init()
{
   Magic = Period()+59172491;
   return(0);
}

//+------------------------------------------------------------------+
//| Random entry expert advisor                                      |
//+------------------------------------------------------------------+
int start()
{
   if (AccountFreeMargin() < (Lots*2*1000))
   {
      //Print("Free margin level is too low.");
      return(0); 
   }

  // if (Lots < 0.01)
   //{
     // Print("Lots below 0.01.");
     // return(0);
   //}

   //Wait for the new Bar in a chart.
   static datetime Time0;
   if (Time0 == Time[0]) return(0); Time0 = Time[0];


   int count = 0;
   int total = OrdersTotal();
   for (int pos = 0; pos < total; pos++)
   {
      if (OrderSelect(pos, SELECT_BY_POS) == false) continue;
      if ((OrderMagicNumber() == Magic) && (OrderSymbol() == Symbol()))
      {
         OT = OrderTicket();
         count = 1;
         Duration++;
      }
   }
   
   if (Duration == MaxDuration)
   {
      CloseOrder(OT);
      count = 0;
   }
   

   //Let's check if the last bar was highest high or lowest low in BarsToLookUp last bars
   int H = 0;
   int L = 0;
   for (int i = 2; i <= (BarsToLookUp+1); i++)
   {
      if ((High[1] - High[i]) > MinDiff*Point) H++;
      if ((Low[i] - Low[1]) > MinDiff*Point) L++;
   }
   
   //Trade only on given days - Tuesdays, Wednesdays and Thursdays by default:
   if ((DayOfWeek() == 0) && (!TradeOnSundays)) return(0);
   if ((DayOfWeek() == 1) && (!TradeOnMondays)) return(0);
   if ((DayOfWeek() == 2) && (!TradeOnTuesdays)) return(0);
   if ((DayOfWeek() == 3) && (!TradeOnWednesdays)) return(0);
   if ((DayOfWeek() == 4) && (!TradeOnThursdays)) return(0);
   if ((DayOfWeek() == 5) && (!TradeOnFridays)) return(0);
   if ((DayOfWeek() == 6) && (!TradeOnSaturdays)) return(0);
   
   if ((H == BarsToLookUp) && (L == BarsToLookUp))
   {
      if (count > 0) return(0);
      if (Close[1] >= Open[1]) fBuy();
      else fSell();
   }
   else if (H == BarsToLookUp)
   {
      if ((count > 0) && (OrderType() == OP_SELL)) CloseOrder(OT);
      if (!((count > 0) && (OrderType() == OP_BUY))) fBuy();
   }
   else if (L == BarsToLookUp)
   {
      if ((count > 0) && (OrderType() == OP_BUY)) CloseOrder(OT);
      if (!((count > 0) && (OrderType() == OP_SELL))) fSell();
   }
   return(0);
}

//+------------------------------------------------------------------+
//| Buy                                                              |
//+------------------------------------------------------------------+
void fBuy()
{
        RefreshRates();
        int result = OrderSend(Symbol(),OP_BUY,Lots,Ask,Slippage,0,0,"BreakPicker",Magic,0); 
        if (result == -1)
        {
                int e = GetLastError();
                Print(e);
        }
        Duration = 0;
}

//+------------------------------------------------------------------+
//| Sell                                                             |
//+------------------------------------------------------------------+
void fSell()
{
        RefreshRates();
        int result = OrderSend(Symbol(),OP_SELL,Lots,Bid,Slippage,0,0,"BreakPicker",Magic,0);
        if (result == -1)
        {
                int e = GetLastError();
                Print(e);
        }
        Duration = 0;
}

//+------------------------------------------------------------------+
//| Close all orders opened by this EA                               |
//+------------------------------------------------------------------+
void CloseOrder(int ot)
{
   RefreshRates();
   if (OrderType() == OP_BUY) OrderClose(ot, Lots, Bid, Slippage);
   if (OrderType() == OP_SELL) OrderClose(ot, Lots, Ask, Slippage);
   Duration = 0;
}

int OpenOrders(int nOrderType, string strSymbol, int nMagic)
{
   int nOrderCount=0;
   for (int i=OrdersTotal()-1 ; i>=0 ; i--)
   {
      if (!OrderSelect(i,SELECT_BY_POS)) continue;
      if (OrderType() == nOrderType)
         if (OrderMagicNumber() == nMagic)
            if (OrderSymbol() == strSymbol)
               nOrderCount++;
   }
   return(nOrderCount);
}  
MTcrafter
57
MTcrafter 2016.01.09 11:33  

Update:

After further testing I did find that turning off the Genetic Analysis option seems to have eliminated the problem, (so far).

Possibly this means the Genetic processing algo uses a random seed generator changing per initialization?

My concern is that the Genetic Algo is a huge convenience on larger optimize passes.  If there is an odd / underlying platform bug that might trigger the variation within Genetics, I'm a bit concerned to develop against the unknown if it means being stuck working without Genetic reduction being stable.

Still hopeful others might find same and comment if this is an undisclosed bug or bad code practices.

Thanks more.

ffoorr
985
ffoorr 2016.01.09 12:24  

OrderSelect() is missing, so the EA is not properly coded,

This is meaningless (?) , duration is the number of  orders

for (int pos = 0; pos < total; pos++)
   {
      if (OrderSelect(pos, SELECT_BY_POS) == false) continue;
      if ((OrderMagicNumber() == Magic) && (OrderSymbol() == Symbol()))
      {
         OT = OrderTicket();
         count = 1;
         Duration++;
      }
   }


but it will work because the EA will enter into those loop once per bar, and duration is declared on a global scope,

and only one order is taken, from what I understand

MTcrafter
57
MTcrafter 2016.01.09 14:37  

Thank you @ ffoorr,

I don't see that "OrderSelect()" is missing... it is in the first line of the loop you noted above.

The false condition iterates the loop looking for orders belonging to this EA's Magic on this pair.

As it finds a matching order the ticket of the single order is given to "OT", count is updated or re-updated to 1 and Duration incremented one bump greater.

Seems Duration would be a bar counter of how many bars the given order has been present, I believe.

Duration appears to be global as it's referenced globally in various functions declared below this point of the code.

I don't see how this should cause a problem in the genetic processor with how the code is written however.

All these variables should be reinitialized each time the optimizer runs AFAIK

Yes only one order is processing at a time that I have seen.

ffoorr
985
ffoorr 2016.01.09 14:54  
MTcrafter:

Seems Duration would be a bar counter of how many bars the given order has been present, I believe.


  That's it, duration count the number of Bars, not the time

it should be like that to simplify the code and make it readable :


 int order  0;
for (int pos = 0; pos < total; pos++)
   {
      if (OrderSelect(pos, SELECT_BY_POS) == false) continue;
      if ((OrderMagicNumber() == Magic) && (OrderSymbol() == Symbol()))
      {
       // should be added the orderType, to be used later on      
        tip = OrderType();
        OT = OrderTicket();
         count = 1;
         order++;
      }
   }

 if( order > 0) Duration++;
 else           Duration = 0;


OrderSelect () is missing all along the code

  else if (H == BarsToLookUp)
   {
      if ((count > 0) && (OrderType() == OP_SELL)) CloseOrder(OT);
      if (!((count > 0) && (OrderType() == OP_BUY))) fBuy();
   } 


 Inside this loop, what will the OrderType(), be refered as ?

 Then better no confusing code :

 

 else if (H == BarsToLookUp)
   {
      if ((count > 0) && tip== OP_SELL)) CloseOrder(OT);
      if (!((count > 0) && tip == OP_BUY))) fBuy();
   }


Or :

for (int pos = 0; pos < total; pos++)
   {
      if (OrderSelect(pos, SELECT_BY_POS) == false) continue;
      if ((OrderMagicNumber() == Magic) && (OrderSymbol() == Symbol()))
      {
 if ((H == BarsToLookUp) && (L == BarsToLookUp))
   {
      if (count > 0) return(0);
      if (Close[1] >= Open[1]) fBuy();
      else fSell();
   }
 else if (H == BarsToLookUp)
   {
      if ((count > 0) && (OrderType() == OP_SELL)) CloseOrder(OT);
      if (!((count > 0) && (OrderType() == OP_BUY))) fBuy();
   }
   else if (L == BarsToLookUp)
   {
      if ((count > 0) && (OrderType() == OP_BUY)) CloseOrder(OT);
      if (!((count > 0) && (OrderType() == OP_SELL))) fSell();
   }

MTcrafter
57
MTcrafter 2016.01.09 15:56  

Thank you @ ffoorr,


Ok, so you are noting the Order structure is analyzed at parts of the program where Select() has not first been used at the given point of the tick event progress through the code.

This would suggest the original coder is perhaps trusting the order state has not changed since last Select() ran, possibly for efficiency, otherwise frank run-time errors should result.

I need to check if this is the case and if the Select() function MUST always be run explicitly per event, (recalling the only event is a new bar in this code).

I admit I have used this "cheat" myself in the past, so long as the scope remained.

Yes, code clarity if nothing else.


I have also found additional conditions of this odd behavior...

When only 1 variable is being optimized it seems there is never a problem and even frequently on two "simple" variables at fairly high step count iterations.

HOWEVER, when the number of iterations per step increases or a third or more variables are optimized, the cache file begins to reflect the odd behavior.

This may also be true when the variables concerned are higher precision or rely on greater far calls within the compiler and resulting variable type complexity.

1) Running the optimizer twice or more will sometimes stabilize this effect on lower iteration counts, assuming it is flushing the cache file and second pass uses the newly conditioned cache file redundantly when no changes in profile are made.

2) Running the optimizer on 3 or more variables begins to cause inconsistent results continuously, even on smaller step iterations, at least on the current code set observed.

3) It is not uncommon for application coders doing genetic design to rely on a randomized seed as a mean sampling threshold for comparison in the algo.  This may be a feature as a permitted inconsistency leading to this effect.

4) I'm running an over-clocked, multi-core unlocked processor on Win 10 Windows optimized instruction set.  There have been instances noted where core scheduling on the cpu / mobo can have conflicts in optimization algos which may be especially true if there has been key assembly code development done in the genetic algo's design for highest efficiency.  This may be a stretch which can only be confirmed by reducing the windows version or worse running in a single core machine to confirm it.  If this is the case it would be self defeating as single core machines rarely will keep pace today with higher demands of optimization, short of dangerously high over-clocked configurations.  It may be a call for virtualized agents to distribute the task but the difficulty of establishing reliability in that manner would become even more difficult if there was continued inconsistency.


In most cases I'm finding those who rely heavily on Genetic Algos for EA optimization, recommend using the Genetic Algo ONLY for getting the gross number of iterations and / or variable counts down to lower values (centering and limiting).  After such efforts, some will then optimize WITHOUT the Genetic option to more accurately finish the finer resolution of optimization.  This would make sense given any degree of variation the GA module might cause, whether as a bug or a permitted feature.  It is perhaps that this has not been well identified or discussed by MQ engineering for us, or at least as far as I have seen yet in prior searches, which may have changed further as of Mt5 and again Mt4 600+ build.  In my past experience, this fine of a reduction in optimization invites increased curve fitting except in well controlled conditions, so this remains an area of preference and some opinion among many.

Possibly others or MQ will comment to clarify further yet in time here.

Thank you again !!

ffoorr
985
ffoorr 2016.01.09 16:09  

I don't kown much about genetic algorithm, this cannot be the cause of an error, as far as I understand,  my inglish not very good ;-)


OrderSelect() is a fonction which select Orders, it cannot be used once for all, at the beginning of an EA. Once the fonction is closed with a brace, one cannot know what OrderTicket() or OrderType(), belong to (the actual order, the last order or a ramdom  order.)

 but as there is only one order open, maybe the OrderSelect() is not needed, not sure of that, it should be tested.

/
To add comments, please log in or register