Likely coding error scanning OrderHistory

New comment
 

Hi,

Rather than try to find one home run trading strategy, I've opted to find a collection of high probability but low gain strategies, the collection of which would be attractive.  I noticed different back test results when testing them singly and together several months ago, and parsed my code down to try and isolate the error assuming it was an error in my code. 

Whether it is an error in my code (my thought) or a weakness of the backtester, it appears to be associated either with my use searching through the order history to find the last open time for a trade of a given magic number. 


After some initial code my EA calls a sequence of individual trade signals.  

int start()
  {
      RefreshRates();      
      CheckZombieOrders();                           //VERIFY ALL ORDERS HAVE A VALID STOP LOSS AND CLOSE IF NOT
      CheckTrailingStops();                          //MOVE STOP LOSS IF MARKET MOVED

      if(!IS_TEST) store_ticks();                    //stores ticks in the array and updates the indices             
      Comment("Tick Count=", LastItem,"      BAS= ",iBAS(6), "   Server HR is -- ",TimeHour(TimeCurrent()));
      
      set_defaults();                                //set broker digits
      
      
      
      /******************ALL TEST RESULTS ON 1 YEAR TEST***********************/
      /******************CONFIRMED ON MULTIPLE YEARS******************************/
      RSI_Cross_H4();      //Tests a clean x.xx% on .005 risk 
      StochRetrace4H();    //Tests a clean x.xx% on .005 risk
      //SomeSignal();      //Tests a clean x.xx% on .005 risk
      //OtherSignal();     //Tests an uneven x.xx% on .005 risk
      
      return(0);
  }


At the beginning of each signal I check to see if enough time has elapsed since the last trade on that signal.  If I comment out all but one signal call then I get what "appears" to be the expected behavior.  If I uncomment one of the signals so it is called then the behavior changes--the expected delay between trades does not appear to be enforced and I can get multiple trades one after another as the preceding one closes out (I typically only allow one trade per signal in the same direction at the same time).  I require a delay between trades so as to not trade on a stale signal if I am attempting to trade on the four hour chart with the trade executing at the start of a bar and closing very soon after, and to prevent multiple trades executing on a stale signal after a stopped out trade. 

Here is the typical line of code that checks the delay.

int RSI_Cross_H4()
   {

      
      int magic = 501;                                                     //magic number assigned to this signal
      int my_period = GetTradeParameters(magic, mode_PERIOD);                                                  
      

      //Now get the allowed delay in seconds.  Check if the last open time 
      //was long enough ago to allow a new trade
      if(!CheckLastOpenTime(magic, GetTradeParameters(magic, mode_DELAY))) return(0);    //<-------------------------

      //more code follows of course....

   }


Here is how the variable mode_DELAY is returned.

double GetTradeParameters(int magic, int mode)   
   {    
      /*    This function returns the valid parameter.  The default is to return ZERO so
            for parameters such as returning hours to trade we enter the allowed hours here and
            if it is not listed then then hour will not be counted as a valid hour to trade
            as the return value will be ZERO.
       */
   
      switch(magic){
                case 501:  //RSI_CROSS H4
                  if(mode == mode_VALID_MAGIC_NUMEBR)       return(VALID);    //Indicates this is a Valid Magic Number
                  if(mode == mode_PS)                       return(0);        //Profit Stop
                  if(mode == mode_TS)                       return(1500);     //Trailing Stop where applies
                  if(mode == mode_TS_BASE)                  return(20);       //TS BASE
                  if(mode == mode_TS_FIRST)                 return(30);       //TS FIRST
                  if(mode == mode_TS_BURST)                 return(70);       //TS BURST
                  if(mode == mode_TS_STEP)                  return(2);        //TS STEP
                  if(mode == mode_PERIOD)                   return(PERIOD_H4);         //Allowed Chart Timeframe
                  if(mode == mode_DELAY)                    return(60*16*PERIOD_H4);   //Delay between trades   <-----
                  if(mode == mode_RISK)                     return(.005);
                  if(mode == mode_MAX_ORDERS)               return(2);      //Max simultaneous orders Long and Short
                  if(mode == mode_MAX_ORDERS_LONG)          return(1);      //Max simultaneous orders Long only
                  if(mode == mode_MAX_ORDERS_SHORT)         return(1);      //Max simultaneous orders Short only
	          //more code clipped out 

      }
   }


In this case the delay is expected to be PERIOD_H4 (or 240 minutes) * 60 seconds * 16 which as I understand it is 16 four hour bars or 64 hours. 

The mode_DELAY variable is returned as a DOUBLE (without a decimal) that is passed as an INT to the function that checks the last open time.  I check the time that the last trade of the relevant magic number was opened at the beginning of all the signals.


bool CheckLastOpenTime(int magic, int delay)
   {
      datetime now  = TimeCurrent();
      datetime then = LastOpenTime(magic);

      if(now - delay > then) return(true);         //ENOUGH TIME HAS PASSED SINCE LAST ORDER 
      return(false);
   }



datetime LastOpenTime(int magic){

    datetime LastTime  = 0;                    
    int      pos       = 0;
    
    //Need to select from both the open and closed pools
    //SOMETHING WRONG WITH THIS FUNCTION STILL ???!!!
    
    
    for(pos = OrdersTotal()-1; pos >= 0 ; pos--)                              //FIRST CHECK THE OPEN AND ACTIVE ORDERS
    {
      if(       OrderSelect(pos, SELECT_BY_POS, MODE_TRADES)                  // Only my orders w/
            &&  OrderMagicNumber()  == magic                                  // my magic number
            &&  OrderSymbol()       == Symbol()                               // and my pair.
            &&  OrderOpenTime()     >=  LastTime)
    
        {
            if(LastTime <= 0) LastTime   = OrderOpenTime();
            if(OrderOpenTime() > LastTime) LastTime = OrderOpenTime();
            
        }
    }

    //Now check the orderhistory for closed orders
    int position = OrdersHistoryTotal()-1;
    if(position > 100) position = 100;                              //If more than 100 orders in the history then only check the most recent 100 
    for(pos = position; pos >= 0 ; pos--)                           
    {
      if(      OrderSelect(pos, SELECT_BY_POS, MODE_HISTORY)                  // Only my orders w/
            &&  OrderMagicNumber()  == magic                                  // my magic number
            &&  OrderSymbol()       == Symbol()                               // and my pair.
            &&  OrderOpenTime()     >=  LastTime)
    
        {
            if(LastTime <= 0) LastTime   = OrderOpenTime();
            if(OrderOpenTime() > LastTime) LastTime = OrderOpenTime();
            
        }
    }

    if(LastTime < 0) Alert("LastTime < 0");
    return(LastTime); 
}


As I review the back test results it is as if when I add the second signal then there is no prior trade of the first signal's magic number found in the order/order history.  I can see a potential gotcha with my assumption that I would only need to check the most recent 100 orders in the order history as I might be able to compile enough small signals for that to not be enough.  In this case neither signal trades over 100 times in the one year backtest, so I remain stumped. 

I've read that people have struggled some with TimeCurrent() in the back tester but I think the error is in my code in my search of the Order Histories..... I'm just not seeing it.

I do apologize for the longer question, but would be grateful for some assistance from those with keener eyes than mine,

Kind regards,

Nathan

 
  1. niqmadu: with my assumption that I would only need to check the most recent 100 orders in the order history as I might be able to compile enough small signals for that to not be enough.  In this case neither signal trades over 100 times in the one year backtest, so I remain stumped. 
    You assume that history is ordered.
    dabbler: Order History is nice in that it is there for free. It is unreliable in the sense that the order of entries is mysterious (by actual test)Could EA Really Live By Order_History Alone? (ubzen) - MQL4 forum
  2.             if(LastTime <= 0) LastTime   = OrderOpenTime();
            if(OrderOpenTime() > LastTime) LastTime = OrderOpenTime();
    No need for the first line. No need for the if() on the second line, as you already did it as part of the if(OrderSelect.
  3. In the history loop you need to filter out non-trades (deleted pending orders, credits and balance adjustments.)
  4.     int position = OrdersHistoryTotal()-1;
    if(position > 100) position = 100;      //If more than 100 orders in the history then only check the most recent 100 
    for(pos = position; pos >= 0 ; pos--)  
    This code doesn't check the 100 most recent ([OHT-1 .. OHT-100],) it checks the oldest 101 ([100 .. 0].)
 
WHRoeder:
  1. niqmadu: with my assumption that I would only need to check the most recent 100 orders in the order history as I might be able to compile enough small signals for that to not be enough.  In this case neither signal trades over 100 times in the one year backtest, so I remain stumped. 
    You assume that history is ordered.
    dabbler: Order History is nice in that it is there for free. It is unreliable in the sense that the order of entries is mysterious (by actual test)Could EA Really Live By Order_History Alone? (ubzen) - MQL4 forum
  2. No need for the first line. No need for the if() on the second line, as you already did it as part of the if(OrderSelect.
  3. In the history loop you need to filter out non-trades (deleted pending orders, credits and balance adjustments.)
  4. This code doesn't check the 100 most recent ([OHT-1 .. OHT-100],) it checks the oldest 101 ([100 .. 0].)


WHR -- Thanks so much for the code review and thoughts, very clear and to the point.  I will review and revise.

Kind regards,

N

 
niqmadu:


WHR -- Thanks so much for the code review and thoughts, very clear and to the point.  I will review and revise.

Kind regards,

N




I can see the very real problems with my assumption that order history is in any given order, and I had completely omitted the fact that all the ordermodify results from my adjustment of the trailing stops would be in there as well, which would blow the history past my expected threshold of 100 quite easily.   Then there is the question of how long or short the order history might be set to be.... 

I think my preferred solution would be to simply write to a file for each magic number and track the last open time of the respective order.  The only concern I can think of on initial review would be the latency of reading multiple files (albeit very short ones) in a single EA.  My guess is that the operating system should be able to keep it in RAM and not have to seek to the hard drive in most cases, alleviating that issue.

If I am off base in my thought process I certainly would be grateful for the feedback,

Many thanks to all those who contribute to the forum,

N

 

and I had completely omitted the fact that all the ordermodify results from my adjustment of the trailing stops would be in there as well,

They are not stored in the order history

New comment
Reason: