Backtesting Control Points - What's the calculation ? What's the solution ?

RussB 2007.06.18 23:32

I'm very puzzled. I programmed this awesome expert advisor that produces the results on this image. Details can be viewed on the attached file ...

I've been testing this on almost every currency pair that MT4 has. I'm using the software as provided by InterbankFX. My strategy is similar to scalping, however I'm using the technical indicators in a certain way that will allow for a "sure" exit strategy that does not count on "profit" being greater than (>) $1.

My problem however is this ...

The demonstrated performance only works when tested on the "control point" model. If this strategy is ever going to be used live it would be wise to back-test on the "every tick" model. Having done so, the expert advisor seems to crash every time. I mean that the performance as demonstrated above will ultimately lead to failure. Please take my code and test for yourself.

I'm asking for anywone who would know of a solution for a module that I could add to my program that will resemble the control point model. It will be necessary for the back-test conducted on "every-tick" to equal the back-test conducted on "control points".

I've reviewed almost every article I could find on the MQL4 forums, but the answer I'm looking for still eludes me.

Customized Indicator:

//+------------------------------------------------------------------+
//|                                              ExpertIndicator.mq4 |
//|                      Copyright © 2007, MetaQuotes Software Corp. |
//|                                        https://www.metaquotes.net/ |
//+------------------------------------------------------------------+
#property copyright "Copyright © 2007, MetaQuotes Software Corp."
#property link      "https://www.metaquotes.net/"
 
#property indicator_chart_window
 
#property indicator_buffers 3
#property indicator_color1 Orange
#property indicator_width1 2
#property indicator_color2 Red
#property indicator_width2 2
#property indicator_color3 Green
#property indicator_width3 2
//---- buffers
double MovingAverage[];
double UpperEnvelope[];
double LowerEnvelope[];
 
 
extern int DeviationPeriod = 3;
extern int EnvelopePeriod = 5;
extern int EnvelopeLookback = 3;
extern double EnvelopeDeviation = 0.2;
 
 
//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int init()
  {
   
//---- indicators
   SetIndexStyle(0,DRAW_LINE);
   SetIndexBuffer(0,MovingAverage);
   SetIndexLabel(0, "Deviation");
   SetIndexStyle(1,DRAW_LINE);
   SetIndexBuffer(1,UpperEnvelope);
   SetIndexLabel(1, "Upper Envelope");
   SetIndexStyle(2,DRAW_LINE);
   SetIndexBuffer(2,LowerEnvelope);
   SetIndexLabel(2, "Lower Envelope");
//----
   return(0);
  }
//+------------------------------------------------------------------+
//| Custom indicator deinitialization function                       |
//+------------------------------------------------------------------+
int deinit()
  {
//----
   
//----
   return(0);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int start()
  {
   int    counted_bars=IndicatorCounted();
//----
 
//----   
   for(int i = Bars; i >= 0; i--) 
     {              
       MovingAverage[i] = iMA(NULL, 0, DeviationPeriod, 0, MODE_EMA, PRICE_CLOSE, i);        
       UpperEnvelope[i] = iEnvelopes(NULL, 0, EnvelopePeriod, MODE_SMA, 0, PRICE_CLOSE, EnvelopeDeviation, MODE_UPPER, i + EnvelopeLookback);  
       LowerEnvelope[i] = iEnvelopes(NULL, 0, EnvelopePeriod, MODE_SMA, 0, PRICE_CLOSE, EnvelopeDeviation, MODE_LOWER, i + EnvelopeLookback);  
     }
//----
   return(0);
  }
//+------------------------------------------------------------------+

Expert Advisor:

//+------------------------------------------------------------------+
//|                                                TradingExpert.mq4 |
//|                      Copyright © 2007, MetaQuotes Software Corp. |
//|                                        https://www.metaquotes.net/ |
//+------------------------------------------------------------------+
#property copyright "Copyright © 2007, MetaQuotes Software Corp."
#property link      "https://www.metaquotes.net/"
 
 
extern double Lots = 1;
extern int StopLoss = 20;
 
 
extern int DeviationPeriod = 3;
extern double DeviationPressure = 0.5;
extern int EnvelopePeriod = 5;
extern int EnvelopeLookback = 3;
extern double EnvelopeDeviation = 0.02;
 
 
 
//+------------------------------------------------------------------+
//| expert initialization function                                   |
//+------------------------------------------------------------------+
int init()
  {
//----
   
//----
   return(0);
  }
//+------------------------------------------------------------------+
//| expert deinitialization function                                 |
//+------------------------------------------------------------------+
int deinit()
  {
//----
   
//----
   return(0);
  }
//+------------------------------------------------------------------+
//| expert start function                                            |
//+------------------------------------------------------------------+
int start()
  {
//----
 
   double deviation = iCustom(NULL, 0, "ExpertIndicator", DeviationPeriod, EnvelopePeriod, EnvelopeLookback, EnvelopeDeviation, 0, 0);
   double upper = iCustom(NULL, 0, "ExpertIndicator", DeviationPeriod, EnvelopePeriod, EnvelopeLookback, EnvelopeDeviation, 1, 0);
   double lower = iCustom(NULL, 0, "ExpertIndicator", DeviationPeriod, EnvelopePeriod, EnvelopeLookback, EnvelopeDeviation, 2, 0);
   
   
   for (int i = 0; i < OrdersTotal(); i++)
   {
      OrderSelect(i, SELECT_BY_POS);
      
      if (OrderType() == OP_BUY && OrderClosePrice() > deviation && OrderProfit() > 0) 
      {
         OrderClose(OrderTicket(), OrderLots(), OrderClosePrice(), 0, 0);
      }
      
      if (OrderType() == OP_SELL && OrderClosePrice() < deviation && OrderProfit() > 0) 
      {
         OrderClose(OrderTicket(), OrderLots(), OrderClosePrice(), 0, 0);
      }
   }
   
    
   
   static datetime last_trade = NULL;
   if (last_trade == Time[0]) return (0);
   
   
   
   double sl = 0;
   
   if (Close[0] < lower && deviation > lower + ((upper - lower) * DeviationPressure))
   {
      if (StopLoss > 0) sl = Ask - StopLoss * Point;
      OrderSend(Symbol(), OP_BUY, Lots, Ask, 0, sl, 0, "", 0, 0, Blue);
      last_trade = Time[0];
   }
   else if (Close[0] > upper && deviation < upper - ((upper - lower) * DeviationPressure))
   {
      if (StopLoss > 0) sl = Bid + StopLoss * Point;
      OrderSend(Symbol(), OP_SELL, Lots, Bid, 0, sl, 0, "", 0, 0, Red);
      last_trade = Time[0];
   }
   
   
   
 
 
//----
   return(0);
  }
//+------------------------------------------------------------------+

ps. Please, take my code and see for yourself. I'm expecting a lot of responses to this thread!

zap 2007.06.19 08:12 #1

I don't want to ruin your happiness here, but you should big a bit deeper in understanding the back tester.

My most important point is: bars of a chart are nothing else, but arbitrary simplifications of the only one thing that matters: PRICE. When you make H1 bars from price flow, you compress data: you take only 2 real points (open and close, which are even too close) and 2 relative (high and low. I call these relative, because you cannot really tell succession of ticks: you don't know which was sooner: High or Low?). You throw the rest of the data away. BUT! You analyze the very lossy compressed format, and still trade the PRICE in real time. For some questionably good purpose (like speed), you can BACKTEST the LOSSY FORMAT altough you can only trade PRICE. Control point backtesting does nothing, but ignores roughly 96% of the data, which you CANNOT DO REAL TIME. This is a trap, holding out carrot with you, call it what you want...
This causes the too good results, not the smart strategy. It loses in tick mode, it will lose a lot more real time.

I write it down what the default MT4 backtester settings do:

OHLC mode: takes the O,H,L,C of the bars, and runs the EA once at every point. You cannot simulate real time trading with this at all, as you will never know where is the high and low of your current timeframe, when you are real time. If you have an EA that analyzes data and trades only at the close of a bar, this method is enogh. (I've never seen such, working EA though)

Control Point Mode: comes a _bit_ closer to reality. Not only four, but 12 points are made, based on the closest less timeframe. If you trade H1 chart, it will build the 12 control points based on how the M30 chart looks like. You will get a _very_ rough price flow simulation here. You will know the _approximate_ DIRECTION how the price moved, but no real data at all. This is not good for anything. If you have an EA that trades on the only 2 sure points (O/C) OHLC mode is enough, but this rough estimate won't be enough for price sensitive EAs. Advantage? Not that I can think of, maybe speed. But this is a very bad excuse for a thing that can lose money.

Every tick mode: It is the most accurate, but still not enough for REAL backtesting. Lets assume we have all frame data, and we trade H1 charts. First it checks the amount of ticks on the current bar, so it will know huw much REAL POINTS (that is ticks) it should fill with data(around between 100s to 1000s per hour). Then as it doesn't have all the necessary data (that is tick data) at hand, it takes the smallest timeframe data, and tries to guess (based on a not known pattern) how the ticks could have been. Well, why it doesn't really work: even the smallest timeframe is lossy format compressed price data, and iwe want to unzip it. Oh if it could be done, the IT world would have such a boom, world have never seen. It is like skipping about 8 characters (including spaces) out of 10 in a book, and then giving it to someone, to restore the content from it. It can be done more or less, but if I add, that you CAN LOOSE MONEY on a word that is missing or not correnct, would you try it? Tlt tuscd'k (You could try to decompress it if you wanna know the answer;)

So you could get unrealistic results even with every tick model (unless you don't have real historical tick data).
On the other hand, it is possible to get pretty decent accuracy results with historical tick. I can tell more via email (forexzap<at>gmail<dot>com) if anyone is interested.

BeckTest EA Perfect, Real 9.2 Billion profit in 1minute OHLC vs every

RussB 2007.06.20 00:49 #2

The code I previously posted was based on the timespan within a bar. Are the control points actually based on price / 12 ? I mean, (High - Low) / 12 ?

Once we have established this interval we then apply it so that it only becomes activated when the tick movements crosses from one interval to another ?

For control points to work, would the data be taken from the previous "completed" bar on the chart ? Or, would the data be taken from the current bar and the calculation is updated for every time the High and Low distance on the current bar changes ?

Let me know ...

MACD indicator Only look at data Need a simple warning

Point Zero 2013.09.26 08:28 #3

Control Point Mode: comes a _bit_ closer to reality. Not only four, but 12 points are made, based on the closest less timeframe. If you trade H1 chart, it will build the 12 control points based on how the M30 chart looks like. You will get a _very_ rough price flow simulation here. You will know the _approximate_ DIRECTION how the price moved, but no real data at all. This is not good for anything. If you have an EA that trades on the only 2 sure points (O/C) OHLC mode is enough, but this rough estimate won't be enough for price sensitive EAs. Advantage? Not that I can think of, maybe speed. But this is a very bad excuse for a thing that can lose money.

Actually control points is quite good if you trade using HLOC values every new bar, or, say 10 minutes before the bar close. I have found it quite reliable in my coding.

High[] and Low[] question Trade an EA on The time at which

Simon Gniadkowski 2013.09.26 08:42 #4

flaab:
Actually control points is quite good if you trade using HLOC values every new bar, or, say 10 minutes before the bar close. I have found it quite reliable in my coding.

Why are you replying to a 6 year old thread ?

Thread start date - 2007.06.19

Randomtrader 2023.01.25 19:28 #5

Simon Gniadkowski #:

Why are you replying to a 6 year old thread ?

Thread start date - 2007.06.19

Hehe. Today we are writing 2023 and the answer is still valuable for me :)

New comment