Filtering Signals Based on Statistical Data of Price Correlation

How This Began

The idea that led to writing this article appeared after I had read the book by Larry Williams "Long-Term Secrets to Short-Term Trading", in which the world record holder in investments (during 1987 he increased his capital by 11,000%) is completely dispelling the myths by "... college professors and other academics, who are rich in theory and poor in knowledge of the market..." about the absence of any correlation between the past behavior of prices and the future trends.

If you toss a coin 100 times, 50 times it will fall up heads and 50 times - tails. With each successive toss, the probability of heads is 50%, the same as of tails. The probability does not change from toss to toss, because this game is random and has no memory. Suppose the markets are behaving like a coin, in a chaotic manner.

Consequently, when a new bar appears, a price has equal opportunity to go up or down, and the previous bars do not affect even the slightest way the current one. Idyll! Create a trading system, set the take profit larger than the stop loss (i.e., set the math. expectation to the positive zone), and the trick is done. Simply breathtaking. However, the problem is that our assumption about the behavior of the market is not quite true. Frankly speaking, it's absurd! And I will prove it.

Let's create an Expert Advisor template using the MQL5 Wizard and by using simple alphanumeric interventions, present it in a condition suitable for the fulfillment of the task. We will encode an Expert Advisor to simulate buy that follows one, two and three bars closed down. Simulation means that the program will simply remember the parameters of analyzed bars. Sending orders (a more usual way) in this case will not work, because the spreads and swaps are able to question the reliability of the information received.

Here is the code:

//+------------------------------------------------------------------+
//|                                                     explorer.mq5 |
//|                        Copyright 2011, MetaQuotes Software Corp. |
//|                                              https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2011, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"
//---Variables---
double profit_percent,open_cur,close_cur;
double profit_trades=0,loss_trades=0,day_cur,hour_cur,min_cur,count;
double open[],close[];
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
   return(0);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
/* Calculate percent of closures with increase from the total number */
   profit_percent=NormalizeDouble(profit_trades*100/(profit_trades+loss_trades),2);
   Print("Percent of closures with increase ",profit_percent,"%");   // Enter data to the Journal
  }
//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
//---find out the time---
   MqlDateTime time;                        // Create a structure to store time
   TimeToStruct(TimeCurrent(),time);         // Structuring the data
   day_cur=time.day_of_week;              // Receive the value of the current day
   hour_cur=time.hour;                    // Receive the current hour
   min_cur=time.min;                      // Receive the current minute
//---Find out the prices---
   CopyOpen(NULL,0,0,4,open);ArraySetAsSeries(open,true);
   CopyClose(NULL,0,0,4,close);ArraySetAsSeries(close,true);

   if(close[1]<open[1]/*&&close[2]<open[2]&&close[3]<open[3]*/ && count==0) // If it closed with a loss
     {
      open_cur=open[0];                   // Remember open price of the current bar
      count=1;
     }
   if(open_cur!=open[0] && count==1)      // The current bar has closed
     {
      close_cur=close[1];                 // Remember the close price of the formed bar
      count=0;
      if(close_cur>=open_cur)profit_trades+=1;  // If the close price is higher than open,
      else loss_trades+=1;                      // +1 to closures with profit, otherwise +1 to closures with loss
     }
  }
//+------------------------------------------------------------------+

The test will be carried out on EUR/USD, on the interval from January 1, 2000 to December 31, 2010:

Figure 1. The percentage of closures with the increase

(The first column shows data for the whole period, the second, third and fourth - after a single, double and triple closing down)

That is what I was talking about! The previous bars have a fairly significant impact on the current one, because the price is always seeking to win losses back.

Another step forward

Great! Once we are sure that the behavior of prices is not accidental, we should use this amazing fact as soon as possible. Of course, it is not enough for an independent trading system, but it will be a fine tool that can free you from the tedious and often erroneous signals. Let's implement it!

So that's what we need:

1. A self-trading system, showing positive results at least for the last year.
2. Some amusing example that confirms the presence of correlations in the behavior of prices.

I found a lot of useful ideas in the book by L. Williams. I will share one of them with you.

The TDW (Trade Day Of Week) strategy. It will allow us to see what will happen if some of the days of the week we will only buy, and the other ones - open only short positions. After all, we can assume that the price within one day grows in a larger percent of cases than within the other. What's the reason for that? The geopolitical situation, macroeconomic statistics, or, as written in the book by A. Elder, Monday and Tuesday are the days of laymen, while Thursdays and Fridays are the time of professionals? Let's try to understand.

First, we will only by each day of the week, and then only sell. At the end of the study we will match the best results, and this will be a filter to our trading system. By the way, I have a couple of words about it. It is a pure classic!

The system is based on two MAs and MACDake. Signals:                                                            

1. If the fast moving average crosses the slow one from the bottom up and the MACD histogram is below the zero line, then BUY.
2. If the fast moving average crosses the slow one from upside down and MACD is above zero, then SELL. 

Exit a position using a trailing stop from one point. The lot is fixed - 0,1. 

 For the purpose of convenience, I've placed the Expert Advisor class in a separate header file:

//+------------------------------------------------------------------+
//|                                                       moving.mqh |
//|                        Copyright 2011, MetaQuotes Software Corp. |
//|                                              https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2011, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
//+------------------------------------------------------------------+
//| Класс my_expert                                                  |
//+------------------------------------------------------------------+
class my_expert
  {                                                  // Creating a class
   // Closed class members
private:
   int               ma_red_per,ma_yel_per;          // Periods of MAs
   int               ma_red_han,ma_yel_han,macd_han; // Handles
   double            sl,ts;                          // Stop orders
   double            lots;                           // Lot
   double            MA_RED[],MA_YEL[],MACD[];       // Arrays for the indicator values
   MqlTradeRequest   request;                         // Structure of a trade request
   MqlTradeResult    result;                          // Structure of a server response
                                                    // Open class members   
public:
   void              ma_expert();                                   // Constructor
   void get_lot(double lot){lots=lot;}                               // Receiving a lot  
   void get_periods(int red,int yel){ma_red_per=red;ma_yel_per=yel;} // Receiving the periods of MAs
   void get_stops(double SL,double TS){sl=SL;ts=TS;}                  // Receiving the values of stops
   void              init();                                         // Receiving the indicator values
   bool              check_for_buy();                                // Checking for buy
   bool              check_for_sell();                               // Checking for sell
   void              open_buy();                                     // Open buy
   void              open_sell();                                    // Open sell
   void              position_modify();                              // Position modification
  };
//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
/* Function definition */
//---Constructor---
void my_expert::ma_expert(void)
  {
//--- Reset the values of variables
   ZeroMemory(ma_red_han);
   ZeroMemory(ma_yel_han);
   ZeroMemory(macd_han);
  }
//---The function for receiving the indicator values---
void  my_expert::init(void)
  {
   ma_red_han=iMA(_Symbol,_Period,ma_red_per,0,MODE_EMA,PRICE_CLOSE); // Handle of the slow MA
   ma_yel_han=iMA(_Symbol,_Period,ma_yel_per,0,MODE_EMA,PRICE_CLOSE); // Handle of the fast MA
   macd_han=iMACD(_Symbol,_Period,12,26,9,PRICE_CLOSE);               // Handle of MACDaka
//---Copy data into arrays and set indexing like in a time-series---
   CopyBuffer(ma_red_han,0,0,4,MA_RED);
   CopyBuffer(ma_yel_han,0,0,4,MA_YEL);
   CopyBuffer(macd_han,0,0,2,MACD);
   ArraySetAsSeries(MA_RED,true);
   ArraySetAsSeries(MA_YEL,true);
   ArraySetAsSeries(MACD,true);
  }
//---Function to check conditions to open buy---   
bool my_expert::check_for_buy(void)
  {
   init();  //Receive values of indicator buffers
/* If the fast MA has crossed the slow MA from bottom up between 2nd and 3rd bars, 
   and there was no crossing back. MACD-hist is below zero */
   if(MA_RED[3]>MA_YEL[3] && MA_RED[1]<MA_YEL[1] && MA_RED[0]<MA_YEL[0] && MACD[1]<0)
     {
      return(true);
     }
   return(false);
  }
//----Function to check conditions to open sell---
bool my_expert::check_for_sell(void)
  {
   init();  //Receive values of indicator buffers
/* If the fast MA has crossed the slow MA from up downwards between 2nd and 3rd bars,
  and there was no crossing back. MACD-hist is above zero */
   if(MA_RED[3]<MA_YEL[3] && MA_RED[1]>MA_YEL[1] && MA_RED[0]>MA_YEL[0] && MACD[1]>0)
     {
      return(true);
     }
   return(false);
  }
//---Open buy---
/* Form a standard trade request to buy */
void my_expert::open_buy(void)
  {
   request.action=TRADE_ACTION_DEAL;
   request.symbol=_Symbol;
   request.volume=lots;
   request.price=SymbolInfoDouble(Symbol(),SYMBOL_ASK);
   request.sl=request.price-sl*_Point;
   request.tp=0;
   request.deviation=10;
   request.type=ORDER_TYPE_BUY;
   request.type_filling=ORDER_FILLING_FOK;
   OrderSend(request,result);
   return;
  }
//---Open sell---
/* Form a standard trade request to sell */
void my_expert::open_sell(void)
  {
   request.action=TRADE_ACTION_DEAL;
   request.symbol=_Symbol;
   request.volume=lots;
   request.price=SymbolInfoDouble(Symbol(),SYMBOL_BID);
   request.sl=request.price+sl*_Point;
   request.tp=0;
   request.deviation=10;
   request.type=ORDER_TYPE_SELL;
   request.type_filling=ORDER_FILLING_FOK;
   OrderSend(request,result);
   return;
  }
//---Position modification---
void my_expert::position_modify(void)
  {
   if(PositionGetSymbol(0)==_Symbol)
     {     //If a position is for our symbol
      request.action=TRADE_ACTION_SLTP;
      request.symbol=_Symbol;
      request.deviation=10;
      //---If a buy position---
      if(PositionGetInteger(POSITION_TYPE)==POSITION_TYPE_BUY)
        {
/* if distance from price to stop loss is more than trailing stop
   and the new stop loss is not less than the previous one */
         if(SymbolInfoDouble(Symbol(),SYMBOL_BID)-PositionGetDouble(POSITION_SL)>_Point*ts)
           {
            if(PositionGetDouble(POSITION_SL)<SymbolInfoDouble(Symbol(),SYMBOL_BID)-_Point*ts)
              {
               request.sl=SymbolInfoDouble(Symbol(),SYMBOL_BID)-_Point*ts;
               request.tp=PositionGetDouble(POSITION_TP);
               OrderSend(request,result);
              }
           }
        }
      //---If it is a sell position---                
      else if(PositionGetInteger(POSITION_TYPE)==POSITION_TYPE_SELL)
        {
/*  if distance from price to stop loss is more than the trailing stop value
   and the new stop loss is not above the previous one. Or the stop loss from the moment of opening is equal to zero */
         if((PositionGetDouble(POSITION_SL)-SymbolInfoDouble(Symbol(),SYMBOL_ASK))>(_Point*ts))
           {
            if((PositionGetDouble(POSITION_SL)>(SymbolInfoDouble(Symbol(),SYMBOL_ASK)+_Point*ts)) || 
               (PositionGetDouble(POSITION_SL)==0))
              {
               request.sl=SymbolInfoDouble(Symbol(),SYMBOL_ASK)+_Point*ts;
               request.tp=PositionGetDouble(POSITION_TP);
               OrderSend(request,result);
              }
           }
        }
     }
  }
//+------------------------------------------------------------------

My humble obeisances to the author of the article "Writing an Expert Advisor using the MQL5 Object-Oriented Approach". What would I do without it! I recommend reading this article to anyone who is not very well versed in this evil, but extremely functional Object-oriented programming.

Add the file with the class to the main code of the Expert Advisor? crate an object and initialize functions:

//+------------------------------------------------------------------+
//|                                                       Moving.mq5 |
//|                        Copyright 2011, MetaQuotes Software Corp. |
//|                                              https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2011, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"
//---Include a file with the class---
#include <moving.mqh>
//---External Variables---
input int MA_RED_PERIOD=7; // The period of a slow MA
input int MA_YEL_PERIOD=2; // The period of a fast MA
input int STOP_LOSS=800;   // Stop loss
input int TRAL_STOP=800;   // Trailing stop
input double LOTS=0.1;     // Lot
//---Create an object---
my_expert expert;
//---Initialize the MqlDataTime structure---
MqlDateTime time;
int day_of_week;
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
//---Initialize the EA
   expert.get_periods(MA_RED_PERIOD,MA_YEL_PERIOD);   // Set the MA periods
   expert.get_lot(LOTS);                              // Set the lot
   expert.get_stops(STOP_LOSS,TRAL_STOP);             // Set stop orders  
   return(0);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
  }
//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
   TimeToStruct(TimeCurrent(),time);
   day_of_week=time.day_of_week;
   if(PositionsTotal()<1)
     {
      if(day_of_week==5 && expert.check_for_buy()==true){expert.open_buy();}
      else if(day_of_week==1 && expert.check_for_sell()==true){expert.open_sell();}
     }
   else expert.position_modify();
  }
//+------------------------------------------------------------------+

Done! I'd like to note some special features. To identify the days of the week at the software level, I used the MqlDateTime structure. First, we transform the current server time into a structured format. We obtain an index of the current day (1-Monday, ..., 5-Friday) and compare it with the value that we've set.

Trying out! In order not to burden you with tedious researches and extra digits, I'm bringing all of the results in the table.

Here it is: 

Table 1. Summary of Buys on every day of the week

Table 2. Summary of Sells on every day of the week

The best results are highlighted in green, the worst ones are orange.

I make a reservation, that after the above-described actions the system must ensure profit in combination with low relative drawdown, a good percentage of winning trades (here, the less trades the better) and a relatively high profit per trade.

Obviously, the most effective system is buying on Friday and selling on Monday. Combine both of these conditions:

if(PositionsTotal()<1){
      if(day_of_week==5&&expert.check_for_buy()==true){expert.open_buy();}
      else if(day_of_week==1&&expert.check_for_sell()==true){expert.open_sell();}}
   else expert.position_modify();

Now the Expert Advisor opens positions in both directions, but on strictly defined days. For clarity, I'll draw the diagrams obtained without and with the filter:

Figure 2. The results of EA testing without using a filter (EURUSD, H1, 01.01.2010-31.12.2010,)

Figure 3. The results of EA testing using the filter (EURUSD, H1, 01.01.2010-31.12.2010)

How do you like the result? By using the filter the trading system became more stable. Before the modifications, the Expert Advisor was mainly increasing the balance in the first half of the testing period, after the "upgrade" it is increasing throughout the whole period.

We compare the reports:

Table 3. Results of testing before and after using the filter

The only distressing factor, which can not be ignored, is the fall of the net profit by almost 1000 USD (26%). But we are reducing the number of trades almost in 3,5 times, i.e. significantly reducing, firstly, the potential to make a negative trade and, secondly, the expenses for the spread (218*2-62*2=312 USD and it's only for EUR/USD). Winning percentage is increased to 57%, which is already significant. While profit per trade increases by 14% to 113 USD. As L. Williams would say: "This is the amount, which is worth trading!"

Conclusion

Prices do not behave randomly - it is a fact. This fact can and should be used. I have given only one example, which is a tiny fraction of the innumerable variations and techniques that can improve the performance of your trading system. However, this diversity hides a vice. Not every filter can be integrated, so it must be choose carefully, thinking through all possible scenarios.

Do not forget that no matter how perfect the filter is, it weeds out profitable trades as well, i.e. your profit... Good luck!