MQL5 Cookbook: Reducing the Effect of Overfitting and Handling the Lack of Quotes
Anatoli Kazharski | 24 September, 2013
Introduction
I believe a lot of traders have repeatedly puzzled over the optimal parameters for their trading systems. Indeed, a trading algorithm alone is not enough. One needs to see how it can further be used. Whatever trading strategy you use, whether it's simple or complex, with a single or multiple instruments, you cannot avoid the question of what parameters to choose to ensure future profits.
We tend to check trading systems with parameters that showed good results over the period of optimization (backtesting) and over the subsequent period (forward testing). Forward testing is in fact not so necessary. The relevant results can be obtained using historical data.
This method gives rise to a huge question that cannot be given a definite answer: what amount of historical data should be used to optimize a trading system? The thing is there are lots of options. It all depends on the range of price fluctuations you expect to capitalize on.
Getting back to the amount of history required for optimization, we conclude that the available data may well be sufficient for intra-hour trading. This is not always true for longer time frames. The more the number of repetitions of a consistent pattern, i.e. the more trades we have, the more truthful is the performance of the tested trading system that we can expect to see in the future.
What if price data of a certain instrument are not enough to get a sufficient number of repetitions and feel more certain? Answer: use data from all available instruments.
Example in NeuroShell DayTrader Professional
Before we proceed to programming in MetaTrader 5, let's review an example in NeuroShell DayTrader Professional. It offers great features for optimizing parameters of a trading system (compiled with a constructor) for multiple symbols. You can set the required parameters in the trading module settings, optimize parameters for each symbol separately or find an optimal parameter set for all symbols at once. This option can be found in the Optimization tab:
Fig. 1. The Optimization tab in the trading module of NeuroShell DayTrader Professional
In our case, any simple trading system will do as we only need to compare results of two optimization methods, so the choice of the system is currently of little importance.
You can find information on how to compile trading strategies in NeuroShell DayTrader Professional in other articles of my blog (you can search or use tags to locate the relevant information). I would also recommend you to read an article entitled "How to Prepare MetaTrader 5 Quotes for Other Applications" that describes and demonstrates how using a script you can download quotes from MetaTrader 5 in the format compatible with NeuroShell DayTrader Professional.
To do this test, I prepared data obtained from daily bars for eight symbols from year 2000 until January 2013:
Fig. 2. List of symbols for a test in NeuroShell DayTrader Professional
The figure below shows two optimization results. The upper part displays the result of the optimization where each symbol gets its own parameters, while the lower part shows the result where the parameters are common for all symbols.
Fig. 3. Comparison of the results of two parameter optimization modes
The result showing common parameters does not look as good as the one where parameters are different for each symbol. Yet it inspires more confidence as the trading system goes through a number of various price behavior patterns (volatility, number of trends/flats) with the same parameters for all symbols.
Continuing on the same subject, we can logically find another argument in favor of the optimization using greater amount of data. It may well be that the price behavior of a certain currency pair, e.g. EURUSD, will be quite different afterwards (in two, five or ten years). For instance, GBPUSD price trends will be similar to the past price behavior of EURUSD and vice versa. You should be ready for that as this is true for any instrument.
An Example in MetaTrader 5
Let's now see what parameter optimization modes are offered in MetaTrader 5. Below you can see the All Symbols Selected in Market Watch optimization mode marked with an arrow in the drop-down list of optimization modes.
Fig. 4. Optimization modes in the MetaTrader 5 Strategy Tester
This mode allows you to only test an EA with the current parameters on each symbol one by one. The symbols used in testing are the ones that are currently selected in the Market Watch window. In other words, parameter optimization is in this case not performed. However, MetaTrader 5 and MQL5 allow you to implement this idea on your own.
Now, we need to see how to implement such an EA. The symbol list will be provided in a text file (*.txt). Further, we will implement the possibility of storing several sets of symbol lists. Each set will be in a separate section with its own header featuring a section number. Numbers are necessary to facilitate visual checking.
Note that it is important to have # in front of the number so as to allow the Expert Advisor to get the right data set when filling the array of symbols. Generally, the header may contain any symbols but it must have # at all times. The number sign can be replaced with any other symbol according to which the Expert Advisor will determine/count sections. In that case the replacement will have to be reflected in the code.
Below you can see the SymbolsList.txt file that contains three symbol sets for testing. This file as shown is going to be further used when testing the method.
Fig. 5. Several symbol sets provided in a text file for testing
In the external parameters, we will add another parameter, SectionOfSymbolList, to indicate the set of symbols that the Expert Advisor should use in the current test. This parameter takes on the value (from zero up) that defines the symbol set. If the value exceeds the number of available sets, the Expert Advisor will write a corresponding entry to the log and testing will only be done on the current symbol.
SymbolsList.txt must be located in the local terminal directory under Metatrader 5\MQL5\Files. It can also be placed in the common folder but in this case it will not be available for parameter optimization in the MQL5 Cloud Network. Further, to allow access to the file and the relevant custom indicators for testing, we need to write the following lines at the beginning of the file:
//--- Allow access to the external file and indicator for optimization in the cloud #property tester_file "SymbolsList.txt" #property tester_indicator "EventsSpy.ex5"
Our Expert Advisor will be based on the ready-made multi-currency Expert Advisor featured in the article "MQL5 Cookbook: Developing a Multi-Currency Expert Advisor with Unlimited Number of Parameters". Its underlying trading strategy is quite simple but it will suffice to test the efficiency of the method. We will just remove the unnecessary parts, add what we need and correct the existing relevant code. We will certainly enhance our Expert Advisor with the report saving feature extensively described in the previous article of the series "MQL5 Cookbook: Writing the History of Deals to a File and Creating Balance Charts for Each Symbol in Excel". Balance charts for all the symbols will also be required to evaluate the efficiency of the method under consideration.
The external parameters of the Expert Advisor should be modified as follows:
//--- External parameters of the Expert Advisor sinput int SectionOfSymbolList = 1; // Section number in the symbol lists sinput bool UpdateReport = false; // Report update sinput string delimeter_00=""; // -------------------------------- sinput long MagicNumber = 777; // Magic number sinput int Deviation = 10; // Slippage sinput string delimeter_01=""; // -------------------------------- input int IndicatorPeriod = 5; // Indicator period input double TakeProfit = 100; // Take Profit input double StopLoss = 50; // Stop Loss input double TrailingStop = 10; // Trailing Stop input bool Reverse = true; // Position reversal input double Lot = 0.1; // Lot input double VolumeIncrease = 0.1; // Position volume increase input double VolumeIncreaseStep = 10; // Volume increase step
All the arrays associated with the external parameters should be deleted as they will not be needed and must further be replaced by the external variables throughout the entire code. We should only leave the dynamic array of symbols, InputSymbols[], whose size will depend on the number of symbols used from one of the sets in the SymbolsList.txt file. If the Expert Advisor is used outside the Strategy Tester, the size of that array will be equal to 1 as in real-time mode the Expert Advisor will work with one symbol only.
The corresponding changes must also be made in the array initialization file - InitializeArrays.mqh. That is, all the functions responsible for initialization of the arrays of external variables should be deleted. The InitializeArraySymbols() function now looks as shown below:
//+------------------------------------------------------------------+ //| Filling the array of symbols | //+------------------------------------------------------------------+ void InitializeArraySymbols() { int strings_count =0; // Number of strings in the symbol file string checked_symbol =""; // To check the accessibility of the symbol on the trade server //--- Test mode message string message_01="<--- All symbol names in the <- SymbolsList.txt -> file are incorrect ... --->\n" "<--- ... or the value of the \"Section of List Symbols\" parameter is greater, " "than the number of file sections! --->\n" "<--- Therefore we will test only the current symbol. --->"; //--- Real-time mode message string message_02="<--- In real-time mode, we only work with the current symbol. --->"; //--- If in real-time mode if(!IsRealtime()) { //--- Get the number of strings from the specified symbol set in the file and fill the temporary array of symbols strings_count=ReadSymbolsFromFile("SymbolsList.txt"); //--- Iterate over all symbols from the specified set for(int s=0; s<strings_count; s++) { //--- If the correct string is returned following the symbol check if((checked_symbol=GetSymbolByName(temporary_symbols[s]))!="") { //--- increase the counter SYMBOLS_COUNT++; //--- set/increase the array size ArrayResize(InputSymbols,SYMBOLS_COUNT); //--- index with the symbol name InputSymbols[SYMBOLS_COUNT-1]=checked_symbol; } } } //--- If all symbol names were not input correctly or if currently working in real-time mode if(SYMBOLS_COUNT==0) { //--- Real-time mode message if(IsRealtime()) Print(message_02); //--- Test mode message if(!IsRealtime()) Print(message_01); //--- We will work with the current symbol only SYMBOLS_COUNT=1; //--- set the array size and ArrayResize(InputSymbols,SYMBOLS_COUNT); //--- index with the current symbol name InputSymbols[0]=_Symbol; } }
The ReadSymbolsFromFile() function code should also be modified. It used to read the entire symbol list whereas now we want it to only read the specified symbol set. Below is the modified function code:
//+------------------------------------------------------------------+ //| Returning the number of strings (symbols) from the specified | //| set in the file and filling the temporary array of symbols | //+------------------------------------------------------------------+ //--- When preparing the file, symbols in the list should be separated with a line break int ReadSymbolsFromFile(string file_name) { ulong offset =0; // Offset for determining the position of the file pointer string delimeter ="#"; // Identifier of the section start string read_line =""; // For the check of the read string int limit_count =0; // Counter limiting the number of the possibly open charts int strings_count =0; // String counter int sections_count =-1; // Section counter //--- Message 01 string message_01="<--- The <- "+file_name+" -> file has not been prepared appropriately! --->\n" "<--- The first string does not contain the section number identifier ("+delimeter+")! --->"; //--- Message 02 string message_02="<--- The <- "+file_name+" -> file has not been prepared appropriately! --->\n" "<--- There is no line break identifier in the last string, --->\n" "<--- so only the current symbol will be involved in testing. --->"; //--- Message 03 string message_03="<--- The <- "+file_name+" -> file could not be found! --->" "<--- Only the current symbol will be involved in testing. --->"; //--- Open the file (get the handle) for reading in the local directory of the terminal int file_handle=FileOpen(file_name,FILE_READ|FILE_ANSI,'\n'); //--- If the file handle has been obtained if(file_handle!=INVALID_HANDLE) { //--- Read until the current position of the file pointer // reaches the end of the file or until the program is deleted while(!FileIsEnding(file_handle) || !IsStopped()) { //--- Read until the end of the string or until the program is deleted while(!FileIsLineEnding(file_handle) || !IsStopped()) { //--- Read the whole string read_line=FileReadString(file_handle); //--- If the section number identifier has been found if(StringFind(read_line,delimeter,0)>-1) //--- Increase the section counter sections_count++; //--- If the section has been read, exit the function if(sections_count>SectionOfSymbolList) { FileClose(file_handle); // Close the file return(strings_count); // Return the number of strings in the file } //--- If this is the first iteration and the first string does not contain the section number identifier if(limit_count==0 && sections_count==-1) { PrepareArrayForOneSymbol(strings_count,message_01); //--- Close the file FileClose(file_handle); //--- Return the number of strings in the file return(strings_count); } //--- Increase the counter limiting the number of the possibly open charts limit_count++; //--- If the limit has been reached if(limit_count>=CHARTS_MAX) { PrepareArrayForOneSymbol(strings_count,message_02); //--- Close the file FileClose(file_handle); //--- Return the number of strings in the file return(strings_count); } //--- Get the position of the pointer offset=FileTell(file_handle); //--- If this is the end of the string if(FileIsLineEnding(file_handle)) { //--- Go to the next string if this is not the end of the file // For this purpose, increase the offset of the file pointer if(!FileIsEnding(file_handle)) offset++; //--- move it to the next string FileSeek(file_handle,offset,SEEK_SET); //--- If we are not in the specified section of the file, exit the loop if(sections_count!=SectionOfSymbolList) break; //--- Otherwise, else { //--- if the string is not empty if(read_line!="") { //--- increase the string counter strings_count++; //--- increase the size of the array of strings, ArrayResize(temporary_symbols,strings_count); //--- write the string to the current index temporary_symbols[strings_count-1]=read_line; } } //--- Exit the loop break; } } //--- If this is the end of the file, terminate the entire loop if(FileIsEnding(file_handle)) break; } //--- Close the file FileClose(file_handle); } else PrepareArrayForOneSymbol(strings_count,message_03); //--- Return the number of strings in the file return(strings_count); }
You can see that some strings in the above code are highlighted. Those strings contain the PrepareArrayForOneSymbol() function that simply prepares an array for one (current) symbol in case of an error.
//+------------------------------------------------------------------+ //| Preparing an array for one symbol | //+------------------------------------------------------------------+ void PrepareArrayForOneSymbol(int &strings_count,string message) { //--- Print the message to the log Print(message); //--- Array size strings_count=1; //--- Set the size of the array of symbols ArrayResize(temporary_symbols,strings_count); //--- Write the string with the current symbol name to the current index temporary_symbols[0]=_Symbol; }
Now everything is ready to test the parameter optimization method. But before we proceed to testing, let's add another data series to the report. Earlier, in addition to balances of all symbols, the report file contained all drawdowns from local maxima expressed as percentage. Now the report will also cover all drawdowns in monetary terms. At the same time, we will modify the CreateSymbolBalanceReport() function where the report is generated.
The CreateSymbolBalanceReport() function code is provided below:
//+------------------------------------------------------------------+ //| Creating test report on deals in .csv format | //+------------------------------------------------------------------+ void CreateSymbolBalanceReport() { int file_handle =INVALID_HANDLE; // File handle string path =""; // File path //--- If an error occurred when creating/getting the folder, exit if((path=CreateInputParametersFolder())=="") return; //--- Create a file to write data in the common folder of the terminal file_handle=FileOpen(path+"\\LastTest.csv",FILE_CSV|FILE_WRITE|FILE_ANSI|FILE_COMMON); //--- If the handle is valid (file created/opened) if(file_handle>0) { int digits =0; // Number of decimal places in the price int deals_total =0; // Number of deals in the specified history ulong ticket =0; // Deal ticket double drawdown_max =0.0; // Drawdown double balance =0.0; // Balance string delimeter =","; // Delimiter string string_to_write =""; // To generate the string for writing static double percent_drawdown =0.0; // Drawdown expressed as percentage static double money_drawdown =0.0; // Drawdown in monetary terms //--- Generate the header string string headers="TIME,SYMBOL,DEAL TYPE,ENTRY TYPE,VOLUME," "PRICE,SWAP($),PROFIT($),DRAWDOWN(%),DRAWDOWN($),BALANCE"; //--- If more than one symbol is involved, modify the header string if(SYMBOLS_COUNT>1) { for(int s=0; s<SYMBOLS_COUNT; s++) StringAdd(headers,","+InputSymbols[s]); } //--- Write the report headers FileWrite(file_handle,headers); //--- Get the complete history HistorySelect(0,TimeCurrent()); //--- Get the number of deals deals_total=HistoryDealsTotal(); //--- Resize the array of balances according to the number of symbols ArrayResize(symbol_balance,SYMBOLS_COUNT); //--- Resize the array of deals for each symbol for(int s=0; s<SYMBOLS_COUNT; s++) ArrayResize(symbol_balance[s].balance,deals_total); //--- Iterate in a loop and write the data for(int i=0; i<deals_total; i++) { //--- Get the deal ticket ticket=HistoryDealGetTicket(i); //--- Get all the deal properties GetHistoryDealProperties(ticket,D_ALL); //--- Get the number of digits in the price digits=(int)SymbolInfoInteger(deal.symbol,SYMBOL_DIGITS); //--- Calculate the overall balance balance+=deal.profit+deal.swap+deal.commission; //--- Calculate the max drawdown from the local maximum TesterDrawdownMaximum(i,balance,percent_drawdown,money_drawdown); //--- Generate a string for writing using concatenation StringConcatenate(string_to_write, deal.time,delimeter, DealSymbolToString(deal.symbol),delimeter, DealTypeToString(deal.type),delimeter, DealEntryToString(deal.entry),delimeter, DealVolumeToString(deal.volume),delimeter, DealPriceToString(deal.price,digits),delimeter, DealSwapToString(deal.swap),delimeter, DealProfitToString(deal.symbol,deal.profit),delimeter, DrawdownToString(percent_drawdown),delimeter, DrawdownToString(money_drawdown),delimeter, DoubleToString(balance,2)); //--- If more than one symbol is involved, write their balance values if(SYMBOLS_COUNT>1) { //--- Iterate over all symbols for(int s=0; s<SYMBOLS_COUNT; s++) { //--- If the symbols are equal and the deal result is non-zero if(deal.symbol==InputSymbols[s] && deal.profit!=0) { //--- Display the deal in the balance for the corresponding symbol // Take into consideration swap and commission symbol_balance[s].balance[i]=symbol_balance[s].balance[i-1]+ deal.profit+ deal.swap+ deal.commission; //--- Add to the string StringAdd(string_to_write,","+DoubleToString(symbol_balance[s].balance[i],2)); } //--- Otherwise write the previous value else { //--- If the deal type is "Balance" (the first deal) if(deal.type==DEAL_TYPE_BALANCE) { //--- the balance is the same for all symbols symbol_balance[s].balance[i]=balance; StringAdd(string_to_write,","+DoubleToString(symbol_balance[s].balance[i],2)); } //--- Otherwise write the previous value to the current index else { symbol_balance[s].balance[i]=symbol_balance[s].balance[i-1]; StringAdd(string_to_write,","+DoubleToString(symbol_balance[s].balance[i],2)); } } } } //--- Write the generated string FileWrite(file_handle,string_to_write); //--- Mandatory zeroing out of the variable for the next string string_to_write=""; } //--- Close the file FileClose(file_handle); } //--- If the file could not be created/opened, print the appropriate message else Print("Error creating the file! Error: "+IntegerToString(GetLastError())+""); }
We used to calculate drawdowns in the DrawdownMaximumToString() function. This is now performed by the TesterDrawdownMaximum() function, while the drawdown value is converted to a string using the basic DrawdownToString() function.
The TesterDrawdownMaximum() function code is as follows:
//+------------------------------------------------------------------+ //| Returning the max drawdown from the local maximum | //+------------------------------------------------------------------+ void TesterDrawdownMaximum(int deal_number, double balance, double &percent_drawdown, double &money_drawdown) { ulong ticket =0; // Deal ticket string str =""; // The string to be displayed in the report //--- To calculate the local maximum and drawdown static double max =0.0; static double min =0.0; //--- If this is the first deal if(deal_number==0) { //--- There is no drawdown yet percent_drawdown =0.0; money_drawdown =0.0; //--- Set the initial point as the local maximum max=balance; min=balance; } else { //--- If the current balance is greater than in the memory, then... if(balance>max) { //--- Calculate the drawdown using the previous values: // in monetary terms money_drawdown=max-min; // expressed as percentage percent_drawdown=100-((min/max)*100); //--- Update the local maximum max=balance; min=balance; } //--- Otherwise else { //--- Return zero value of the drawdown money_drawdown=0.0; percent_drawdown=0.0; //--- Update the minimum min=fmin(min,balance); //--- If the deal ticket by its position in the list has been obtained, then... if((ticket=HistoryDealGetTicket(deal_number))>0) { //--- ...get the deal comment GetHistoryDealProperties(ticket,D_COMMENT); //--- Flag of the last deal static bool last_deal=false; //--- The last deal in the test can be identified by the "end of test" comment if(deal.comment=="end of test" && !last_deal) { //--- Set the flag last_deal=true; //--- Update the drawdown values: // in monetary terms money_drawdown=max-min; // expressed as percentage percent_drawdown+=100-((min/max)*100); } } } } }
The code of the DrawdownToString() function is provided below:
//+------------------------------------------------------------------+ //| Converting drawdown to a string | //+------------------------------------------------------------------+ string DrawdownToString(double drawdown) { return((drawdown<=0) ? "" : DoubleToString(drawdown,2)); }
Now everything is set and ready for testing of the Expert Advisor and analysis of the results. Early in the article, we saw an example of the ready-made file. Let's do as follows: optimize parameters for the symbols in the second set (there are three symbols: EURUSD, AUDUSD and USDCHF) and following the optimization run the test using all the symbols from the third set (seven symbols in total) to see the results for the symbols whose data were not involved in parameter optimization.
Optimizing Parameters and Testing Expert Advisor
The Strategy Tester needs to be set as shown below:
Fig. 6. The Strategy Tester settings for optimization
The Expert Advisor settings for parameter optimization are provided below:
Fig. 7. The Expert Advisor settings for parameter optimization
Since the optimization involves three symbols and the position volume increase is enabled for each of them, we set the minimum lot for the purposes of opening a position and increasing the position volume. In our case, the value is 0.01.
After the optimization, we select the top result by the maximum recovery factor and set the VolumeIncrease parameter to 0.1 for the lot. The result is shown below:
Fig. 8. The test result in MetaTrader 5
Below, you can see the result as shown in Excel 2010:
Fig. 9. The test result for three symbols as shown in Excel 2010
The drawdown in monetary terms is displayed as green marks in the lower chart in terms of the second (auxiliary) scale.
You should also be aware of the charting limits in Excel 2010 (the complete list of specifications and limits can be found on the Excel specifications and limits page of the Microsoft Office website).
Fig. 10. Charting specifications and limits in Excel 2010
The table shows that we can run the test for 255 symbols at the same time and display all the results in the chart! We are only limited by computer resources.
Let's now run the test for seven symbols from the third set with the current parameters and check the result:
Fig. 11. The test result for seven symbols as shown in Excel 2010
With seven symbols under consideration, we have 6901 deals. The data in the chart are quite quickly updated in Excel 2010.
Conclusion
I believe the introduced method is noteworthy for the fact that even a simple trading strategy like the one we used showed good results. Here, we should have in mind that the optimization was only performed for three symbols out of seven. We can try to improve the result by optimizing parameters for all symbols at once. However, first and foremost, we should aim at improving the trading system, or better yet, at having a portfolio of various trading systems. We will get back to this idea later.
That's about it. We have got a fairly useful tool for studying results of multi-currency trading strategies. Below is the downloadable zip file with the files of the Expert Advisor for your consideration.
After extracting the files, place the ReduceOverfittingEA folder into the MetaTrader 5\MQL5\Experts directory. Further, the EventsSpy.mq5 indicator must be placed into MetaTrader 5\MQL5\Indicators. SymbolsList.txt must be located under MetaTrader 5\MQL5\Files.