Download MetaTrader 5

Watch how to download trading robots for free

Interesting script?
So post a link to it -
let others appraise it

You liked the script? Try it in the MetaTrader 5 terminal

2016.05.20 13:39
RegularExpressions in MQL5 for working with regular expressions

RegularExpressions in MQL5 for working with regular expressions - library for MetaTrader 5

| English Русский 中文 Español Deutsch 日本語 Português

Published by:
MetaQuotes Software Corp.
Views:
1553
Rating:
votes: 29
\MQL5\Experts\Examples\
Tests.mq5 (139.23 KB)view
\MQL5\Include\Controls\\MQL5\Include\Internal\
Array.mqh (23.41 KB)view
Wrappers.mqh (16.97 KB)view
\MQL5\Include\Internal\Generic\
Collection.mqh (12.87 KB)view
Dictionary.mqh (33.94 KB)view
IComparer.mqh (5.59 KB)view
IList.mqh (5.89 KB)view
LinkedList.mqh (27.74 KB)view
List.mqh (38 KB)view
\MQL5\Include\Internal\TimeSpan\
TimeSpan.mqh (32.08 KB)view
\MQL5\Include\RegularExpressions\
Regex.mqh (57.51 KB)view
RegexCode.mqh (22.55 KB)view
RegexGroup.mqh (8.93 KB)view
RegexMatch.mqh (23.45 KB)view
RegexNode.mqh (32.8 KB)view
RegexParser.mqh (95.21 KB)view
RegexRunner.mqh (31.64 KB)view
RegexTree.mqh (8.66 KB)view
RegexWriter.mqh (23.37 KB)view

Real author:

Microsoft Corporation. Source code taken from .Net Framework 4.6.1

Note: The library works on the MetaTrader 5 build 1285 and higher.

Unzip the archive into the terminal_data_folder.
The library codes are located in the: <terminal_data_folder>\MQL5\Include\RegularExpressions\
Sample test scripts can be found in the <terminal_data_folder>\MQL5\Scripts\RegularExpressionsExamples\

RegularExpressions for MQL5 is the implementation of the.NET Framework 4.6.1 regular expression engine.

Regular expressions provide a formal language for quick and flexible processing of texts. Each regular expression is a pattern (mask), for which the regular expression engine tries to find matches in the source text. Patterns consist of one or more character literals, operators, or constructs.

For the implementation of the regular expression engine the .NET Framework uses the traditional NFA machine(NonDeterministic Finite-state Automaton), which performs backtracking for regular expressions. Similar regular expression engine was used in Perl, Python, Emacs and Tcl. Use of the NFA machine distinguishes it from the faster but less functional DFA machines (Deterministic finite-state automaton), which are designed for the regular expressions only and are used in awk, egrep or lex.

The .NET Framework regular expression engine includes the basic functions of the NFA machine, an also the full set of the structure elements, which allows to control the backtracking process. Using these structures the programmer has the ability to significantly speed up the search or to select the preferred extensions.

The main features of the .NET Framework regular expression engine:

    Lazy quantifiers: ??, *?, +?, {n,m}?.
    Positive lookahead: (?=subexpression).
    Negative lookahead: (?!subexpression).
    Conditional evaluation: (?(expression)yes|no) and (?(name)yes"> no ) and (?(name)yes|no) ).
    Balancing group definitions: (?<name1-name2> subexpression).
    Nonbacktracking (greedy) subexpressions: (?>subexpression).
    Right-to-left matching. For this purpose, pass the RegexOptions::RightToLeft parameter to the static instance matching method or to the Regex class constructor itself.
    Positive and negative lookbehind: (?<=subexpression) for positive lookbehind, and (?<!subexpression) for negative lookbehind.

With this library all this features will be available by the means of MQL5.


Description:

Here is a translation of the RegularExpressions from .Net Framework 4.6.1.

To work with the library, include the Regex.mqh file from the "\MQL5\Include\RegularExpressions\" directory in your code.

Several illustrative examples are provided with the library, which at the same time serve as the test cases. All the samples are taken from the official site of Microsoft Corporation, they vividly demonstrate the main differences from the regular expressions in C# and the features of their use in the MQL5.

Below is more detailed information about RegularExpressions MQL5 ported library packages:

Packages
Description
CharUnicodeInfo.mqh
Archived txt file to determine the Unicode categories for all symbols (including non-Latin characters).
RegexCapture.mqh
Represents the results from a single successful subexpression capture.
RegexCaptureCollection.mqh
Represents the set of captures made by a single capturing group.
RegexGroup.mqh
Represents the results from a single capturing group.
RegexGroupCollections.mqhRepresents a collection of Group objects.
RegexMatch.mqhRepresents the results from a single regular expression match.
RegexMatchCollection.mqhRepresents a collection of successful matches found by iteratively applying the regular expression pattern to the input string.
Regex.mqhRepresents an immutable regular expression
RegexOptions.mqh Provides enumerated values to use to set regular expression options.

The regular expression parameters from the file RegexOptions.mqh:

Parameter
Description
None
Specifies that no options are set.
IgnoreCase
Specifies case-insensitive matching.
MultilineSpecifies multiline mode.
ExplicitCaptureDo not capture unnamed groups. Specifies that the only valid captures are explicitly named or numbered groups of the form (?<name> subexpression).
SinglelineSpecifies single-line mode.
IgnorePatternWhitespaceEliminates unescaped white space from the pattern and enables comments marked with #.
RightToLeftSpecifies that the search will be from right to left instead of from left to right.
DebugSpecifies that the program works in the debug mode.
ECMAScriptEnables ECMAScript-compliant behavior for the expression. This value can be used only in conjunction with the IgnoreCase and Multiline values.


Working with RegularExpressions for MQL5:

  1. As withe the version for .Net, this library implements a storage (static cache) of regular expressions. All implicitly created regular expressions (instances of the Regex class) are written to that storage. This approach speeds up the operation of the scripts, as it eliminates the need to rebuild the regular expression if its pattern matches any of the existing ones. The default size of the storage is 15. The Regex::CacheSize() method returns or sets the maximum number of entries in the current static cache of the complied regular expressions.
  2. The second feature of working with regular expression in MQL5 directly follows from the first one. And it lies in the fact that the above storage must be cleared. To do that, call the Regex::ClearCache() static function. It is recommended to call this function only after the work with the regular expressions has been completed, otherwise there is a risk to delete necessary pointers.
  3. Unlike the .Net, the MQL5 does not implement the foreach loop, and hence the enumeration handling will be different. Example:
    //--- Code in C#
    Regex rx = new Regex(@"\b(?<word>\w+)\s+(\k<word>)\b", RegexOptions.IgnoreCase);   
    string text = "The the quick brown fox  fox jumped over the lazy dog dog.";
    MatchCollection matches = rx.Matches(text);
    foreach (Match match in matches) 
      {
       //--- handling
      }
    
    //--- Code in MQL5
    Regex *rx = new Regex("\\b(?<word>\\w+)\\s+(\\k<word>)\\b", RegexOptions::IgnoreCase);        
    string text = "The the quick brown fox  fox jumped over the lazy dog dog.";
    MatchCollection *matches = rx.Matches(text);
    IEnumerator<Match*> *en = matches.GetEnumerator();
    while(en.MoveNext()) 
      {
       Match *match = en.Current();
       //--- handling
      }
    delete rx;
    delete matches;
    delete en;
    Regex::ClearCache();
    
  4. As can be seen from the above example, the C# syntax allows to put the '@' symbol in front of the strings to ignore all formatting marks in it. In MQL5, this approach is not provided, so all control characters in a regular expression pattern should be explicitly defined.


Example of working with RegularExpressions for MQL5:

As an example of regular expressions, consider their application to parse the trading history downloaded from the terminal in the form of an html file.

To do that, create an expert with a single input parameter of the 'string' type, which will be the name of the sandbox file:


This document contains two main tables: "Orders" and "Deals". 

Create a regular expression to parse the file:

Regex *rgx=new Regex("(>)([^<>]*)(<)");

Consider this regular expression:

(>)
Find the '>' character
(^[<>]*)
Any symbol except '>' and '<', repeated zero or more times
(<)
Fin the '<' symbol

Next, read the file line by line and get all the matches which correspond to this regular expression:

string str=FileReadString(m_handel);
MatchCollection *matches=rgx.Matches(str);

The strings of the html file, that are entries (tuples) for the "Orders" and "Deals" tables, will have the most of such matches, namely 23 and 27 matches respectively. Therefore, the only thing left to do is to extract all the necessary information from these strings.

For the "Orders" table:

if(matches.Count()==23)
  {
   string in[11];
   for(int i=0,j=1; i<11; i++,j+=2)
     {
      in[i]=StringSubstr(matches[j].Value(),1,StringLen(matches[j].Value())-2);
     }
   m_list1.Add(new OrderRecord(in));
  }

It has been confirmed that the number of matches is 23, and therefore, the handled entry is from the "Orders" table. Create a string array as the initial representation of our entry. Create iteration over all the odd matches, get the values of those matches using the matches[j][.Value() method, trim the first and the last characters of each match, which correspond to the characters '>' and '<'. Each formatted match is written to a previously declared in array. After that create an instance of the OrderRecord(in) class, which represents a single entry of the "Orders" table, and add it to the m_list1 list. This list will interpret the "Orders" table.

The "Deals" table will be processed in a similar way:

if(matches.Count()==27)
  {
   string in[13];
   for(int i=0,j=1; i<13; i++,j+=2)
     {
      in[i]=StringSubstr(matches[j].Value(),1,StringLen(matches[j].Value())-2);
     }
   m_list2.Add(new DealRecord(in));
  }

Here m_list2 is a list of pointers to the DealRecord class. It represents the "Deals" table.

Both these lists are members of the TradeHistory class. This class is the representation of the entire original html file. In addition, it allows to apply simple filters to the "Orders" and "Deals" tables using the methods: FindAllOrders(const int columnIndex,const T value) and FindAllDeals(const int columnIndex,const T value).

Create a simple graphical interface in the EA to demonstrate these capabilities:


When working with this form, select the desired table, select a column and its value to filter the table. Press the Find button, and the filtered table will be displayed below, along with some statistics on it. The Save button stores the currently displayed table to a csv file. The saved file will be located in the sandbox as well, and have the name Result.csv.

To learn more about the RegularExpressions for MQL5 and its features, use the provided Tests.mqh expert. It implements numerous examples of regular expression usage, which cover all the main functionality of the library.

Translated from Russian by MetaQuotes Software Corp.
Original code: https://www.mql5.com/ru/code/15242

Accelerator Oscillator (AC) Accelerator Oscillator (AC)

The Acceleration/Deceleration Indicator (AC) measures acceleration and deceleration of the current driving force.

Average Directional Movement Index (ADX) Average Directional Movement Index (ADX)

The Average Directional Movement Index Indicator (ADX) helps to determine if there is a price trend.

LinesProfitLoss LinesProfitLoss

Calculates profit (loss) of the current orders on the symbol.

FFC - Forex Factory Calendar FFC - Forex Factory Calendar

Modified version of FF Calendar Indicator with new features.