Discussion of article "Calculating mathematical expressions (Part 2). Pratt and shunting yard parsers"

[MetaQuotes]

New article Calculating mathematical expressions (Part 2). Pratt and shunting yard parsers has been published:

In this article, we consider the principles of mathematical expression parsing and evaluation using parsers based on operator precedence. We will implement Pratt and shunting-yard parser, byte-code generation and calculations by this code, as well as view how to use indicators as functions in expressions and how to set up trading signals in Expert Advisors based on these indicators.

If you launch the EA in the Strategy Tester, the result will most likely be not very good. However, what's important, is that the EA trades and trading is managed by parsers. They do not give ready-made profitable systems but provide and additional tool for finding strategies.

An example of trading using signals calculated by expressions

Author: Stanislav Korotky

[Dmitry Fedoseev]

When you compare performance, compare it to pure MQL.

[Stanislav Korotky]

The comparison of performance taking into account native MQL5 is as follows:

ExpresSParserS (EURUSD,D1)      Evaluation: 105109
ExpresSParserS (EURUSD,D1)      Compilation: 26090
ExpresSParserS (EURUSD,D1)      Pratt bytecode: 24030
ExpresSParserS (EURUSD,D1)      Pratt: 26567
ExpresSParserS (EURUSD,D1)      ShuntingYard: 23884
ExpresSParserS (EURUSD,D1)      MQL5: 12901

MQL5 is about 2 times faster than the fastest parsers presented.

Added test code:

class FuncCalc
{
  private:
    double a;
    double b;
    double c;
  public:
    void setup(double _a, double _b, double _c)
    {
      a = _a;
      b = _b;
      c = _c;
    }
    double execute()
    {
      return (a + b) * (c > 10000 ? c / 4 : c * 4);
    }
};

ulong testMQL5(const int n)
{
  ulong ul = 0, total = 0;
  double r;
  FuncCalc f;

  for(int i = 0; i < n; i++)
  {
    f.setup(rand(), rand(), rand());
    ul = GetMicrosecondCount();
    r = f.execute();
    total += GetMicrosecondCount() - ul;
  }
  return total;
}

This is a rough estimate. There is still a lot of room for research (for example, with different types of expressions) and parser optimisation.

[Maxim Kuznetsov]

now I can only praise.

Of course I should have mentioned Dijkstra, as historically the Shooting Yard is more associated with him.

I had an idea to make a similar article, I was even typing the material for myself, you beat me to it (about part 2), but I had a habit of rolling everything into fort and stack machines.

you managed not to roll the article into it (or lisp) !

The cherry on the cake would be the Y combinator.

PS. But bytecodes you have turned out to be "wide" and killed by classes. You can whistle and tolerate 64 bits. Well, the boss is the boss :-)

[Stanislav Korotky]

Maxim Kuznetsov:

now I can only praise.

Well of course it would be worth mentioning Dijkstra, as historically the Shooting Yard is more associated with him

I had an idea to make a similar article, I was even typing the material for myself, you beat me to it (about part 2), but I had all the usual things rolled into the fort and stack machines

you managed not to roll the article into it (or lisp) !

The cherry on the cake would be the Y-combinator.

PS. But your bytecodes look "too wide" and killed by classes. You can whistle and tolerate 64 bits. Well, the boss is the boss :-)

You can write a sequel with byte-code shrinking, acceleration and new types of parsers. I didn't manage to cover everything, in fact, I expected to write one article, but as usual it didn't fit.

[Maxim Kuznetsov]

Stanislav Korotky:

You can write a sequel with byte-code shrinking, acceleration and new types of parsers. I didn't manage to cover everything, in fact I expected to write one article, but as usual it didn't fit.

Bytecode depends on the virtual machine. On the one hand, I'm tempted to give 128 bits or more for the op-code "addition of two numbers" ....

and new types of parsers have not been invented yet, they are all years old ;-) and there are only 2 and a half of them.

PS the sorting station for the needs of terminal users is over-the-hill - you can enter a familiar formula in the input field and get the result.

[Реter Konow]

Unlike some of the other articles, these two articles are very good. Clear, detailed, accessible. Thank you.

[Stanislav Korotky]

The updated version of calculators 1.1 is attached to the article about particle swarm optimisation. Plus there is a small bugfix in the discussion.

[fxsaber]

Please show how to use a parser of such a string.

"(EURUSD^2) / (GBPUSD * AUDUSD)"

The difficulty is that we need to automatically determine in which case and where to substitute bid/ask.

In the example above it should be like this.

Value_Bid = (EURUSD_Bid * EURUSD_Bid / (GBPUSD_Ask * AUDUSD__Ask);
Value_Ask = (EURUSD_Ask * EURUSD_Ask / (GBPUSD_Bid * AUDUSD__Bid);

The algorithm for determining Bid/Ask is like this. Using the same example.

F(EURUSD, GBPUSD, AUDUSD) = (EURUSD^2) / (GBPUSD * AUDUSD);

bool EURUSD_flag = (F(1, 1, 1) < F(2, 1, 1));
bool GBPUSD_flag = (F(1, 1, 1) < F(1, 2, 1));
bool AUDUSD_flag = (F(1, 1, 1) < F(1, 1, 2));

Value_Bid = F(EURUSD_flag ? EURUSD_Bid : EURUSD_Ask,
              GBPUSD_flag ? GBPUSD_Bid : GBPUSD_Ask,
              AUDUSD_flag ? AUDUSD_Bid : AUDUSD_Ask);

Value_Ask = F(EURUSD_flag ? EURUSD_Ask : EURUSD_Bid,
              GBPUSD_flag ? GBPUSD_Ask : GBPUSD_Bid,
              AUDUSD_flag ? AUDUSD_Ask : AUDUSD_Bid);

[fxsaber]

fxsaber:
Please show me how to use the parser for such a string.

The main difficulty is to determine the names of all variables in the expression. So that you can write something similar.

TestSuiteEvaluator evaluator("EURUSD=1.5;GBPUSD=2.5;AUDUSD=5");

[Stanislav Korotky]

From the parser's point of view, a variable cannot have 2 values: bid and ask. It is probably possible to wrap components in functions (either introduce Bid(symbol), Ask(symbol) functions, or the whole "basket" function if the number of components is predefined). Basically, the original problem is not clear: if we are talking about a synthetic/basket of three symbols, then in it each component is acquired unambiguously by either Ask or Bid, depending on the direction. You can also consider the option of different expressions depending on the direction of transactions.