Best Practices for json Implementation

New comment
 

I have fiddled about looking for design concepts to implement visuals of the data collection, aggregation, et al (pics below). As much as I love the MT4/5 platform, visuals are not its strong point. First, real estate is expensive; Second, I/O overhead on graphics rendering is high - the more objects added, the higher the performance loss.

So, I'm looking for solutions/recommendations on approaches to decouple the visuals from the trading platform altogether.

Solutions I've considered so far is a named pipe solution - but, as it turns out - tends to be very platform specific. As an aside, I have finally succeeded in throwing Gates/Windows out the window and have moved entirely to linux.

I found a few developers seriously implementing json/js solutions - and wow, ok - pretty powerful stuff. Sample MT4 script below:

#include <Class/JAson.mqh>

CJAVal queue;

struct RequestRec
       {
          int Action;
          int Type;
          double Lots;
          double Price;
          double TakeProfit;
          double StopLoss;
          string Memo;
       };

RequestRec requests[];

void Serialize()
  {
    CJAVal request;

    for (int node=0;node<2;node++)
    {
        request["Action"]      = requests[node].Action;
        request["Type"]        = requests[node].Type;
        //request["Lots"]        = DoubleToStr(requests[node].Lots,2);
        //request["Price"]       = DoubleToStr(requests[node].Price,Digits);
        //request["TakeProfit"]  = DoubleToStr(requests[node].TakeProfit,Digits);
        //request["StopLoss"]    = DoubleToStr(requests[node].StopLoss,Digits);
        request["Lots"]        = requests[node].Lots;
        request["Price"]       = requests[node].Price;
        request["TakeProfit"]  = requests[node].TakeProfit;
        request["StopLoss"]    = requests[node].StopLoss;
        request["Memo"]        = requests[node].Memo;

        queue["Request"].Add(request);
}
    
    Print(queue.Serialize());
  }

void OnStart (void)
  {
    ArrayResize(requests,2,10);

    requests[0].Action      = OP_BUY;
    requests[0].Type        = OP_BUYLIMIT;
    requests[0].Lots        = 0.10;
    requests[0].Price       = 1.096434;
    requests[0].TakeProfit  = 1.110565;
    requests[0].StopLoss    = 1.0912356;
    requests[0].Memo        = "Serialize Test #1";

    requests[1].Action      = OP_BUY;
    requests[1].Type        = OP_BUYLIMIT;
    requests[1].Lots        = 0.10;
    requests[1].Price       = 1.096434;
    requests[1].TakeProfit  = 1.110565;
    requests[1].StopLoss    = 1.0912356;
    requests[1].Memo        = "Serialize Test #2";

    Serialize();
  }

This is merely a wireframe/mockup of what I am actually attempting to achieve. More seriously, I'd like to work from a single .json file containing all of the class objects thus:

"%hostname%":
  {
    {"Summary":
      [
        {"Type":0,"Equity":0.0,"Margin":,0.00,"Balance":0.00},
        {"Type":1,"Equity":5.2,"Margin":,0.20,"Balance":10525.23},
        {"Type":2,"Equity":5.2,"Margin":,0.20,"Balance":10525.23}
      ]
    },
    {"Request":
      [
        {"Action":0,"Type":2,"Lots":0.10000000,"Price":1.09643400,"TakeProfit":1.11056500,"StopLoss":1.09123560,"Memo":"Serialize Test #1"},
        {"Action":0,"Type":2,"Lots":0.10000000,"Price":1.09643400,"TakeProfit":1.11056500,"StopLoss":1.09123560,"Memo":"Serialize Test #2"}
      ]
    },
    {"Order":
      [
        {"Action":0,"Type":2,"Status":"Hold","Lots":0.10000000,"Price":1.09643400,"TakeProfit":1.11056500,"StopLoss":1.09123560,"Memo":"Serialize Test #1"},
        {"Action":0,"Type":2,"Status":"Collect","Lots":0.10000000,"Price":1.09643400,"TakeProfit":1.11056500,"StopLoss":1.09123560,"Memo":"Serialize Test #2"}
      ]
    }
  }

Then use javascript/html to render web content thus:

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>MetaTrader</title>
    <script src="https://unpkg.com/ag-grid-community/dist/ag-grid-community.min.noStyle.js"></script>
    <link rel="stylesheet" href="https://unpkg.com/ag-grid-community/styles/ag-grid.css" />
    <link rel="stylesheet" href="https://unpkg.com/ag-grid-community/styles/ag-theme-balham.css" />
</head>

<body>
    <section id="container">
        <h1>Terminal Access</h1>
    </section>
    <div id="myGrid" class="ag-theme-balham-dark" style="height: 750px; width: 1600px"></div>
    <script type="text/javascript">
        const eGridDiv = document.getElementById('myGrid');
        const gridOptions = {
            columnDefs: [
                { field: 'Action' },
                { field: 'Type' },
                { field: 'Lots' },
                { field: 'Price' },
                { field: 'TakeProfit' },
                { field: 'StopLoss' },
                { field: 'Memo' },
            ],
            enableSorting: true,
            enableFilter: true,
            pagination: true,
        };

        new agGrid.Grid(eGridDiv, gridOptions);

        fetch("./link/qreq.json")
          .then(response => response.json())
          .then(data => gridOptions.api.setRowData(data))
          .catch(err => console.log(err));
    </script>
</body>
</html>

Producing results thus:

So, I'm running into some issues - not show stoppers - but, rather, best practices in design issues.

Using js and a typical grid utility [that supports .json] to render web content, the .json ('apparently') must be raw as in:

      [
        {"Type":0,"Equity":0.0,"Margin":,0.00,"Balance":0.00},
        {"Type":1,"Equity":5.2,"Margin":,0.20,"Balance":10525.23},
        {"Type":2,"Equity":5.2,"Margin":,0.20,"Balance":10525.23}
      ]

or

      [
        {"Action":0,"Type":2,"Lots":0.10000000,"Price":1.09643400,"TakeProfit":1.11056500,"StopLoss":1.09123560,"Memo":"Serialize Test #1"},
        {"Action":0,"Type":2,"Lots":0.10000000,"Price":1.09643400,"TakeProfit":1.11056500,"StopLoss":1.09123560,"Memo":"Serialize Test #2"}
      ]

or

      [
        {"Action":0,"Type":2,"Status":"Hold","Lots":0.10000000,"Price":1.09643400,"TakeProfit":1.11056500,"StopLoss":1.09123560,"Memo":"Serialize Test #1"},
        {"Action":0,"Type":2,"Status":"Collect","Lots":0.10000000,"Price":1.09643400,"TakeProfit":1.11056500,"StopLoss":1.09123560,"Memo":"Serialize Test #2"}
      ]

This indicates that my implementation would require either a dedicated .json by object type (an approach that may have a tendency to scale out of control), or a function to deserialize the .json before interfacing with the grid.

But then, deserialization appears to be an issue - how does one deserialize mutliple object 'classes' into local containers?

So, I am going to cut and run using dedicated class by class .json 'threads' - it just feels dirty.


Any suggestions?

 
Dennis Jorgenson: So, I'm looking for solutions/recommendations on approaches to decouple the visuals from the trading platform altogether.

Even if it is on Linux, it is still running under Wine, so why not just use DLL calls to the WinAPI to build windows and panels?

You could even create your own DLL's in C/C++ for more sophisticated solutions.

I am assuming this is for your own use and not for sale in the Market, so DLL calls should not be an issue.

 

Wow. Am I really this far behind? Just found a superb article that may be precisely what I'm looking for.

https://stackoverflow.com/questions/39954177/how-to-send-a-post-with-a-json-in-a-webrequest-call-using-mql4

Why create a .json file ('socket') when all one needs to do is to Post a webrequest? Going to attempt this as an approach.

How to send a POST with a JSON in a WebRequest() call using MQL4?
How to send a POST with a JSON in a WebRequest() call using MQL4?
  • 2016.10.10
  • koul koul 461 3 3 gold badges 10 10 silver badges 20 20 bronze badges
  • stackoverflow.com
I would like to send a from -script, using a JSON-format to a Node-server. I've tried the standard function in , based on the following documentation, but it did NOT success. From MQL4 Documentation: Does anyone know how to perfom it?
 
Fernando Carreiro #:

Even if it is on Linux, it is still running under Wine, so why not just use DLL calls to the WinAPI to build windows and panels?

You could even create your own DLL's in C/C++ for more sophisticated solutions.

I am assuming this is for your own use and not for sale in the Market, so DLL calls should not be an issue.

This is true - MT4 running under wine. I wanted to go with with DLL's initially - but don't have enough samples to find a starting point. Most of the samples are windows based using the dreaded .net framework.

That said, even though my project is personally developed - my source is public on github. I've been working with MT4 since 2008 off and on - now, full time. If you know of anyone willing to lend a hand, please do offer.

 
Dennis Jorgenson #: This is true - MT4 running under wine. I wanted to go with with DLL's initially - but don't have enough samples to find a starting point. Most of the samples are windows based using the dreaded .net framework. That said, even though my project is personally developed - my source is public on github. I've been working with MT4 since 2008 off and on - now, full time. If you know of anyone willing to lend a hand, please do offer.

There are not that many examples of DLL written C/C++ for MQL, because they are not much different from any other DLL usage for normal C/C++ applications.

The available examples are enough to show that you can just write them is if you were coding for a pure Windows application.

However, here is an example with Linux in mind ...

Articles

Develop a Proof-of-Concept DLL with C++ multi-threading support for MetaTrader 5 on Linux

Wasin Thonkaew, 2023.01.27 16:22

We will begin the journey to explore the steps and workflow on how to base development for MetaTrader 5 platform solely on Linux system in which the final product works seamlessly on both Windows and Linux system. We will get to know Wine, and Mingw; both are the essential tools to make cross-platform development works. Especially Mingw for its threading implementations (POSIX, and Win32) that we need to consider in choosing which one to go with. We then build a proof-of-concept DLL and consume it in MQL5 code, finally compare the performance of both threading implementations. All for your foundation to expand further on your own. You should be comfortable building MT related tools on Linux after reading this article.
 
Fernando Carreiro #:

There are not that many examples of DLL written C/C++ for MQL, because they are not much different from any other DLL usage for normal C/C++ applications.

The available examples are enough to show that you can just write them is if you were coding for a pure Windows application.

However, here is an example with Linux in mind ...

Going to give this another shot - followed the steps to this point -


There are a *lot* of .h files in a *lot* of directories pulled from the git clone. This pic shows a *very* short list...:

Therefore, a cp command doesn't get us there - but rather a tarball especially if all the sub dirs are relative. I'm willing to go down this rabbit hole but need a little more grounding.

 
Dennis Jorgenson #:

Going to give this another shot - followed the steps to this point -


There are a *lot* of .h files in a *lot* of directories pulled from the git clone. This pic shows a *very* short list...:

Therefore, a cp command doesn't get us there - but rather a tarball especially if all the sub dirs are relative. I'm willing to go down this rabbit hole but need a little more grounding.

LOL! Figured this out *finally*. It was always in the linker ..

First, I started by downloading a pure windows-based 32bit gcc compiler (TDM-GCC-32) - this may not be needed as long as long as gcc multilib is installed; (to use the gcc multilib, add the params -m32 $* to your cli compile opts)

Having successfully compiled the dll (i.e., 'error-free'), for 'some odd reason' the EA was unable to import the exported functions.

wine g++.exe -v -shared -fdiagnostics-color=always -g "$MT4_HOME/Scripts/Examples/DLL/DLLSampleTest.cpp" -o "$MT4_HOME/Scripts/Examples/DLL/DLLSampleTest.dll"

So, I downloaded the dependency walker and saw that all of the function exports had an ordinal attached (below). The exported function names attached an ordinal ('@NN') - which is why the .def file was supplied with the sample.


Not wanting to use a static .def file, after many hours, I stumbled into upon how to list the linker options and found the '--kill-at' linker option.



wine g++.exe -v -shared -fdiagnostics-color=always -g "$MT4_HOME/Scripts/Examples/DLL/DLLSampleTest.cpp" -o "$MT4_HOME/Scripts/Examples/DLL/DLLSampleTest.dll" -Wl,--dll,--compat-implib,--kill-at

Compile worked, error free and all of the attached ordinals were removed.



Then, the rerun:


Success!

*** Edit: One Other thing - very important ***

I added 'extern "C"' to the MT4_EXPFUNC #define...


That is all. Questions?
New comment
Reason: