Libraries: JSON Serialization and Deserialization (native MQL) - page 8

 
o_o:

The library works in MQL4 and MQL5

Gratitude is accepted in the form of examples of your practical work with web resources from MQL. ;)


Good afternoon!

Thank you for the library!

I used your library to monitor the work of computers connected to a pool for cryptocurrency mining.

The pool (it's website) in response to a status query gives a response in the form of:

/*

{"status": "OK",

"data":[

{"worker": "riga1", "lastSeen":1511432055, "currentHashrate":2746.6666666666665,"validShares":103,"invalidShares":0,"staleShares":null},

{"worker":"riga2","lastSeen":1511432068,"currentHashrate":3040,"validShares":114,"invalidShares":1,"staleShares":null},

{"worker":"riga3","lastSeen":1511432057,"currentHashrate":2426.6666666666665,"validShares":91,"invalidShares":0,"staleShares":null},

{"worker":"riga4","lastSeen":1511432055,"currentHashrate":2266.6666666666665,"validShares":85,"invalidShares":0,"staleShares":null},

{"worker":"riga5","lastSeen":1511432061,"currentHashrate":2666.6666666666665,"validShares":100,"invalidShares":0,"staleShares":null},

{"worker":"riga6","lastSeen":1511432050,"currentHashrate":2293.3333333333335,"validShares":86,"invalidShares":0,"staleShares":null},

{"worker":"riga7","lastSeen":1511432069,"currentHashrate":1013.3333333333334,"validShares":38,"invalidShares":0,"staleShares":null}]}

*/

riga1, riga2, etc. - these are computers, we need to monitor that their "currentHashrate" does not fall below a threshold value.

The code is very simple:

void Fly_API_Monitor()
{

int res=0; string url;  string cook=""; string ref=""; string res_header=""; 
char result[]; 
char data[];
string strRes="";
// Тут вписывается свой кошелек
string wallet="t1L8S ............";
ResetLastError();
url="https://api-zcash.flypool.org/miner/" + wallet + "/workers/monitor"; 
res=WebRequest("GET", url, cook, ref, 5000, data,0, result, res_header);
strRes=CharArrayToString(result,CP_ACP);
if (res !=-1)
{
//Print("No error");
//Print(strRes);
CJAVal dataM;
dataM.Deserialize(strRes);
string stat=dataM["status"].ToStr();
int i=0;
for(i=0; i< ArraySize(dataM["data"].m_e); i++) 
{ 
Print(dataM["data"].m_e[i]["worker"].ToStr()+" Hashrate: "+ dataM["data"].m_e[i]["currentHashrate"].ToStr());
}
}
else 
{
Print("Error "+IntegerToString(GetLastError()));
Print(strRes);
}
}

outputs like:

riga7 Hashrate: 826.6666

riga6 Hashrate: 2293.333

riga5 Hashrate: 3200.

riga4 Hashrate: 1866.666

riga3 Hashrate: 2800

riga2 Hashrate: 2853.333

riga1 Hashrate: 2933.333


 

Thanks a lot to the developers of the library, but it worked very slowly, usually before you put out any product or code, it should be profiled, but in this case, since the library is free, there is no need to complain, the library works very slowly, if you do not do this:

было:

        virtual CJAVal* AddBase(const CJAVal &item) { int c=ArraySize(m_e); ArrayResize(m_e, c+1); m_e[c]=item; m_e[c].m_parent=GetPointer(this); return GetPointer(m_e[c]); } // adding
        virtual CJAVal* NewBase() { int c=ArraySize(m_e); ArrayResize(m_e, c+1); return GetPointer(m_e[c]); } // adding

стало:

        virtual CJAVal* AddBase(const CJAVal &item) { int c=ArraySize(m_e); ArrayResize(m_e, c+1, 50000); m_e[c]=item; m_e[c].m_parent=GetPointer(this); return GetPointer(m_e[c]); } // adding
        virtual CJAVal* NewBase() { int c=ArraySize(m_e); ArrayResize(m_e, c+1, 50000); return GetPointer(m_e[c]); } // adding

Otherwise here is what was before (the Web request itself on this screen took 16,718,439 in time, all the rest of the processing):

and after tweaking (the arrow shows the same Web request):

The speedup was almost 20 times faster.

 
Daniil Kurmyshev:

Why 50,000?

not 20,000,000 or 100,000?

--

what kind of test data did you use?

 

on the data, which in total turns out to be from 10000 - 30000, for this reason I chose 50000 with a reserve), but earlier I used this library on values of about 280 in the array, in this version and 500 is enough, 50000 is certainly for those who have the resources of the system allow.

 

according to the documentation, the reservation parameter simply reduces the number of memory reallocations

please check on your test sample with parameter 100 (1000, 10000)

what will be the speed gain in this case on the profiling screen

 
o_o:

according to the documentation, the reservation parameter simply reduces the number of memory reallocations

please check on your test sample with parameter 100 (1000, 10000)

what will be the speed gain on the profiling screen?


Yes it is exactly like that, I personally picked up the optimal parameter of redistribution, now the longest function of all is deserialisation..., but it is clear in principle, although in idea as I understand in this library everything is made by methods of search and maximum universality, but I still think that somewhere else in deserialisation there is a nuance that can be optimized.... I have attached a screenshot of profiling, memory reallocation parameters 5000.


 

1.12

set step 100 in ArrayResize

Files:
JAson.mqh  31 kb
 
Note that the profiler works on unoptimised code.

Therefore, when recompiling in a release after a very good optimiser, bottlenecks will go to completely different zones.

We will try to use semi-release optimisation for the profiler a little later. There are quite a lot of problematic issues there.
[Deleted]  
Hi there, thanks for this native MQL and fast library, it will be more helpful if there is a documentation, special when working with JSON arrays and objects, how to extract data from them:
Suppose:
[[0.88678,true],[0.88668,false]]
How can I map this to MQL types after Deserialize?

 

Dear specialists! Please help me to understand the library. ( I am not strong in object-oriented programming yet, so I can't understand the code of the library itself and in the article dedicated to the BTC-E exchange. that's why I ask for a simple example as in the post ).

I need to parse two public api of WEX.NZ exchange.

https://wex.nz/api/3/info

{
  "server_time": 1519120845,
  "pairs": {
    "btc_usd": {
      "decimal_places": 3,
      "min_price": 0.1,
      "max_price": 500000,
      "min_amount": 0.001,
      "hidden": 0,
      "fee": 0.2
    },
    "btc_rur": {
...

from here, I need to get all available pairs (btc_usd, btc_rur, ...) in an array (just pair names, without parameters, I can get parameters by known names...).


https://wex.nz/api/3/depth/btc_usd

{
  "btc_usd": {
    "asks": [
      [
        11550,
        0.00998
      ],
      [
        11559.9,
        0.01731136
      ],
...
      ]
    ],
    "bids": [
      [
        11506.031,
        1.756
      ],
      [
        11506.03,
        1.23
      ],
...
      ]
    ]
  }
}


and from here to two arrays (asks and bids) to get the price and value of orders.

I will be very grateful for help!!! Thanks in advance!