Libraries: JSON Parser

[MetaQuotes]

JSON Parser:

This is a Class for the new MQL4 that parses JSON. It requires the Hash.mqh class.

Author: ydrol

[femto]

Hello,

 

I've found some bugs in this library.

Some help will be nice.

Unfortunatelly I sent an email to ydrol some days ago but I didn't get any reply about it.

 

JSON string was:

      string s="{ \"firstName\": \"John\", \"lastName\": \"Smith\", \"age\": 25, \"size\": 1.78, \"address\": { \"streetAddress\": \"21 2nd Street\", \"city\": \"New York\", \"state\": \"NY\", \"postalCode\": \"10021\" }, \"phoneNumber\": [ { \"type\": \"home\", \"number\": \"212 555-1234\" }, { \"type\": \"fax\", \"number\": \"646 555-4567\" } ], \"gender\":{ \"type\":\"male\" }  }";

 

First: 

isNumber() function doesn't return true when a number is in JSON string.

            JSONValue*jsv;

            jsv=jo.getValue("age");

            unittest.assertFalse(jsv.isNull()); // there is a bug here

            unittest.assertTrue(jsv.isNumber()); // there is a bug here

 

 

Second:

Safe access to an integer number in a JSON string doesn't  work as expected.

 

            int age = 0;

            if (jo.getInt("age",age) ) { // there is a bug here (return false... should return true)

               unittest.assertEquals(age, 25);

            } else {

               unittest.assertTrue(false, "Can't access age as int"); // there is a bug here       

            }

 

 

 

This library (and also hashmap implementation) is really interesting I think we should put it on GitHub in order to improve it.

I wrote some unittest code using https://github.com/femtotrader/mt4-unittest 

 

See as attachment a unittest code to show you problems.

 

I hope someone here can help. If I can't fix that I will only use string data in JSON. 

 

Kind regards 

[femto]

I've just found an other bug in this library.

{

"params": [], 

"jsonrpc": "2.0", 

"method": "callit", 

"id": "a3885e29-2158-4ba1-a366-3ac5aaedfb95"

} 

is a valid JSON RPC request (without parameters) (see http://jsonlint.com/ )

but JSONParser fails with the following error:

(non-string passed) JSONParser::Error error parsing value at position 12 expected }(125) got ,(44) expected " or }  at 13 [, "jsonrpc...]

It seems that with library doesn't like empty lists 

 

I've found a workaround ...

{

"params": [""], 

"jsonrpc": "2.0", 

"method": "callit", 

"id": "a3885e29-2158-4ba1-a366-3ac5aaedfb95" 

 }

But that's not a very good idea... fixing JSONParser will be better

 

see as attachment my unittest for this JSON parser 

 

[erdeszt]

hi femto!

lately i've built a json parser in mql5. it's working fine for the examples you gave above.

i don't have examples for it but the interfaces pretty clear(check the jsonvalue constructors and json wrapper).

it currently uses a simple binary search tree implementation for objects which will obviously slow but you can change it to a self balancing bst or a hashmap.

i hope it'll be useful, here's the link:  https://gist.github.com/erdeszt/78db84a11e31cbd92d31 

[astera]

erdeszt:

hi femto!

lately i've built a json parser in mql5. it's working fine for the examples you gave above.

i don't have examples for it but the interfaces pretty clear(check the jsonvalue constructors and json wrapper).

it currently uses a simple binary search tree implementation for objects which will obviously slow but you can change it to a self balancing bst or a hashmap.

i hope it'll be useful, here's the link:  https://gist.github.com/erdeszt/78db84a11e31cbd92d31 

Hi erdeszt!

Great JSON parser you built! Could you maybe give an example of how you would use it to apply it to the example in mql4 shown below and how this would be converted to use in mql5? It will really help me a lot.

string s = "{ \"firstName\": \"John\", \"lastName\": \"Smith\", \"age\": 25, "+
"\"address\": { \"streetAddress\": \"21 2nd Street\", \"city\": \"New York\", \"state\": \"NY\", \"postalCode\": \"10021\" },"+
" \"phoneNumber\": [ { \"type\": \"home\", \"number\": \"212 555-1234\" }, { \"type\": \"fax\", \"number\": \"646 555-4567\" } ],"+
" \"gender\":{ \"type\":\"male\" }  }";

    JSONParser *parser = new JSONParser();

    JSONValue *jv = parser.parse(s);

    if (jv == NULL) {
        Print("error:"+(string)parser.getErrorCode()+parser.getErrorMessage());
    } else {


        Print("PARSED:"+jv.toString());
        
        if (jv.isObject()) { // check root value is an object. (it can be an array)

            JSONObject *jo = jv;

            // Direct access - will throw null pointer if wrong getter used.
            Print("firstName:" + jo.getString("firstName"));
            Print("city:" + jo.getObject("address").getString("city"));
            Print("phone:" + jo.getArray("phoneNumber").getObject(0).getString("number"));

            // Safe access in case JSON data is missing or different.
            if (jo.getString("firstName",s) ) Print("firstName = "+s);

            // Loop over object keys
            JSONIterator *it = new JSONIterator(jo);
            for( ; it.hasNext() ; it.next()) {
                Print("loop:"+it.key()+" = "+it.val().toString());
            }
            delete it;
        }
        delete jv;
    }
    delete parser;

Thanks.

[erdeszt]

astera:
Hi erdeszt!

Great JSON parser you built! Could you maybe give an example of how you would use it to apply it to the example in mql4 shown below and how this would be converted to use in mql5? It will really help me a lot.

Thanks.

Hi!

 Sorry for the late answer, hope it still helps. 

Also I've updated the gist on the link because I've noticed an off-by one error and a missing check in the .at function.

So here's your example translated to my lib: 

#include <Json.mqh>
#include <SmartPtr.mqh>

void OnInit() {
   string s = "{ \"firstName\": \"John\", \"lastName\": \"Smith\", \"age\": 25, \"address\": { " +
                         "\"streetAddress\": \"21 2nd Street\", \"city\": \"New York\", \"state\": \"NY\", \"postalCode\": \"10021\"" + 
                   " }," +
                 " \"phoneNumber\": [ " +
                        "{ \"type\": \"home\", \"number\": \"212 555-1234\" }, " + 
                        "{ \"type\": \"fax\", \"number\": \"646 555-4567\" }" + 
                  " ],"+
                 " \"gender\": { \"type\":\"male\" }  }";
                                  
   Json JSON;
   
   SmartPtr pResult(JSON.parse(ss));
   JsonParseResult* result = pResult.value();
   
   if (result.isSuccessful()) {
      Print("Successfully parsed json");      
      Print("firstName:" + result.value().get("firstName").stringValue());
      Print("city:" + result.value().get("address").get("city").stringValue());
      Print("phone:" + result.value().get("phoneNumber").at(0).get("number").stringValue());

      // The .get function returns NULL on a missing key and the .at function dies when the index is out of range,
      // but feel free to change these deaults.
      // Iteration is supported through:
      // obj.arrayValue() : CArrayObj*
      // and
      // obj.objectValue() : Map* (which has a keys() : CArrayString* method)
   }
   else {
      Print("Something went wrong! " + result.error());
   }
}

[paulbeestonuk]

Hi Erdeszt,

Thanks for sharing this library.
Please can you help me with a specific example of iterating over an array accessing the elements in it?
The elements I am trying to access are in the JSON below.
I can see your comments about handling arrays above but I cannot work out what I need to do.
The method below works to access the values of a single JSON element - I just need some help accessing the rest!

Thanks

This code works for a single element:

void TestJSON()
{
   string cookie = NULL;
   string headers;
   int timeout = 5000;
   int res = 0;
   char post[];
   uchar get[];
   string str_get;
   PredictedTrade preds[10];
   bool finished = false;
   string test_str;
               
   res = WebRequest("GET", "http://localhost/atp/trades/1", cookie, NULL, timeout, post, 0, get, headers);
   
   str_get = CharArrayToString(get, 0, -1, CP_UTF8);
   
   Json JSON;
   SmartPtr pResult(JSON.parse(str_get));
   JsonParseResult* result = pResult.value();

   if (result.isSuccessful())
   {
     preds[0].id = StringToInteger(result.value().get("id").stringValue());
     preds[0].instrument = result.value().get("instrument").stringValue();
   }
}

 

This is the JSON I am trying to read. 

[
    {
        "id": 1,
        "instrument": "FTSE",
        "experiment_id": 9,
        "trade_id": 936,
        "date_time_opened": "0000-00-00 00:00:00",
        "date_time_closed": "0000-00-00 00:00:00",
        "status": "Pending",
        "position": "Long",
        "st_entry_price": 6487.9702148438,
        "st_bet_size": 6,
        "st_stop_price": 6465.2998046875,
        "st_risk_pc": 1.5,
        "st_target_price": 6603.8999023438
    },
    {
        "id": 2,
        "instrument": "DJIA",
        "experiment_id": 9,
        "trade_id": 935,
        "date_time_opened": "2014-11-04 03:03:00",
        "date_time_closed": "0000-00-00 00:00:00",
        "status": "Pending",
        "position": "Long",
        "st_entry_price": 17366.240234375,
        "st_bet_size": 2,
        "st_stop_price": 17310.19921875,
        "st_risk_pc": 1.5,
        "st_target_price": 17481.599609375
    }
]

[femto]

Hi Erdeszt, hi paulbeestonuk,

 

Sorry I didn't noticed that this topic get replies.

Thanks erdeszt for this JSON parser.

you might use my unit test library.

https://github.com/femtotrader/mt4-unittest

 

It's a great way to both provide samples of your JSON parser and to ensure that it's working correctly. 

 

Maybe you should make a GitHub project for this (with issues, ...)

 

Kind regards

 

Femto 

[ydrol]

Hi Femto, sorry I did not reply to your emails and bug reports earlier. Work+Life :) I now have some free time and will fix the bugs, and perhaps re-visit trading again (after reviewing the landscape) Thanks Erdeszt for mql5 version, can I ask we separate these out to avoid confusion?

[ydrol]

I've fixed the code - The new code should be on the external site.  I'm waiting to hear from Service Desk to find out how to update the codebase here because I cant find the required links ...

[marquez]

erdeszt:

hi femto!

lately i've built a json parser in mql5. it's working fine for the examples you gave above.

i don't have examples for it but the interfaces pretty clear(check the jsonvalue constructors and json wrapper).

it currently uses a simple binary search tree implementation for objects which will obviously slow but you can change it to a self balancing bst or a hashmap.

i hope it'll be useful, here's the link:  https://gist.github.com/erdeszt/78db84a11e31cbd92d31 

Hi,

I hope this is not already offtopic. The Link causes a 404 error. Can someone upload the JSON MQL5 Parser, or can someone tell me if there is a JSON Parser Library available which works with MT5 & MT4 ? The MQL4 JSON Parser isn´t such as easy to port to MQL5 without many changes.

 

Thanks.

marquez