I am curious, has anyone suggestion on how to trace MQH function calls in such way that it is possible to print to the

Dominik Egert 2023.04.28 14:51 #41

I am curious, has anyone a suggestion on how to solve following task.

The main idea is to trace MQL API function calls in such way, that it is possible to print to the journal following informations:

File and line where the call to a function has taken place, and if provided, the return value and the current error code state.

For this, I defined a macro for each function name by the list of functions from the MQL documentation.

Now I would want to substitute the function name with some code, so I can inject these details. - For this I have created an object.

            // MQL Function return value message injection object
            static class pass_through
            { public:
                string _type;
                string _msg;
                pass_through() : _msg (NULL)                            { };
                pass_through(const pass_through& s)                     { _msg = s._msg; _type = s._type; }
                pass_through* msg(const string in, const string t_in)   { _type = t_in; _msg = in; ResetLastError(); return(GetPointer(this)); }
                template <typename T>
                const T operator=(T in)                                 {
                                                                            if(LIB_DBG_NAMESPACE(dbg_lib, trace_api_calls))
                                                                            {
                                                                                const int err_no = GetLastError();
                                                                                const string _typename = LIB_DBG_NAMESPACE(dbg_lib, dbg_typename_to_string)(typename(T), true);
                                                                                LIB_DBG_NAMESPACE(dbg_lib, dbg_printf)("%s", StringFormat(_msg, (err_no != ERR_SUCCESS) ? StringFormat("Error: %i", err_no) : ((StringFind(_type, _typename) == -1) ? StringFormat("Type mismatch. Expected: %s, Got: %s", _type, _typename) : LIB_DBG_NAMESPACE(dbg_lib, var_out)("returns", in, NULL, "", 45))));
                                                                                _msg = NULL;
                                                                            }
                                                                            return(in);
                                                                        }
            } LIB_DBG_NAMESPACE_DEF(dbg_lib, dbg_mql_api_retval);

now this would change a function call

double some_value = ChartGetDouble(ChartID(), CHART_PRICE_MIN);

into this:

double some_value = dbg_mql_api_retval = ChartGetDouble(ChartID(), CHART_PRICE_MIN);

The Object dbg_mql_api_retval would now handle the output accordingly by using the overloaded operator= as a function call being called after the ChartGetDouble-call is performed. - So far so good, but I have issues with when this is inside an if-statements condition, like this:

if(ChartGetDouble(ChartID(), CHART_PRICE_MIN) == DBL_MIN)
{ ... }

ChartGetDouble() is not the best example here, but the problem is the evaluation of the result, as it could be also the other way around, first DBL_MIN, then the function call.

The relevan code is included in lib_debug.mqh already, and can be found at lines 984 to 986 and1084 to 1119 as well as 2287 to 2882. - I had begun to change the function macros, please ignore the second parameter for now, the string constant of the return type)

My issue is with the evaluation of the return values within conditional evaluation blocks, like if-statements.

I had to solve it this way, because only at that point are the informations available, I want to print. - So even rewriting all functions with a wrapper would not solve the problem. - I need the values from the complier macros __FILE__, __FUNCTION__ and __LINE__.

Any ideas??

EDIT (updated):

static bool trace_api_calls = true;
static int trace_call_depth = NULL;

#define DBG_OUTPUT_PREFIX                     "   "
#define DBG_CODE_LOCATION_STRING              __FILE__, __FUNCTION__, __LINE__
#define DBG_OUTPUT_STRING                     "%s>%s< %s(){ @%i: %s }"
#define DBG_OUTPUT_FORMAT(prefix, message)    DBG_OUTPUT_STRING, ((prefix == "") ? "" : prefix + " "), DBG_CODE_LOCATION_STRING, message


#define DBG_MSG_MQLAPI(x)                       printf((trace_api_calls) ? "%*s%s" : "", trace_call_depth * 2, "", StringFormat(DBG_OUTPUT_PREFIX + DBG_OUTPUT_PREFIX + DBG_OUTPUT_FORMAT("", (x))))
#define DBG_MSG_MQLFUNC(x)                      DBG_MSG_MQLAPI(StringFormat((trace_api_calls) ? "MQL5-API Function => %s() [void]" : "", #x)); x
#define DBG_MSG_MQLFUNC_RETURN(x, y)            dbg_mql_api_retval.msg(StringFormat("%*s%s", trace_call_depth * 2, "", StringFormat(DBG_OUTPUT_PREFIX + DBG_OUTPUT_PREFIX + DBG_OUTPUT_FORMAT("", (StringFormat("%s => %s() %35s", "MQL5-API Function", #x, "%s"))))), y) = x



            // Enumerations and unknown types
            template <typename T>
            const string    var_out(const string name, T&                           val,    const int shift = 0, const string   prefix = "",    const int offset = 0)                             { return(StringFormat("%s[obj]  %-" + IntegerToString(60 - ((offset < 60) ? offset : NULL)) + "s = '%s'",                 prefix, name, typename(val))); }


            static class pass_through
            { public:
                string _type;
                string _msg;
                pass_through() : _msg (NULL)                            { };
                pass_through(const pass_through& s)                     { _msg = s._msg; _type = s._type; }
                pass_through* msg(const string in, const string t_in)   { _type = t_in; _msg = in; ResetLastError(); return(GetPointer(this)); }
                template <typename T>
                const T operator=(T in)                                 {
                                                                            if(trace_api_calls)
                                                                            {
                                                                                const int err_no = GetLastError();
                                                                                const string _typename = typename(T);
                                                                                printf("%s", StringFormat(_msg, (err_no != ERR_SUCCESS) ? StringFormat("Error: %i", err_no) : ((StringFind(_type, _typename) == -1) ? StringFormat("Type mismatch. Expected: %s, Got: %s", _type, _typename) : var_out("returns", in, NULL, "", 45))));
                                                                                _msg = NULL;
                                                                            }
                                                                            return(in);
                                                                        }
            } dbg_mql_api_retval;



    #define FileClose                               DBG_MSG_MQLFUNC(FileClose)
    #define FileOpen                                DBG_MSG_MQLFUNC_RETURN(FileOpen, "int")
    #define FileSize                                DBG_MSG_MQLFUNC_RETURN(FileSize, "ulong")


class CTestObj
{   
    public:
    CTestObj()
    { };
    CTestObj(const CTestObj& p_in)
    { };
    
    const bool operator==(int p_in) const { printf("%s", "Call operator==()"); return(true); }
} test_obj;


CTestObj* Test_Func(const int f_h)
{ printf("%s", "Call Test_Func()"); return(GetPointer(test_obj)); };



void OnInit()
{

    int f_h = FileOpen("some_file.txt", FILE_READ | FILE_COMMON);
    ulong size_of_file = FileSize(f_h);

    // Order of operation is not as required for wanted results
    if(dbg_mql_api_retval.msg("FileSize()", "ulong") = Test_Func(f_h) == 5)
    {
            printf("Filesize != 0");
    }
    if(FileSize(f_h) == 5)
    {
            printf("Filesize != 0");
    }


    // Order of operation is correct for wanted results
    if((dbg_mql_api_retval.msg("FileSize()", "ulong") = Test_Func(f_h)) == 5)
    {
            printf("Filesize != 0");
    }
    if((FileSize(f_h)) == 5)
    {
            printf("Filesize != 0");
    }
    
    
    // Order of operation is not as required for wanted results
    if(FileSize(f_h) == 5)
    {
            printf("Filesize == 5");
    }
    

    // Order of operation is correct for wanted results
    if(5 == (FileSize(f_h)))
    {
            printf("5 == FileSize");
    }
    
    FileClose(f_h);
};

OOP, templates and macros OrderModify Error 130 with Personal indicator variable to

Alain Verleyen 2023.04.28 19:50 #42

Dominik Christian Egert #:

I am curious, has anyone a suggestion on how to solve following task.

The main idea is to trace MQL API function calls in such way, that it is possible to print to the journal following informations:

File and line where the call to a function has taken place, and if provided, the return value and the current error code state.

For this, I defined a macro for each function name by the list of functions from the MQL documentation.

Now I would want to substitute the function name with some code, so I can inject these details. - For this I have created an object.

now this would change a function call

into this:

The Object dbg_mql_api_retval would now handle the output accordingly by using the overloaded operator= as a function call being called after the ChartGetDouble-call is performed. - So far so good, but I have issues with when this is inside an if-statements condition, like this:

ChartGetDouble() is not the best example here, but the problem is the evaluation of the result, as it could be also the other way around, first DBL_MIN, then the function call.

The relevan code is included in lib_debug.mqh already, and can be found at lines 984 to 986 and1084 to 1119 as well as 2287 to 2882. - I had begun to change the function macros, please ignore the second parameter for now, the string constant of the return type)

My issue is with the evaluation of the return values within conditional evaluation blocks, like if-statements.

I had to solve it this way, because only at that point are the informations available, I want to print. - So even rewriting all functions with a wrapper would not solve the problem. - I need the values from the complier macros __FILE__, __FUNCTION__ and __LINE__.

Any ideas??

I would like to help but I don't have time to dive into your code. So I would suggest you to ask small well targeted question if possible.

Dominik Egert 2023.04.28 22:58 #43

Alain Verleyen #:
I would like to help but I don't have time to dive into your code. So I would suggest you to ask small well targeted question if possible.

Thank you for the offer.... Let me try.

As far as I can see, my problem is the sequence of operators.

Let me try to show:

// A normal function call to an MQL-API-Function.
double arr[];
int some_var = ArraySize(arr);

// This will get expanded by the macro as follows:
int some_var = dbg_mql_api_retval.msg("ArraySize()", "ulong") = ArraySize(arr);

// This will result in the member-function "msg" being called and returning a pointer to the object dbg_mql_api_retval.
// Next the overloaded operator "=" will be called from this object. This operator/function is designed as template and takes in whatever is on the right side of the "=" and returns this as a return value.
// Now the "=" from the assignment to the variable "some_var" is called and the value returnd by the operator=()-function will be assigned.

Inside of an evaluation block, for example within an if()-statement, following happens:

int arr[];

if(ArraySize(arr) == 5)
{ printf("%s", "Arraysize is 5"); }

// This will get expanded by the macro to following actual code:
if(dbg_mql_api_retval.msg("ArraySize", "int") = ArraySize(arr) == 5)
{ printf("%s", "Arraysize is 5"); }

// What happens now is following sequence:
// 1. member-function .msg gets called
// 2. ArraySize gets called
// 3. "==" operator is executed, resulting in a boolean value.
// 4. The boolean value gets assigned to the object returned by the member function .msg()
// The operator=() function returns the value assigned to it/passed to it. - Which is in this case "boolen true".

And here is what I need to happen ( Pos 1 and 2 can be swapped):

1. Member function "msg" gets called

2. ArraySize gets called

3. return value of ArraySize calls operator=()-function from object.

4. returned value from operator=() gets compared by "==" to the right side. (5)

I know by doing following, it can be done easily:

if((dbg_mql_api_retval.msg("ArraySize", "int") = ArraySize(arr)) == 5)
{ printf("%s", "Arraysize is 5"); }

But I cannot insert these brackets by macro...

So what I need to find is some way to influence the execution order without editing/substituting anithing after the MQL-API-Function-Name, since this is the only thing I can replace with a macro.

The solution to be used ineeds to be able to handle the other way around as well, like this:

if(5 == dbg_mql_api_retval.msg("ArraySize", "int") = ArraySize(arr))
{ printf("%s", "Arraysize is 5"); }

How could this be solved?

Addendum, the main issue is getting the "debug-infos" from where the function call takes place... - I am open to any approach on solving this, as it would enhance the lib_debug.mql library greatly. (And finally bring it to version 5.0...)

CTrade Stop Loss trigger Libraries: Benchmark what log system to

Alain Verleyen 2023.04.29 01:20 #44

Dominik Christian Egert #:

Thank you for the offer.... Let me try.

As far as I can see, my problem is the sequence of operators.

Let me try to show:

Inside of an evaluation block, for example within an if()-statement, following happens:

And here is what I need to happen ( Pos 1 and 2 can be swapped):

1. Member function "msg" gets called

2. ArraySize gets called

3. return value of ArraySize calls operator=()-function from object.

4. returned value from operator=() gets compared by "==" to the right side. (5)

I know by doing following, it can be done easily:

But I cannot insert these brackets by macro...

So what I need to find is some way to influence the execution order without editing/substituting anithing after the MQL-API-Function-Name, since this is the only thing I can replace with a macro.

The solution to be used ineeds to be able to handle the other way around as well, like this:

How could this be solved?

Addendum, the main issue is getting the "debug-infos" from where the function call takes place... - I am open to any approach on solving this, as it would enhance the lib_debug.mql library greatly. (And finally bring it to version 5.0...)

Very clear, thank you. I will think about it.

Dominik Egert 2023.04.29 20:24 #45

Alain Verleyen #:
Very clear, thank you. I will think about it.

I think I solved it.

The solution seems to be here:

https://www.mql5.com/en/docs/basis/operations/rules

By using operator=, the execution is at the lower end of the chain.

So I changed the operator to "*", this ensures the correct execution order in this case.

I am currently implementing the code, and it looks promising.

Maybe it is consistent enough to work in all cases. At least my testing code works now.

Edit:

By you asking me to formulate a narrow question, I came across this idea.

Thank you for the inspiration

Can i use an Help with icustom() Buffer Indicator Miscellaneous Questions

amrali 2023.04.29 22:09 #46

Dominik Christian Egert #:

I think I solved it.

The solution seems to be here:

Great.

I am waiting for the release of the "API tracer".

Alain Verleyen 2023.04.29 22:48 #47

Dominik Christian Egert #:

I think I solved it.

The solution seems to be here:

https://www.mql5.com/en/docs/basis/operations/rules

By using operator=, the execution is at the lower end of the chain.

So I changed the operator to "*", this ensures the correct execution order in this case.

I am currently implementing the code, and it looks promising.

Maybe it is consistent enough to work in all cases. At least my testing code works now.

Edit:

By you asking me to formulate a narrow question, I came across this idea.

Thank you for the inspiration

Glad to help ;-)

Dominik Egert 2023.04.30 13:23 #48

amrali #:

Great.

I am waiting for the release of the "API tracer".

If you would like to test the current version, here is how to activate MQL-API-Tracing.

Please note, it is considered "alpha" stage at the moment as I am strugling with another issue at the moment. - See post here: https://www.mql5.com/en/forum/446559

I would appreciate testing and helping to make the code more stable in the process.

To test the functionality:

// Include the library 

// Activate debugging mode
#define LIB_DEBUG
//#define LIB_DEBUG_LOGFILE
//#define LIB_DEBUG_NO_JOURNAL_OUTPUT

// Activate MQL-API-Tracing 
//      (This will activate tracing when a function is in debug trace mode (LIB_DEBUG code applied as documented)
#define LIB_DEBUG_MQLAPI

// Additionally force API-Tracing to be on at all times. 
#define LIB_DEBUG_MQLAPI_ALL_CALLS

// Actually include the library
#include <MQLplus/lib_debug.mqh>

Since this code is alpha/early beta, it is only available via cloud storage and part of the "MQL plus" project.

EDIT:

Do not use MQL-API-Calls with other macros from the library, this is currently broken when MQL-API-TRACING is enabled. Example:

DBG_MSG_VAR(TimeCurrent());

This will make the program crash with an "access violation" error.

Possible Preprocessor Bug

2023.04.30
www.mql5.com

General: Possible Preprocessor Bug

Access violation write to how to make a DLL string function does

amrali 2023.04.30 16:42 #49

Dominik Christian Egert #:

If you would like to test the current version, here is how to activate MQL-API-Tracing.

Please note, it is considered "alpha" stage at the moment as I am strugling with another issue at the moment. - See post here: https://www.mql5.com/en/forum/446559

What do you think of implementing the API tracer like this:

It handles all operators, nicely. https://www.mql5.com/en/docs/basis/operations/rules

bool trace_api_calls = true;

#define DBG_CODE_LOCATION_STRING              __FILE__, __FUNCTION__, __LINE__
#define DBG_MSG_MQLFUNC_RETURN(x, y)          pass_through(DBG_CODE_LOCATION_STRING, y, x)


            template <typename T>
            T pass_through(string file, string func, int line, string title, T retval)
              {
               if (trace_api_calls)
                 {
                  string prefix = StringFormat(">%s< %s() @%i: %s returns ", file, func, line, title);
                  Print(prefix, retval);
                 }
               return retval;
              }


    #define ArraySize(x)                            DBG_MSG_MQLFUNC_RETURN(ArraySize(x), "ArraySize")
    #define ArrayBsearch(x, v)                      DBG_MSG_MQLFUNC_RETURN(ArrayBsearch(x, v), "ArrayBsearch")
    #define MathRand()                              DBG_MSG_MQLFUNC_RETURN(MathRand(), "MathRand")

    #define EXPAND_MACRO(macro)                     Print(#macro)


void OnStart()
  {
   string arr[] = {"a", "b", "c", "d"};

   int some_var = ArraySize(arr);                 // >test.mq5< OnStart() @29: ArraySize returns 4
   Print(ArraySize(arr));                         // >test.mq5< OnStart() @30: ArraySize returns 4
   Print(some_var = ArraySize(arr));              // >test.mq5< OnStart() @31: ArraySize returns 4
   Print(ArraySize(arr) == 4);                    // >test.mq5< OnStart() @32: ArraySize returns 4
   Print(4 == ArraySize(arr));                    // >test.mq5< OnStart() @33: ArraySize returns 4
   Print(ArraySize(arr) - 1 == 3);                // >test.mq5< OnStart() @34: ArraySize returns 4
   Print(arr[ArraySize(arr) - 1]);                // >test.mq5< OnStart() @35: ArraySize returns 4
   Print(arr[ArraySize(arr) - 1] == "d");         // >test.mq5< OnStart() @36: ArraySize returns 4

   Print(MathRand() % ArraySize(arr));            // >test.mq5< OnStart() @38: MathRand returns 20794
                                                  // >test.mq5< OnStart() @38: ArraySize returns 4

   Print(typename(ArraySize(arr)));               // int
   Print(sizeof(ArraySize(arr)));                 // 4

   EXPAND_MACRO(ArraySize(arr));                  // pass_through(__FILE__,__FUNCTION__,__LINE__,ArraySize,ArraySize(arr))
   EXPAND_MACRO(MathRand() % ArraySize(arr));     // pass_through(__FILE__,__FUNCTION__,__LINE__,MathRand,MathRand())%pass_through(__FILE__,__FUNCTION__,__LINE__,ArraySize,ArraySize(arr))
  }

Documentation on MQL5: Language Basics / Operations and Expressions / Precedence Rules

www.mql5.com

Precedence Rules - Operations and Expressions - Language Basics - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5

Alain Verleyen 2023.04.30 17:26 #50

amrali #:

What do you think of implementing the API tracer like this:

It handles all operators, nicely. https://www.mql5.com/en/docs/basis/operations/rules

Typo in "EXAPND"...should be "EXPAND".

Libraries: MQL Plus Enhanced Debugging Support - page 5