Base64 serialisation of POD structures in MML5: What is the maximum size of string that MQL5 can store? - General

Alain Verleyen 2020.02.12 20:40 #26451

Andrei Trukhanovich :

because the template only outputs types by parameter, not by output value.

This is not accurate.

struct A
  {
   int               i;
   double            d;
  };

template<typename T>
T f(int value)
  {
   T result={0};
   return(result);
  }
//+------------------------------------------------------------------+
void OnStart()
  {
   A a;
   int i = 1;
   a = f<A>(i);
  }

Igor Makanu 2020.02.12 20:42 #26452

Alain Verleyen:

That's not accurate.

Thank you!

Yes, it will work, got confused in the syntax

ZS: everything as intended now works:

struct A
  {
   long              l;
   uint              ui;
  };
//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
   A a;
   uchar u_arr[];
   string result = "";
   a.l  = 0xFFFFFFFFFFFFFFFF;
   a.ui = 0x0000;
//--- проверка работоспособности вывода
   StructToCharArray(a, u_arr);
   for(int i = 0; i < ArraySize(u_arr); i++)
     {
      result += StringFormat("%x", u_arr[i]);
     }
   printf("result = %s", result);   // result = ffffffffffffffff0000

//----  
   string s = StructEncodeBase64(a);
   Print(s);
   A b;
   b = StructDecodeBase64<A>(s);

   StructToCharArray(b, u_arr);
   result="";
   for(int i = 0; i < ArraySize(u_arr); i++)
     {
      result += StringFormat("%x", u_arr[i]);
     }
    printf("result = %s", result);

  }
//+------------------------------------------------------------------+
template<typename T>
string StructEncodeBase64(T &s_value)
  {
   const uchar key[1] = {0};
   uchar data[], result[];
   StructToCharArray(s_value, data);
   CryptEncode(CRYPT_BASE64, data, key, result);
   return(CharArrayToString(result));
  }
//+------------------------------------------------------------------+
template<typename T>
T StructDecodeBase64(string value)
  {
   const uchar key[1] = {0};
   uchar data[], decode[];
   T result;
   StringToCharArray(value, data, 0, StringLen(value));
   CryptDecode(CRYPT_BASE64, data, key, decode);
   CharArrayToStruct(result, decode);
   return(result);
  }

Base64 serialisation of POD structures

Andrei Trukhanovich 2020.02.12 21:39 #26453

Alain Verleyen:

That's not for sure.

No, it's not. You noted the explicit setting of template parameter types in the function, and I was talking about output, about the explicit setting fxsaber answered before me. and before you

Alain Verleyen 2020.02.13 00:52 #26454

Andrei Trukhanovich :

No, that's right. You noted the explicit setting of template parameter types in the function, and I was talking about output, about the explicit setting fxsaber answered before me. and before you

Language problem.

Igor Makanu 2020.02.13 09:23 #26455

I couldn't find it on the forum or in the docs, so I have a question:

1. what is the maximum size of string that MQL5 can store?

2. what is the maximum size of string that MQL4 can store?

3. if there is a memory shortage, this max. size will be reduced (or no memory allocated for string) or will there be a runtime error ?

Questions about MQL4 Questions from Beginners MQL4 Questions from Beginners MQL5

Sergey Dzyublik 2020.02.13 11:29 #26456

Defects:
(not fixed by MT5(build 2319)) bug in template class cache operation: "You create complex wrapped object with internal type "C" several times, and it turns out to be quite different data type, maybe "B", maybe "int", whatever you want...".
(not fixed by MT5(build 2319)) bug with template class code generation when using internal class.
(not fixed MT5(build 2319)) bug when defining internal class - no ability to explicitly reference global namespace when specifying a base class.

Suggestions:
ref - about the need to introduce typedef declaration functionality in MQL.
Reference- to allow to force generation of default copy constructors and assignment operators.

Any questions from newcomers [CLOSED] : Compiler bug Assignment operator / default

fxsaber 2020.02.13 14:51 #26457

Sergey Dzyublik:

Defects:

Good on you for keeping track of your bug reports. I didn't, that's why everything got irretrievably lost in the forum.

Maybe you should add some of the same hash record to your bug report posts, so you can search for them all at once.

Beta Testing of MetaTrader How to make a Any questions from newcomers

Igor Makanu 2020.02.13 16:50 #26458

Igor Makanu:

I couldn't find it on the forum or in the docs, so I have a question:

1. what is the maximum size of string that MQL5 can store?

2. what is the maximum size of string that MQL4 can store?

3. if there is insufficient memory this max. size will be reduced (or no memory allocated for string) or will there be a runtime error ?

tried to test a string in MT4

void OnTick()
  {
   static bool frun = true;
   uchar u_arr[102400];
   if(frun)
     {
      frun = false;
      string s = "";
      while(!IsStopped())
        {
         ArrayInitialize(u_arr, uchar(rand() % 255));
         s += CharArrayToString(u_arr);
         printf("StringLen = %i , m_physical = %i, m_total = %i, m_available = %i, m_used = %i", StringLen(s), TerminalInfoInteger(TERMINAL_MEMORY_PHYSICAL), TerminalInfoInteger(TERMINAL_MEMORY_TOTAL),
                TerminalInfoInteger(TERMINAL_MEMORY_AVAILABLE), TerminalInfoInteger(TERMINAL_MEMORY_USED));
         Sleep(150);
        }
     }
  }
//+------------------------------------------------------------------+

The memory leaks slowly (5.1 out of 11.9), in MT4 in the log:

StringLen = 221696000 , m_physical = 12157, m_total = 4095, m_available = 3365, m_used = 729

in general, for sensible tasks, you can use big string, maybe later I will copy the file through a string to check

UPD: replaceduchar u_arr[102400]; by uchar u_arr[524224]; to speed up process

Bottom line: in MT4, the approximate maximum that can be put in string is:

2020.02.13 21:11:24.177 tst_string EURUSD,H1: out of memory

2020.02.13 21:11:24.024 tst_string EURUSD,H1: StringLen = 640601728 , m_physical = 12157, m_total = 4095, m_available = 2556, m_used = 1539

i.e. about 640 MB

SZZ: MT5

2020.02.13 21:56:12.964 tst_string (EURUSD,H1) StringLen = 1073610752 , m_physical = 12157, m_total = 24314, m_available = 21975, m_used = 2339

2020.02.13 21:56:13.140 tst_string (EURUSD,H1) out of memory in 'tst_string.mq5' (41,12)

i.e. about 1GB

How to assign variable iTime and iBarShift return Empty market in the

Alain Verleyen 2020.02.13 20:00 #26459

Igor Makanu :

tried to test string in MT4

The memory is leaking a little bit in Win Manager (5.1 out of 11.9) , in MT4 in the log:

StringLen = 221696000 , m_physical = 12157, m_total = 4095, m_available = 3365, m_used = 729

in general, for sensible tasks, you can use big string, maybe later I will copy the file through a string to check

UPD: replaced uchar u_arr[ 102400 ]; by uchar u_arr[524224]; to speed up process

Bottom line: in MT4 the approximate maximum that can be put in string is:

2020.02.13 21:11:24.177 tst_string EURUSD,H1: out of memory

2020.02.13 21:11:24.024 tst_string EURUSD,H1: StringLen = 640601728 , m_physical = 12157, m_total = 4095, m_available = 2556, m_used = 1539

i.e. about 640 MB

SZZ: MT5

2020.02.13 21:56:12.964 tst_string (EURUSD,H1) StringLen = 1073610752 , m_physical = 12157, m_total = 24314, m_available = 21975, m_used = 2339

2020.02.13 21:56:13.140 tst_string (EURUSD,H1) out of memory in 'tst_string.mq5' (41,12)

i.e. about 1GB

This is string length for Unicode, so in MB it is at least double. 1.3 GB for MT4, which makes sense since you need a continuous memory buffer and MT4 is a 32 bit application. Thus the mql4 limitation is not a limitation in itself, but is only limited by the available memory.

I'm a bit surprised that MT5 doesn't allow more, as it seems like you have a lot of available memory. I'll try my best on my side.

What are some cloud MT4 64 bit is How many MT4 platforms

Igor Makanu 2020.02.13 20:18 #26460

Alain Verleyen:

This is the string length for Unicode, so in MB it is at least double. 1.3 GB for MT4, which makes sense since you need a continuous memory buffer and MT4 is a 32 bit application. So the limitation of mql4 itself is not a limitation, it is only limited by the available memory.

I'm a bit surprised that MT5 doesn't allow more, as it seems like you have a lot of available memory. I'll try my best on my side.

i have 12gb of memory, Windows 10 , free memory about 8gb

i needed this test to find out the maximum allowed line lengths in mt4/ mt5 terminals

I now want to exchange between terminals via the Redis database

Redis is a very simple database and works only with strings (key - value --> strin - string)

Yesterday I've serialized POD structures in Base64 and it increases data size by about 1/3

I think that even in this case I have a possibility to exchange structures with size of 400 Mb per transaction - that's more than enough for any purpose! ;)

Wishes for MT5 Long Only or Short My approach. The core

Errors, bugs, questions - page 2646