New MetaTrader 5 build 2360: Extension of SQLite integration

[MetaQuotes]

The MetaTrader 5 platform update will be released on Friday, March the 6th, 2020. The new version features the following updates:

1. MQL5: SQLite database operation errors can now be analyzed using standard MQL5 tools. The following error codes have been added:
   
   
   • ERR_DATABASE_ERROR — generic error.
   • ERR_DATABASE_INTERNAL — SQLite internal logic error.
   • ERR_DATABASE_PERM — access denied.
   • ERR_DATABASE_BUSY — database file locked.
   • ERR_DATABASE_LOCKED — database table locked.
     
   • ERR_DATABASE_NOMEM — insufficient memory for completing operation.
     
   • ERR_DATABASE_READONLY — attempt to write to read-only database.
     
   • ERR_DATABASE_IOERR — disk I/O error.
   • ERR_DATABASE_CORRUPT — database disk image corrupted.
   • ERR_DATABASE_FULL — insertion failed because database is full.
     
   • ERR_DATABASE_CANTOPEN — unable to open the database file.
   • ERR_DATABASE_PROTOCOL — database lock protocol error.
     
   • ERR_DATABASE_SCHEMA — internal use only.
   • ERR_DATABASE_TOOBIG — string or BLOB exceeds size limit.
   • ERR_DATABASE_CONSTRAINT — abort due to constraint violation.
   • ERR_DATABASE_MISMATCH — data type mismatch.
   • ERR_DATABASE_MISUSE — library used incorrectly.
   • ERR_DATABASE_AUTH — authorization denied.
   • ERR_DATABASE_RANGE — bind parameter error, incorrect index.
   • ERR_DATABASE_NOTADB — file opened that is not database file.
     
   
   
2. MQL5: Fixed operation of the DatabaseImport function which enables data import from a file into a database table.
3. MetaEditor: Fixed output of strings longer than 32KB to logs.
4. MetaEditor: Fixed occasional incorrect encoding in messages sent from the Python console (stdout, stderr) into the Errors section of the editor.
   
5. Updated documentation.
   

The new MetaTrader 5 version will be available through the LiveUpdate system.

[Daniel Weckhuyzen]

Hi,

Thx for the new release.

I've seen a bug that also appeared before in earlier releases in the "Market scanner" part of the backtester.

After a scan I see again the same symbol appearing more than once.

Regards,

Danny

[Sergey Golubev]

Forum on trading, automated trading systems and testing trading strategies

Learn Russian

Sergey Golubev, 2020.03.08 07:11

It is difficult to learn.
You can read/write on the Russian forum using the automatic translation tool which exists on every post (many users from English/Chinese/etc parts of the forum are using this automatic translation tool).

If you really want for some post (technical/programming issue/question) to be sent to the Russian forum (to the threads where the admins are replying on technical questions) - so ask the users to pass your post to Russian forum (because some moderators and users can read Russian/English, and many members are using automatic translation tool with no problem at all).

But it is much more better if you will use this automatic translation tool by yourself without asking anyone (on the same way as many people are using it).

[Daniel Weckhuyzen]

Hi,

I've discouvered a new bug in the backtester. I've added a url with a video that demonstrates the problem.

There's a difference in results when backtesting over a 1 day or 2 days period.

To resume what happens in the video:

first run: using OHLC over 1 day: the backtest shows 2 trades

second run: using OHLC over 2 days: the backtest shows for the most recent day 6 trades

third run: using OHLC (same as first run) : the backtest shows 6 trades

fourth run: using real ticks over 1 day: the backtest shows 2 trades

fifth run: using real ticks over 2 days: the backtest shows for the most recent day 6 trades

sixth run: using real ticks over 1 day (same as fourth run): the backtest shows 6 trades.

I hope the video will clarify the problem.

I've also seen a huge difference between OHLC backtesting and real-tick backtesting; it's like the indicators are wrongly calculated

PS: I've seen the same problem also in the latest beta release 2363

http://users.telenet.be/karaoke/backtest_problem.mp4

[Daniel Weckhuyzen]

Hi,

I've noticed another problem in the backtester and optimiser.

I've been setting breakpoints during visual backtesting and I 've seen that the "commission" costs are still not calculated in.

It's not possible to have reliable backtests when this does not work as expected.

I have tested 'real ticks' and OHLC; both have the same problem.

I have also added a url that demonstrates the problems

http://users.telenet.be/karaoke/commission_problem.mp4

Thx for your quick reply !

Danny

[Marcel Fitzner]

Any update on this? Or is it still an open issue?

[Daniel Weckhuyzen]

Hi, I have downloaded beta version 2400 and I'm experiencing "out of memory" problems during optimising.

I'm optimising over 3 years with OHLC bars; only when I reboot my PC (32 GB RAM, windows 10 64 bit), I can continue optimising and finish the rest of the not checked settings.

With earlier releases, this problem was dissapeared, but now it came back.

Regards,

Danny

[Andrew Saxby]

Hi All

Anyone also having the following error: ???

DatabaseBind issue? Method/function is missing ...... code is straight from the help file and examples 

Any help would be really appreciated 

//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
//--- open the dialog for selecting files with the DAT extension
   string selected_files[];
   if(!FileSelectDialog("Select files to download", NULL,
                       "Data files (*.dat)|*.dat|All files (*.*)|*.*",
                       FSD_ALLOW_MULTISELECT, selected_files, "tester.dat")>0)
     {
      Print("Files not selected. Exit");
      return;
     }
//--- get the size of files
   ulong filesize[];
   int filehandle[];
   int files=ArraySize(selected_files);
   ArrayResize(filesize, files);
   ZeroMemory(filesize);
   ArrayResize(filehandle, files);
   double total_size=0;
   for(int i=0; i<files; i++)
     {
      filehandle[i]=FileOpen(selected_files[i], FILE_READ|FILE_BIN);
      if(filehandle[i]!=INVALID_HANDLE)
        {
         filesize[i]=FileSize(filehandle[i]);
         //PrintFormat("%d, %s handle=%d %d bytes", i, selected_files[i], filehandle[i], filesize[i]);
         total_size+=(double)filesize[i];
        }
     }
//--- check the common size of files
   if(total_size==0)
     {
      PrintFormat("Total files size is 0. Exit");
      return;
     }
 
//--- create or open the database in the common terminal folder
   string filename="dat_files.sqlite";
   int db=DatabaseOpen(filename, DATABASE_OPEN_READWRITE | DATABASE_OPEN_CREATE);
   if(db==INVALID_HANDLE)
     {
      Print("DB: ", filename, " open failed with code ", GetLastError());
      return;
     }
   else
      Print("Database: ", filename, " opened successfully");
//--- if the FILES table exists, delete it
   if(DatabaseTableExists(db, "FILES"))
     {
      //--- delete the table
      if(!DatabaseExecute(db, "DROP TABLE FILES"))
        {
         Print("Failed to drop table FILES with code ", GetLastError());
         DatabaseClose(db);
         return;
        }
     }
//--- create the FILES table
   if(!DatabaseExecute(db, "CREATE TABLE FILES("
                       "NAME           TEXT NOT NULL,"
                       "SIZE           INT  NOT NULL,"
                       "PERCENT_SIZE   REAL NOT NULL,"
                       "DATA           BLOB NOT NULL);"))
     {
      Print("DB: failed to create table FILES with code ", GetLastError());
      DatabaseClose(db);
      return;
     }
//--- display the list of all fields in the FILES table
   if(DatabasePrint(db, "PRAGMA TABLE_INFO(FILES)", 0)<0)
     {
      PrintFormat("DatabasePrint(\"PRAGMA TABLE_INFO(FILES)\") failed, error code=%d at line %d", GetLastError(), __LINE__);
      DatabaseClose(db);
      return;
     }
 
//--- create a parametrized request to add files to the FILES table
   string sql="INSERT INTO FILES (NAME,SIZE,PERCENT_SIZE,DATA)"
              " VALUES (?1,?2,?3,?4);"; // request parameters
   int request=DatabasePrepare(db, sql);
   if(request==INVALID_HANDLE)
     {
      PrintFormat("DatabasePrepare() failed with code=%d", GetLastError());
      Print("SQL request: ", sql);
      DatabaseClose(db);
      return;
     }
 
//--- go through all the files and add them to the FILES table
   bool request_error=false;
   DatabaseTransactionBegin(db);
   int count=0;
   uint size;
   for(int i=0; i<files; i++)
     {
      if(filehandle[i]!=INVALID_HANDLE)
        {
         char data[];
         size=FileReadArray(filehandle[i], data);
         if(size==0)
           {
            PrintFormat("FileReadArray(%s) failed with code %d", selected_files[i], GetLastError());
            continue;
           }
 
         count++;
         //--- set the values of the parameters before adding the file to the table
         if(!DatabaseBind(request, 0, selected_files[i]))
           {
            PrintFormat("DatabaseBind() failed at line %d with code=%d", __LINE__, GetLastError());
            request_error=true;
            break;
           }
         if(!DatabaseBind(request, 1, size))
           {
            PrintFormat("DatabaseBind() failed at line %d with code=%d", __LINE__, GetLastError());
            request_error=true;
            break;
           }
         if(!DatabaseBind(request, 2, double(size)*100./total_size))
           {
            PrintFormat("DatabaseBind() failed at line %d with code=%d", __LINE__, GetLastError());
            request_error=true;
            break;
           }
         if(!DatabaseBindArray(request, 3, data))
           {
            PrintFormat("DatabaseBind() failed at line %d with code=%d", __LINE__, GetLastError());
            request_error=true;
            break;
           }
         //--- execute a request for inserting the entry and check for an error
         if(!DatabaseRead(request)&&(GetLastError()!=ERR_DATABASE_NO_MORE_DATA))
           {
            PrintFormat("DatabaseRead() failed with code=%d", GetLastError());
            DatabaseFinalize(request);
            request_error=true;
            break;
           }
         else
            PrintFormat("%d. %s: %d bytes", count, selected_files[i],size);
         //--- reset the request before the next parameter update
         if(!DatabaseReset(request))
           {
            PrintFormat("DatabaseReset() failed with code=%d", GetLastError());
            DatabaseFinalize(request);
            request_error=true;
            break;
           }
        }
     }
//--- transactions status
   if(request_error)
     {
      PrintFormat("Table FILES: failed to add %d files", count);
      DatabaseTransactionRollback(db);
      DatabaseClose(db);
      return;
     }
   else
     {
      DatabaseTransactionCommit(db);
      PrintFormat("Table FILES: added %d files", count);
     }
 
//--- close the database file and inform of that
   DatabaseClose(db);
   PrintFormat("Database: %s created and closed", filename);
  }

[[Deleted]]

Andrew Saxby:

Hi All

Anyone also having the following error: ???

DatabaseBind issue? Method/function is missing ...... code is straight from the help file and examples 

Any help would be really appreciated 

Check your #include file.

[Sergey Golubev]

There is good discussion in the similar thread in Russian forum (this is the post from MQ) -

Forum on trading, automated trading systems and testing trading strategies

New version of the MetaTrader 5 build 2360 platform: Extending SQLite integration

Renat Fatkhullin , 05/20/07 5:36 p.m.

In the next beta: you can directly download projects from github

We seriously improved the search capabilities of the editor and focused on providing information locally.

The complete overhaul of the intellisense is the next step.

[Keith Watford]

Comments that do not relate to this topic, have been moved to "Off Topic Posts".