Discussion of article "SQLite: Native handling of SQL databases in MQL5" - page 7

New comment
 
Renat Fatkhullin:

It's going to be 2840 in beta tomorrow:

  • SQLite 2.35.2 version

  • permanent WAL mode, allowing to work with an open database from different applications (previously MetaEditor could not work in parallel with the terminal).

Very happy, thanks
 
Edgar Akhmadeev:
Very happy, thank you

Beta 2840 is available, please try it.

 
Renat Fatkhullin:

It's going to be 2840 in beta tomorrow:

  • SQLite 2.35.2 version

  • permanent WAL mode, allowing to work with an open database from different applications (previously MetaEditor could not work in parallel with the terminal)

  • extension of statistical functions
    example:
  • new maths functions
  • JSON support is also included

    We will include new json type in the database creation wizard later.



Renat, thank you very much!!!
I didn't expect it so fast, very pleased))))))
 
fxsaber:

Who has figured it out, please show the implementation of such a task.

  1. There are two Terminals.
  2. It is necessary to transfer real-time quotes of one symbol from Terminal1 to the corresponding custom symbol of Terminal2.
fxsaber:

Within the framework of this task, do I understand correctly that in both cases (when reading the base on Terminal2 and writing to the base in Terminal1) it should be blocked through the transaction mechanism?

What is the cheapest way to determine that the database has been updated?

Fundamentally wrong. You are describing a distributed client-server application, with a scheme of 1 writer, n readers. When designing such systems (and any distributed systems in general) you should try to avoid any locks by using lock-free ways of data organisation and access to them. If the technology used does not allow you to do without locking, then perhaps it is not the best solution for your task at all. However, for other tasks the technology may be great.
In your case it is better to deploy a full-fledged server (you can do it on the same machine as the client) and write quotes to the message queue, look at Kafka, for example. The client will read these quotes from the required index. This is a lock-free data access scheme.

fxsaber:

So it turns out that data exchange has fewer possibilities than via files?

Categorically no. Sharing via files is not atomic at all, so it requires locks on both the reader and writer side. This is the surest way to get a deadlock and get lost in catching hard-to-find and incomprehensible errors.

 
Vasiliy Sokolov:

Fundamentally wrong. You are describing a distributed client-server application with a scheme of 1 writer, n readers. When designing such systems (and any distributed systems in general) you should try to avoid any locks, using lock-free ways of data organisation and access to them. If the technology used does not allow you to do without locking, then perhaps it is not the best solution for your task at all. However, for other tasks the technology may be great.
In your case it is better to deploy a full-fledged server (you can do it on the same machine as the client) and write quotes to the message queue, look at Kafka, for example. The client will read these quotes from the required index. This is a lock-free data access scheme.

Categorically no. Sharing via files is not atomic in any way, so it requires locks on both the reader and writer side. This is the surest way to get a deadlock and get lost in catching hard-to-see and incomprehensible errors.

Thanks for such a detailed response! Unfortunately, I completely forgot what problem I was solving at the time. That's why I can't share my thoughts on the topic.

 
Renat Fatkhullin:

Beta 2840 is available, please try it.

Renat, good morning!

I also noticed a problem in StringFormat, when a rather large input data string is placed in it, for example from a resource file with 3-4 statements, for example %d, %s, %lld, %s, I have checked several substitution variants, it's just a matter of a large amount of input data, which results in error 4003.

I temporarily switched to StringReplace function in projects, it works correctly with large data in the input string.
 
Learned, thanks for sharing.
 
In the DataBasePrepare file there is some mistake. I would replace the "pair" by string is that right?
 

Questions on this solution

- Are there any issues with multiple EAs use the same sqlite database concurrently?

- If MT5 crashes, can some data be lost? How often does it write data to disk?

 

Good afternoon, dear developers!

The function "DatabaseExport" does not want to work in any way...it gives error 5601 (query execution error, but I do not execute the query) when I specify the table name in the parameters,

and when I specify SQL query, it gives error 4022 (cancellation of program execution), probably error inside MQL function, part of the code from my library:


//+------------------------------------------------------------------+
void CSQLite::DataBaseToFile(void)
  {
   uint flags=DATABASE_EXPORT_COMMON_FOLDER | DATABASE_EXPORT_QUOTED_STRINGS;

   long count_rows=0;

   string tables[],
          file_name,
          separator=";",
          query="SELECT name FROM sqlite_master WHERE tbl_name <> 'sqlite_sequence' AND type='table'";

   int total=GetValuesFromDataBase(query,tables); // ТУТ ВСЕ ОК, СПИСОК ТАБЛИЦ ИЗ БАЗЫ ПОЛУЧЕН.

   if(m_handle==NULL)
      Open();

   for(int i=0; i<total; i++)
     {
      file_name=StringFormat("%s.%s",m_name,tables[i]);
      count_rows=DatabaseExport(m_handle,tables[i],file_name,flags,separator); // ОШИБКА ПОСЛЕ ВЫПОЛНЕНИЯ ДАННОЙ ФУНКЦИИ
      Print(StringFormat("Export file: %s, rows: %lld",file_name,count_rows));

      if(count_rows<0)
         Print("DB: ", m_name,", Table: ",tables[i], ", Import failed with code ", GetLastError());
      else
         if(count_rows>0)
            Print(StringFormat("Import file: %s, rows: %lld",file_name,count_rows));
     }

   Finalize();
  }
//+------------------------------------------------------------------+
123456789101112
New comment