Discussion of article "DIY multi-threaded asynchronous MQL5 WebRequest" - page 3

New comment
 
Stanislav Korotky:

WebRequest does not work in indicators (and this library allows you to bypass this limitation). A separate thread in MT is allocated only to experts and scripts, but scripts do not work with events.

I see. Thanks. I didn't know.

 
Stanislav Korotky:

In MT4 1146 the resources are fully functional. The idea can be safely ported to the fourth version.

 
fxsaber:

Yes, that's the idea. But it will work only in MT5.

I have implemented this idea (with chart objects).

use iCustom-indicator as a manager, and it will launch WebRequest in its script for each WebRequestAsync.

The design should be simpler and more reliable, because there will be no additional windows.

The result is as follows.

The scripts don't work because I can't pass a parameter to them at startup - it seems that scripts are not friendly with chart templates at all. The fact that they manage to run them at all with the help of Expert.mqh library (though without parameters), by analogy with Expert Advisors - it seems to be a happy accident.

If you execute web-requests not in scripts but in Expert Advisors, the scheme works halfway. That is, the input parameters work normally and we manage to pass the name of the resource with data to the Expert Advisor. But for some reason, only the OnInit event is triggered in the Expert Advisors launched in chart objects, while, for example, OnTick or OnTimer are not. It is not very nice to execute a web-request in OnInit, but it is possible. However, all this logic slows down terribly, because opening a chart object for each request causes a lot of overhead.

 
Stanislav Korotky:

Implemented this idea (with chart objects).

The result is as follows.

The scripts don't work because I can't pass a parameter to them when launching them - it seems that scripts are not friendly with chart templates at all. The fact that they can be launched using the Expert.mqh library at all (though without parameters), by analogy with Expert Advisors - it seems to be a happy accident.

I had to study the topic. The scripts are run by the template and on a regular chart, but without the ability to set input parameters. So the PlaySound method will definitely work.

If we execute web queries not in scripts but in Expert Advisors, the scheme works halfway. That is, the input parameters work normally and we manage to pass the name of the resource with data to the Expert Advisor. But for some reason, only the OnInit event is triggered in the Expert Advisors launched in chart objects, while, for example, OnTick or OnTimer are not. It is not very nice to execute a web-request in OnInit, but it is possible. However, all this logic slows down terribly, because opening a chart object for each request gives you a lot of overhead.

Nothing works for me.

 
fxsaber:

I had to study the topic. The scripts are run by the template and on a regular chart, but without the ability to set input parameters. So PlaySound method will definitely work.

Nothing is working for me.

The set I sent in a private message works for me through OnInit in the Expert Advisor. It can be seen from the logs. Send me your logs.

 

I am posting an alternative variant of launching web requests - with the help of chart objects, in which a template with an expert helper is applied. It can be used both from experts and indicators. Dependencies are the same as in the article. Directly new files of the approach implementation:

  • multiwebobjectworker.mqh - header file with common classes;
  • multiwebobjectworker.mq5 - working Expert Advisor that performs web queries inside objects-graphs;
  • multiwebindicator.mq5 - bufferless indicator demonstrating the launch of web-queries;

For each query an object is created, an expert-helper is created in it, the object is deleted when the query is finished. Query data and results are exchanged via resources, as in the article.

Judging by the logs, this mechanism does not provide full-fledged parallelism of query execution, as the objects probably share a common event queue. Log example:

2018.11.14 15:11:58.492 multiwebindicator (EURUSD,H1)   129912254742671348: Starting chart object WRS1 129912254742671349
2018.11.14 15:11:58.694 multiwebindicator (EURUSD,H1)   129912254742671348: Starting chart object WRS2 129912254742671350
2018.11.14 15:11:58.819 multiwebindicator (EURUSD,H1)   129912254742671348: Starting chart object WRS3 129912254742671351
2018.11.14 15:11:58.960 multiwebobjectworker (EURUSD,H1)        129912254742671349: OnInit
2018.11.14 15:11:58.960 multiwebobjectworker (EURUSD,H1)        129912254742671349: Reading request \Indicators\multiwebindicator.ex5::WRS_0_129912254742671348
2018.11.14 15:11:58.960 multiwebobjectworker (EURUSD,H1)        129912254742671349: Got 64 bytes in request
2018.11.14 15:11:58.960 multiwebobjectworker (EURUSD,H1)        129912254742671349: GET https://google.com/ User-Agent: n/a 5000 
2018.11.14 15:11:58.975 multiwebobjectworker (EURUSD,H1)        129912254742671350: OnInit
2018.11.14 15:11:58.975 multiwebobjectworker (EURUSD,H1)        129912254742671350: Reading request \Indicators\multiwebindicator.ex5::WRS_1_129912254742671348
2018.11.14 15:11:58.975 multiwebobjectworker (EURUSD,H1)        129912254742671350: Got 60 bytes in request
2018.11.14 15:11:58.975 multiwebobjectworker (EURUSD,H1)        129912254742671350: GET https://ya.ru User-Agent: n/a 5000 
2018.11.14 15:11:59.084 multiwebobjectworker (EURUSD,H1)        129912254742671351: OnInit
2018.11.14 15:11:59.084 multiwebobjectworker (EURUSD,H1)        129912254742671351: Reading request \Indicators\multiwebindicator.ex5::WRS_2_129912254742671348
2018.11.14 15:11:59.084 multiwebobjectworker (EURUSD,H1)        129912254742671351: Got 72 bytes in request
2018.11.14 15:11:59.084 multiwebobjectworker (EURUSD,H1)        129912254742671351: GET https://www.startpage.com/ User-Agent: n/a 5000 
2018.11.14 15:11:59.162 multiwebobjectworker (EURUSD,H1)        129912254742671350: Done in 187 ms
2018.11.14 15:11:59.178 multiwebindicator (EURUSD,H1)   129912254742671348: Result code 200
2018.11.14 15:11:59.178 multiwebindicator (EURUSD,H1)   129912254742671348: Reading result \Experts\multiwebobjectworker.ex5::WRS_0_129912254742671350
2018.11.14 15:11:59.178 multiwebindicator (EURUSD,H1)   129912254742671348: Got 16592 bytes in response
2018.11.14 15:11:59.178 multiwebindicator (EURUSD,H1)   GET https://ya.ru
2018.11.14 15:11:59.178 multiwebindicator (EURUSD,H1)   Received 3734 bytes in header, 12775 bytes in document
2018.11.14 15:11:59.256 multiwebobjectworker (EURUSD,H1)        129912254742671350: OnDeinit
2018.11.14 15:11:59.272 multiwebobjectworker (EURUSD,H1)        129912254742671349: Done in 297 ms
2018.11.14 15:11:59.334 multiwebindicator (EURUSD,H1)   129912254742671348: Result code 200
2018.11.14 15:11:59.334 multiwebindicator (EURUSD,H1)   129912254742671348: Reading result \Experts\multiwebobjectworker.ex5::WRS_0_129912254742671349
2018.11.14 15:11:59.334 multiwebindicator (EURUSD,H1)   129912254742671348: Got 12688 bytes in response
2018.11.14 15:11:59.334 multiwebindicator (EURUSD,H1)   GET https://google.com/.
2018.11.14 15:11:59.334 multiwebindicator (EURUSD,H1)   Received 790 bytes in header, 11813 bytes in document
2018.11.14 15:11:59.350 multiwebobjectworker (EURUSD,H1)        129912254742671349: OnDeinit
2018.11.14 15:11:59.833 multiwebobjectworker (EURUSD,H1)        129912254742671351: Done in 749 ms
2018.11.14 15:11:59.833 multiwebindicator (EURUSD,H1)   129912254742671348: Result code 200
2018.11.14 15:11:59.833 multiwebindicator (EURUSD,H1)   129912254742671348: Reading result \Experts\multiwebobjectworker.ex5::WRS_0_129912254742671351
2018.11.14 15:11:59.833 multiwebindicator (EURUSD,H1)   129912254742671348: Got 45212 bytes in response
2018.11.14 15:11:59.833 multiwebindicator (EURUSD,H1)   GET https://www.startpage.com/
2018.11.14 15:11:59.833 multiwebindicator (EURUSD,H1)   Received 822 bytes in header, 44307 bytes in document
2018.11.14 15:11:59.849 multiwebindicator (EURUSD,H1)   > > > Async WebRequest workers finished 3 tasks in 1357 ms
2018.11.14 15:11:59.880 multiwebobjectworker (EURUSD,H1)        129912254742671351: OnDeinit
Files:
multiwebobject.zip  15 kb
 
Stanislav Korotky:

Try the updated Resource_Data.mqh, lags when transferring web pages should disappear.

#include <fxsaber\TradeTransactions\ResourceData.mqh> // https://www.mql5.com/en/code/22166

#define  BENCH(A)                                                              \
{                                                                             \
  const ulong StartTime = GetMicrosecondCount();                              \
  A;                                                                          \
  Print("Time[" + #A + "] = " + (string)(GetMicrosecondCount() - StartTime)); \
}  

const RESOURCEDATA<uchar> Resource("::" + __FILE__); // Resource for data transfer (bytes)
uchar BytesIn[];
const int Init = ArrayResize(BytesIn, 1000000);

void TestResource()
{
  uchar BytesOut[];

  Resource = BytesIn;
  Print(Resource.Get(BytesOut));
}

void OnStart()
{
  BENCH(TestResource());
}


Old version

1000000
Time[TestResource()] = 103746


New version

1000000
Time[TestResource()] = 5222
 
fxsaber:

Try the updated Resource_Data.mqh, lags when transferring web pages should disappear.

Old version

New version

What did you have to do to speed up 20 times? Compatibility with old code remains, client calls don't need to be changed?

 
Stanislav Korotky:

What did you have to do to speed up 20 times? Compatibility with the old code remains, client calls don't need to be changed?

Compatibility remains.

Forum on trading, automated trading systems and testing trading strategies.

Libraries: TradeTransactions

fxsaber, 2018.12.09 01:23 pm.

In the library sources there is Convert.mqh, which quickly converts an array of one type to an array of another.

TypeToBytes was slow (now only used if neither array type is a multiple of the other) on large arrays.

TypeToBytes is not used for uchar[].

 
Nice article, thanks. How about WinINet Asynchronous mode?
1234567
New comment