Once again, about multithreading - page 5

[dd]

I am far from an expert, so I apologise in advance.

I am writing an EA.

The task:

1) To use several indicators. One and the same indicator may be used with different settings (periods) - how to make them work in parallel, to reduce the time of getting signals?

2) Parallel to the reception of signals from the indicators, perform a constant check and close profitable or losing positions. Besides, we have a lot of open positions, so this process should be parallelized in parts. For example, we can check 1000 orders in 3 or 4 threads.

3) Parallel to the above processes, use the signals received from the indicators in point 1 to buy or sell.

And the most interesting thing = how to parallelize all these tasks so that the tester is available and optimizations in the tester are possible?

Point the way, preferably with examples. Thanks in advance.

[Dmitry Fedoseev]

dd:

I am far from an expert, so I apologise in advance.

Task:

1) To use several indicators. One and the same indicator can be used with different settings (periods) - how to make them work in parallel, to reduce the time for receiving signals.

2) Parallel to the reception of signals from the indicators, perform a constant checking and closing of profitable or losing positions. Besides, we have a lot of open positions, so this process should be parallelized in parts. For example, we can check 1000 orders in 3 or 4 threads.

3) Parallel to the above processes, use the signals obtained from the indicators in point 1 to buy or sell.

And the most interesting thing = how to parallelize all these tasks so that the tester is available and optimizations in the tester are possible?

Point the way, preferably with examples. Thank you in advance.

Have you ever opened the terminal? In general, the tester and the optimizer are always available in parallel with the trading on the account.

I wonder how long it takes to receive signals from the indicators if there is a question of the necessity of paralleling them.

[dd]

Dmitry Fedoseev:

Have you even opened the terminal once? The tester and the optimizer are always available in parallel with the trading on the account.

I wonder how long it takes to receive signals from the indicators, if there is any question about the need to parallelize them?

I did, and obviously longer than yours. One tick is one second. During this time you need to check let's say 4 indicators - build charts. Get signals, buy or sell, close 1000 orders and so on ... And like every millisecond counts... More precisely, in minus accounts. Time is money :)

[Andrey Khatimlianskii]

dd:

1) Use several indicators. One and the same indicator can be used with different settings (periods) - how to make them work in parallel, to reduce the time of getting signals?

2) Parallel to the reception of signals from the indicators, perform a constant check and close profitable or losing positions. Besides, we have a lot of open positions, so this process should be parallelized in parts. For example, we can check 1000 orders in 3 or 4 threads.

3) Parallel to the above processes, use the signals received from the indicators in point 1 to buy or sell.

Indicators launched on one symbol work in one flow. But if they are written well, there will be no delay (it will be counted in milliseconds).

But all trading operations can be parallelized using additional Expert Advisors / scripts. Call them from the main EA if necessary, or keep them running all the time. An example of the implementation can be found in this article. Or use asynchronous OrderSend, but in this case it will be a bit harder to control.

dd:

And the most interesting thing = how to parallelize all these tasks so that the tester and optimization in the tester is available?

You don't need all this for tester, it works synchronously and doesn't skip ticks. Do 2 variants - for tester and for online (if ( IsTesting() ).

[dd]

Andrey Khatimlianskii:

Indicators running on the same tool all run in the same thread. But if you write them properly, there will be no delay (will be counted in milliseconds).

But all trading operations can be parallelized using additional Expert Advisors / scripts. Call them from the main EA if necessary, or keep them running all the time. An example of the implementation can be found in this article. Or, you can use asynchronous OrderSend, but in this case, the control will be a little more difficult.

For the tester this is all unnecessary, it works synchronously and doesn't skip ticks. Make 2 variants - for tester and for online (if ( IsTesting() ).

Indicators launched on one tool work all in one thread. But if you write them well, there will be no delay (will be calculated in milliseconds).

--- I do not understand. I am not going to write my own indicators. I am using OEM indicators. And the point is that I want to use them in parallel to reduce the response time...

You don't need all this for tester, it works synchronously and doesn't skip ticks. Make 2 variants - for tester and for online (if ( IsTesting() ).

---- I don't understand at all. Who doesn't need it? It works synchronously --- but for me it needs vice versa, to take into account my asynchrony... What the hell are two variants? I have one variant of my EA and I want to run it in the tester, but I want to parallelize my processes and show it in the tester.

Because of the fact that everything is running sequentially, I start to have drawdowns.

[Andrey Khatimlianskii]

dd:

--- I don't get it. I'm not going to write my own indicators. I'm using the standard ones. And the point is that I want to use them in parallel to reduce the response time...

Have you measured the response time? You'll lose more on paralleling.

dd:

---- I don't get it at all. Who doesn't need it? It works in sync --- but I need it to take my asynchronicity into account... What the hell are the two variants? I have one variant of my EA and I want to run it in the tester, but I want to parallelize my processes and show it in the tester.

One EA:

if ( IsTesting() )
{
   // работаем нормально
}
else
{
   // распараллеливаем торговые приказы
}

Why test the parallelization in the Strategy Tester?

[Andrey Khatimlianskii]

dd:

Because of the fact that everything is executed sequentially, I am starting to have drawdowns, and I'm having a hell of a lot of drawdowns.

Distribute the sending of trade orders or use OrderSendAsync

If you need to close a bunch of trades, you can open one big counter, lock in a profit and then calmly close OrderCloseBy one over the other.

[dd]

Andrey Khatimlianskii:

Distribute the sending of trade orders or use OrderSendAsync

so it's not just about the order send, there are a bunch of other checks being done. But they all get queued up until one function finishes, the other one is not executed ....

I've already described it in the problem statement. And it's a complex task. DLL or chats or whatever - what interests me is the scenario in which I may parallel these tasks and thus execute them in the tester to understand what's what...

[Dmitry Fedoseev]

dd:

so it's not just the warrant send, there are a bunch of other checks being done. But all of them are queued up until one function finishes, the other is not executed ....

I've described it in the terms of the problem. And it's a complex task. DLL or chats or whatever - what interests me is the scenario in which I may parallelize these tasks and keep running in the tester to see what's what...

And how much precious time you lose and what are the consequences? And what would be the profit if there was no "loss"?

[dd]

Dmitry Fedoseev:

And how much precious time are you losing and what are the consequences? And what would be the profit if there was no "loss"?

It takes me 5 days to run this strategy in the tester with 24 cores at my disposal. And only one core is working. Not to mention the fact that the log, a stupid nobody needs log during this time may bloat to terabytes or two. And all this without optimizations. And this stupid log cannot be disabled in any way ...

How do you think if I sped up the process say - 4 schedules, each on its core - an increase in this point 4 times. Plus a separate process for closing 1000 trades, say, or breaking this process down to 5 or 10 threads - a 10x increase. Plus a separate process for opening positions by signals ...

I think multiple and times ...

But every millisecond + lag to the broker + lag from the broker to the exchange (if it is not a kitchen) is a colossal loss. It is strange that this is not understood in Siberia...