How your EA handles not executed limited orders ?

 

Hello,

So, when the price gets to your limited order price, but the limited order is not executed. Every day I see a few of them.

I am assuming my order is far back in the book queue and never gets a chance to be executed.

But, how do you handle this condition in your EA ?

I see some options:

1)
Stop using limited order and start using market order.
I see as downsides:
Possibly getting a worse price.
Make the EA slower because needs processing on ontick()
Any other ?

2)
Check in ontick() if the price got to the limited order price, but it was not executed, then cancel limited order and execute market order.
I see as downsides:
I believe it can create a race condition where the order was executed when we send the cancel.
Also Make the EA slower because needs processing on ontick()

3)
I am not sure of any other possible solution, like a combination of orders?

The problem I am seeing is described in this article, but I am not sure how to solve it, appreciate any insight:

***

 
There is no need to create pending orders in code.
  1. The pending has the slight advantage, A) you are closer to the top of the queue (filled quicker), B) there's no round trip network delay (filled quicker.)
  2. Don't worry about it unless you're scalping M1 or trading news.
  3. Humans can't watch the screen 24/7, so they use pending orders; EAs can, so no need for pending orders, have it wait until the market reaches the trigger price and just open an order.
 
ramesoj:

Hello,

So, when the price gets to your limited order price, but the limited order is not executed. Every day I see a few of them.

I am assuming my order is far back in the book queue and never gets a chance to be executed.

But, how do you handle this condition in your EA ?

I see some options:

1)
Stop using limited order and start using market order.
I see as downsides:
Possibly getting a worse price.
Make the EA slower because needs processing on ontick()
Any other ?

2)
Check in ontick() if the price got to the limited order price, but it was not executed, then cancel limited order and execute market order.
I see as downsides:
I believe it can create a race condition where the order was executed when we send the cancel.
Also Make the EA slower because needs processing on ontick()

3)
I am not sure of any other possible solution, like a combination of orders?

The problem I am seeing is described in this article, but I am not sure how to solve it, appreciate any insight:

***

If you use Limit orders on current price all you can do is optimize your code, improve ping, and diversify. With diversify i mean instead of running 1 algo, run 10 with slight different parameters with 1/10th of the volume.

Not sure if you are trading at the exchange, if not try if another broker offers better liquidity.

Edit: One other thing. With limit orders on the current price. MT5 waits for the next tick. This is a serious mt5 limitation. The broker can work around this, not all brokers do this or are even aware of this.

Click the image for the animated version|:

https://www.mql5.com/ru/forum/341117/page2#comment_16446599

 
William Roeder #:
There is no need to create pending orders in code.
  1. The pending has the slight advantage, A) you are closer to the top of the queue (filled quicker), B) there's no round trip network delay (filled quicker.)
  2. Don't worry about it unless you're scalping M1 or trading news.
  3. Humans can't watch the screen 24/7, so they use pending orders; EAs can, so no need for pending orders, have it wait until the market reaches the trigger price and just open an order.

Hi William,

So your suggestion is to not use pending orders in EAs at all?

Is that what you do on your EAs?

What I see is that when my pending order gets to an extreme of one bar (for instance, when I set a Buy pending order that will be the lowest price in the bar), it gets unfilled about 30% of the time.

The reason I thought about keeping the pending order and only converting it to a market order if the price got to the pending order price but the order was not filled, was to avoid the slipage when the pending order worked.

Thanks for looking into this

 
Enrique Dangeroux #:

If you use Limit orders on current price all you can do is optimize your code, improve ping, and diversify. With diversify i mean instead of running 1 algo, run 10 with slight different parameters with 1/10th of the volume.

Not sure if you are trading at the exchange, if not try if another broker offers better liquidity.

Edit: One other thing. With limit orders on the current price. MT5 waits for the next tick. This is a serious mt5 limitation. The broker can work around this, not all brokers do this or are even aware of this.

Click the image for the animated version|:

https://www.mql5.com/ru/forum/341117/page2#comment_16446599

Hello Enrique,

Thanks for your insights.

So, you also don't use limited orders in your EAs?

And sorry, but I did not understand the MT5 limitation where it waits for the next tick, I saw the image, but not understood

 
ramesoj #:

Hello Enrique,

Thanks for your insights.

So, you also don't use limited orders in your EAs?

And sorry, but I did not understand the MT5 limitation where it waits for the next tick, I saw the image, but not understood

I only use limit orders. To get in and to get out. Read the thread i linked to from the start it is explained there. You can test yourself with your broker on a slow illiquid market at night to see if the limit order at the current price is instant or waits for the next tick (which can be minutes).

If it waits then do not bother with limits as you will be always late to the party.

Note this applies ONLY to limits at current price. 

 

Hi Enrique,

I have translated the page you linked and I understood the issue, thanks for pointing it out: A limit order placed at current price is only executed (if is executed) on the next tick.

It is possible this is my issue, although I don't think it is probable. I am going to add logs and check if this condition happens.

But, practically, either is this problem or the limit order is not executed because my order is far back in the book, how would you solve this?

I saw in your thread someone putting the same 2 solutions I put above, they are not ideal. Anything else we could do?

From your thread ( I have added number 3):

But it has a significant disadvantage. Let's imagine that you need to modify an already existing limit limit by placing it at the current price . Then there are two options:

  1. Delete the limit limit, and then put the market, which will become the limit limit by the aggregator. But when you remove the limit card, the price may go away and you may not be in time.
  2. Put a market, and then remove the limit. But then, when placing the market, the limit limit may also be fulfilled. Those. we get a double position volume.
  3. Use only market order.
Документация по MQL5: Константы, перечисления и структуры / Торговые константы / Свойства ордеров
Документация по MQL5: Константы, перечисления и структуры / Торговые константы / Свойства ордеров
  • www-mql5-com.translate.goog
Свойства ордеров - Торговые константы - Константы, перечисления и структуры - Справочник MQL5 - Справочник по языку алгоритмического/автоматического трейдинга для MetaTrader 5
 
ramesoj #:

Hi Enrique,

I have translated the page you linked and I understood the issue, thanks for pointing it out: A limit order placed at current price is only executed (if is executed) on the next tick.

It is possible this is my issue, although I don't think it is probable. I am going to add logs and check if this condition happens.

But, practically, either is this problem or the limit order is not executed because my order is far back in the book, how would you solve this?

I saw in your thread someone putting the same 2 solutions I put above, they are not ideal. Anything else we could do?

From your thread ( I have added number 3):

But it has a significant disadvantage. Let's imagine that you need to modify an already existing limit limit by placing it at the current price . Then there are two options:

  1. Delete the limit limit, and then put the market, which will become the limit limit by the aggregator. But when you remove the limit card, the price may go away and you may not be in time.
  2. Put a market, and then remove the limit. But then, when placing the market, the limit limit may also be fulfilled. Those. we get a double position volume.
  3. Use only market order.

The highlighted part is related to being executed on the next tick. You will always be late (one tick behind) == later in the book == no or partial fill. Like i said working one tick behind the market renders trading the current price useless.

The disadvantages mentioned, is with the idea in mind that one is trading on 1 tick after the fact. There is no solution other than change trading platform or work with a broker who has provided a workaround for this issue. 

You will need to check this first.

For the rest it is speed and diversification. Optimize your code. For trading current tick processing your trading decision in 100ms means you are 100ms later in the book. Reduce latency. Get a server in the same dc as broker.


\

Diversify. The picture shows what looks like grid trading, but it is 10 individual algorithms. They do not hit the exact same price. this way you will reduce the psosibility to struck out.


Regarding market orders. I am against market orders. Market orders makes you susceptible to anybody profit freely off of you, the counterpart limits (lq prviders), me or even your broker (who slips in some). So, huge no no. 


To illustrate my piont.  This is USDCHF, 2k+ trades. The profit in points is little over 21k profit, the profit without positive slippage is little over 18k points. So roughly 12-13% of profit made extra trading against market orders. 

Depending in symbol the profit of positive slippage is significant in my trade history. The lowest range about 3-5% of free ticks to the highest ranging in the 20% free ticks. All these are taken from counter parties Market orders. The high range pays the commission for the lots traded.

So less loss abd more profit vs less profit and more loss. 

 

Enrique,

Thanks for laying out your opinions.

In the book “High-Frequency Trading” by “Irene Aldridge”, she tells that we can’t trust a limit order will execute unless the price crosses the limited order price:

" A limit order can be considered to be executed only when the market price “crosses” the order price, as shown in Figure 10.1. When the market price equals the limit order price, the limit order may or may not to be executed in live trading. As a result, simulations of limit order trading generally consider a limit order to be executed when the market price drops below the price of the buy limit order, or when the market price rises above the price of the limit sell order, and the limit order execution is guaranteed.


I think this is what is happening with me:
On 2022-04-11, my strategy running on real account executed 177 trades, all limit orders, in 59 of 177 the price did not cross the limit order price, and 35 of those 59 failed, which is a 60% fail rate. The instrument is very liquid, it goes an average of 8K contracts per minute.

I agree with your assessment of market orders, and I understand your diversity principle, but I wonder that a lot of people should have this issue.
What else could be done?

 
ramesoj #:

Enrique,

Thanks for laying out your opinions.

In the book “High-Frequency Trading” by “Irene Aldridge”, she tells that we can’t trust a limit order will execute unless the price crosses the limited order price:

" A limit order can be considered to be executed only when the market price “crosses” the order price, as shown in Figure 10.1. When the market price equals the limit order price, the limit order may or may not to be executed in live trading. As a result, simulations of limit order trading generally consider a limit order to be executed when the market price drops below the price of the buy limit order, or when the market price rises above the price of the limit sell order, and the limit order execution is guaranteed.


I think this is what is happening with me:
On 2022-04-11, my strategy running on real account executed 177 trades, all limit orders, in 59 of 177 the price did not cross the limit order price, and 35 of those 59 failed, which is a 60% fail rate. The instrument is very liquid, it goes an average of 8K contracts per minute.

I agree with your assessment of market orders, and I understand your diversity principle, but I wonder that a lot of people should have this issue.
What else could be done?

Read from the source, not some book without context. https://library.tradingtechnologies.com/user-setup/b3-verview.html Not sure if you trade there, every exchange publishes their matching algorithm unlike forex.

If you have read it, you will realize it all boils down to be faster then the next guy.

 
Enrique Dangeroux #:

Read from the source, not some book without context. https://library.tradingtechnologies.com/user-setup/b3-verview.html Not sure if you trade there, every exchange publishes their matching algorithm unlike forex.

If you have read it, you will realize it all boils down to be faster then the next guy.

Yes, I trade at B3. And your link shows they use timed based queues for pending orders. Thanks for finding that info for me.

Besides trying to improve the speed,as you mentioned, I am going to create a piece of code that will delete a pending order and issue a market order in case the pending order price was reached, but the pending order was not executed.
I know the price will be worse with the market order, but for my strategy it is much worse to skip the order than to get a worse price. And besides, this situation only happens on ~15% of my orders.

The only parameter I am debating is for recognizing when the pending order was not executed, I mean:

1-I have a pending order at price X
2-Tick shows price X was reach, either by LAST or BID/ASK accordingly
3-Wait up to (50ms, 100ms, 500ms) to see if the pending order was executed.
4-Order executed: everybody is happy
5-Order not executed: delete pending order and execute a market order.

What do you think?

Thanks!

Reason: