OnTradeTransaction weirdnesses

 

I implemented a EA that uses OnTradeTransaction on stock markets. My intention was to remove the SLTP check from the OnTick function and improve the speed of my optimizations.


Firstly I noticed that the events are extremely different among simulation, demo and real environments.


The amount of events changes and the order of events also changes!


Here is an example of orders that I got in the test:


Type Operation step ORDER Position position deal order
IN 1 1 TRA TRADE_TRANSACTION_DEAL_ADD - - 2 ORDER_TYPE_BUY ORDER_STATE_STARTED 2 0 84235 0 0 0 WIN$N 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0   0 0 0 0 0 0 0 0 0   0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0   0 0 0 0 0 0 0
IN 1 2 TRA TRADE_TRANSACTION_ORDER_DELETE - - 2 ORDER_TYPE_SELL ORDER_STATE_FILLED 2 0 84235 0 0 0 WIN$N 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0   0 0 0 0 0 0 0 0 0   0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0   0 0 0 0 0 0 0
IN 1 3 TRA TRADE_TRANSACTION_HISTORY_ADD - - 2 ORDER_TYPE_SELL ORDER_STATE_FILLED 2 0 84235 0 0 0 WIN$N 0 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0   0 0 0 0 0 0 0 0 0   0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0   0 0 0 0 0 0 0
IN 1 4 TRA TRADE_TRANSACTION_REQUEST - - 0 ORDER_TYPE_BUY ORDER_STATE_STARTED 0 0 0 0 0 0   0 ORDER_TIME_GTC 0 REQ TRADE_ACTION_DEAL MI#3#IN#0 10 0 30 0 0 0 84235 0 0 WIN$N 0 ORDER_TYPE_SELL ORDER_FILLING_FOK ORDER_TIME_GTC 1 RES 84240 84235   2 2 84235 0 10009 0 1
IN 1 * TRA TRADE_TRANSACTION_REQUEST - - 2 ORDER_TYPE_SELL ORDER_STATE_FILLED 2 0 84235 0 0 0 WIN$N 1 ORDER_TIME_GTC 0 REQ TRADE_ACTION_DEAL MI#3#IN#0 10 0 30 0 0 0 84235 0 0 WIN$N 0 ORDER_TYPE_SELL ORDER_FILLING_FOK ORDER_TIME_GTC 1 RES 84240 84235                
OUT 1 1 TRA TRADE_TRANSACTION_DEAL_ADD - - 3 ORDER_TYPE_BUY ORDER_STATE_STARTED 2 0 83835 0 0 0 WIN$N 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0   0 0 0 0 0 0 0 0 0   0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0   0 0 0 0 0 0 0
OUT 1 2 TRA TRADE_TRANSACTION_ORDER_DELETE - - 3 ORDER_TYPE_BUY ORDER_STATE_FILLED 2 0 83835 0 0 0 WIN$N 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0   0 0 0 0 0 0 0 0 0   0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0   0 0 0 0 0 0 0
OUT 1 3 TRA TRADE_TRANSACTION_HISTORY_ADD - - 3 ORDER_TYPE_BUY ORDER_STATE_FILLED 2 0 83835 0 0 0 WIN$N 0 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0   0 0 0 0 0 0 0 0 0   0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0   0 0 0 0 0 0 0
OUT 1 4 TRA TRADE_TRANSACTION_REQUEST - - 0 ORDER_TYPE_BUY ORDER_STATE_STARTED 0 0 0 0 0 0   0 ORDER_TIME_GTC 0 REQ TRADE_ACTION_DEAL   10 0 30 0 2 0 83835 0 0 WIN$N 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 1 RES 83835 83830   4 3 83835 1 10009 0 1
OUT 1 5 TRA TRADE_TRANSACTION_DEAL_ADD - - 0 ORDER_TYPE_BUY ORDER_STATE_STARTED 0 0 0 0 0 0   0 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0   0 0 0 0 0 0 0 0 0   0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0   0 0 0 0 0 0 0
OUT 1 * TRA TRADE_TRANSACTION_REQUEST - - 3 ORDER_TYPE_BUY ORDER_STATE_FILLED 2 0 83835 0 0 0 WIN$N 0 ORDER_TIME_GTC 0 REQ TRADE_ACTION_DEAL   10 0 30 0 2 0 83835 0 0 WIN$N 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 1 RES 83835 83830                



Here were the same transactions on a real account:


IN 1 1 TRA TRADE_TRANSACTION_ORDER_ADD - - 43495724 ORDER_TYPE_BUY ORDER_STATE_STARTED 0 0 0 0 0 0 WINZ18 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
IN 1 2 TRA TRADE_TRANSACTION_REQUEST - - 0 ORDER_TYPE_BUY ORDER_STATE_STARTED 0 0 0 0 0 0 0 ORDER_TIME_GTC 0 REQ TRADE_ACTION_DEAL MI#15#IN#0 10 0 30 43495724 0 0 0 0 0 WINZ18 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 1 RES 0 0 0 43495724 0 1 10008 0 1
IN 1 3 TRA TRADE_TRANSACTION_ORDER_UPDATE - - 43495724 ORDER_TYPE_BUY ORDER_STATE_REQUEST_ADD 0 0 0 0 0 0 WINZ18 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
IN 1 4 TRA TRADE_TRANSACTION_ORDER_UPDATE - - 43495724 ORDER_TYPE_BUY ORDER_STATE_PLACED 0 0 0 0 0 0 WINZ18 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
IN 1 5 TRA TRADE_TRANSACTION_REQUEST - - 0 ORDER_TYPE_BUY ORDER_STATE_STARTED 0 0 0 0 0 0 0 ORDER_TIME_GTC 0 REQ TRADE_ACTION_DEAL MI#15#IN#0 10 0 30 0 0 0 0 0 0 WINZ18 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 1 RES 0 0 0 43495724 0 1 10009 0 1
IN 1 6 TRA TRADE_TRANSACTION_DEAL_ADD - - 43495724 ORDER_TYPE_BUY ORDER_STATE_STARTED 43495724 0 85085 0 0 0 WINZ18 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
IN 1 7 TRA TRADE_TRANSACTION_ORDER_DELETE - - 43495724 ORDER_TYPE_BUY ORDER_STATE_FILLED 43495724 0 0 0 0 0 WINZ18 0 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
IN 1 8 TRA TRADE_TRANSACTION_HISTORY_ADD - - 43495724 ORDER_TYPE_BUY ORDER_STATE_FILLED 43495724 0 0 0 0 0 WINZ18 0 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
OUT 1 1 TRA TRADE_TRANSACTION_ORDER_ADD - - 43495792 ORDER_TYPE_SELL ORDER_STATE_STARTED 43495724 0 0 0 0 0 WINZ18 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
OUT 1 2 TRA TRADE_TRANSACTION_REQUEST - - 0 ORDER_TYPE_BUY ORDER_STATE_STARTED 0 0 0 0 0 0 0 ORDER_TIME_GTC 0 REQ TRADE_ACTION_DEAL 0 0 0 43495792 43495724 0 0 0 0 WINZ18 0 ORDER_TYPE_SELL ORDER_FILLING_RETURN ORDER_TIME_GTC 1 RES 0 0 0 43495792 0 2 10008 0 1
OUT 1 3 TRA TRADE_TRANSACTION_ORDER_UPDATE - - 43495792 ORDER_TYPE_SELL ORDER_STATE_REQUEST_ADD 43495724 0 0 0 0 0 WINZ18 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
OUT 1 4 TRA TRADE_TRANSACTION_ORDER_UPDATE - - 43495792 ORDER_TYPE_SELL ORDER_STATE_PLACED 43495724 0 0 0 0 0 WINZ18 1 ORDER_TIME_DAY 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
OUT 1 5 TRA TRADE_TRANSACTION_REQUEST - - 0 ORDER_TYPE_BUY ORDER_STATE_STARTED 0 0 0 0 0 0 0 ORDER_TIME_GTC 0 REQ TRADE_ACTION_DEAL 0 0 0 0 43495724 0 0 0 0 WINZ18 0 ORDER_TYPE_SELL ORDER_FILLING_RETURN ORDER_TIME_GTC 1 RES 0 0 0 43495792 0 2 10009 0 1
OUT 1 6 TRA TRADE_TRANSACTION_DEAL_ADD - - 43495792 ORDER_TYPE_BUY ORDER_STATE_STARTED 43495724 0 85070 0 0 0 WINZ18 1 ORDER_TIME_GTC 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
OUT 1 7 TRA TRADE_TRANSACTION_ORDER_DELETE - - 43495792 ORDER_TYPE_SELL ORDER_STATE_FILLED 43495724 0 0 0 0 0 WINZ18 0 ORDER_TIME_DAY 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0
OUT 1 8 TRA TRADE_TRANSACTION_HISTORY_ADD - - 43495792 ORDER_TYPE_SELL ORDER_STATE_FILLED 43495724 0 0 0 0 0 WINZ18 0 ORDER_TIME_DAY 0 REQ ENUM_TRADE_REQUEST_ACTIONS::0 0 0 0 0 0 0 0 0 0 0 ORDER_TYPE_BUY ORDER_FILLING_FOK ORDER_TIME_GTC 0 RES 0 0 0 0 0 0 0 0 0



In addition to these discrepancies, I noticed delays of up to 4 seconds between a TRADE_TRANSACTION_REQUEST and a TRADE_TRANSACTION_HISTORY_ADD in a broker's real account.


Could you share your experiences with OnTradeTransaction in market actions?

 

Rafael Caetano Pinto:

Firstly I noticed that the events are extremely different among simulation, demo and real environments.

The amount of events changes and the order of events also changes!

In addition to these discrepancies, I noticed delays of up to 4 seconds between a TRADE_TRANSACTION_REQUEST and a TRADE_TRANSACTION_HISTORY_ADD in a broker's real account.

Could you share your experiences with OnTradeTransaction in market actions?

Perfectly normal. And documented.

The arrival of each such transaction to the terminal is the TradeTransaction event. Priority of these transactions' arrival at the terminal is not guaranteed. Thus, you should not expect that one group of transactions will arrive after another one when developing your trading algorithm.

or

Transactions queue length comprises 1024 elements. If OnTradeTransaction() handles yet another transaction for too long, the previous ones can be superseded by new transactions in the queue.

On a backtest you can use it reliably. On a live account you are better to always have a safety measure behind it.

 

So is OnTradeTransaction almost useless? I can use it to a redundant check, but never as the core of my event based system.


To do my EA works, I turn a boolean verification flag (transactionFlag) on when:

- OnTradeTransaction TRADE_TRANSACTION_HISTORY_ADD (if is not in test mode)

        - or when is TRADE_TRANSACTION_REQUEST:ORDER_STATE_FILLED (if is in test mode)


So, I call the Update(transactionFlag) each second (there is symbols that the tick frequence is low)

And I put Update(transactionFlag) as the first thing to do in OnTick()

And I call the Update(true) always before close/open a position.


The Update(bool flag) function updates the current transaction status and turns the transactionFlat off.


Is possible to use OnTradeTransaction only or in your opinion is better validate on each tick as I'm doing now?


I waste weeks on this problem and I don't have any good solution yet :/

 

OnTradeTransaction() is not useless at all.

It can be used to be notified about the result of an order request as soon as possible. From my experience it works very well, however as there is a possibility (even weak) you missed a transaction you should assure your code manage all critical operations even in this case.

I have no idea what you are trying to achieve exactly so I can't answer to your last question.

 
Alain Verleyen:

Perfectly normal. And documented.

or

On a backtest you can use it reliably. On a live account you are better to always have a safety measure behind it.


I think that now I understand... In other words: 

- On backtest I can use OnTradeTransaction to speedup my simulations

- On real the OnTradeTransaction is unreliable.

 
Rafael Caetano Pinto:


- On real the OnTradeTransaction is unreliable.

It's not what I said. See my previous post.

From my experience it's reliable. Things are not black or white.

 
Alain Verleyen:

OnTradeTransaction() is not useless at all.

It can be used to be notified about the result of an order request as soon as possible. From my experience it works very well, however as there is a possibility (even weak) you missed a transaction you should assure your code manage all critical operations even in this case.

I have no idea what you are trying to achieve exactly so I can't answer to your last question.

I have a risk control system based on profit and open positions. I also have a panel showing all EA positions with their respectives profits.


Update on each tick will allow me to use only 4-5 EAs concurrently. But if this OnTradeTransaction works, I can use a lot more on the same machine. So I want to do something to not drain the machine resouce (I think that call Update() every tick is so bad as an Update() loop)

 
Alain Verleyen:

It's not what I said. See my previous post.

From my experience it's reliable. Things are not black or white.

Lets say only that OnTradeTransaction is ugly, hard to understand, but sometime a good heart hero is able to understand it and show to world that this deformity is not so deformed at all, it is only.. misunderstood. :)

 
Rafael Caetano Pinto:

Lets say only that OnTradeTransaction is ugly, hard to understand, but sometime a good heart hero is able to understand it and show to world that this deformity is not so deformed at all, it is only.. misunderstood. :)

Any success Rafael? 

 
Rafael Judson Lima Valle:

Any success Rafael? 

Nops... I forget about OnTradeTransaction and OnTrade.


Try to use PositionSelect and PositionVolume inside onTick() and if you detect any changes, call the position history to get the changes.

Reason: