From theory to practice - page 128

 
Nikolay Demko:

Sorry mate, it's customary to have your own spoon.


He dropped off an archive of exponent-processed tick data a couple of sheets ago, and now it turns out they don't have time stamps.

 
СанСаныч Фоменко:

He dropped off an archive of exponent-processed tick data a couple of sheets ago, and now it turns out they don't have time stamps.


Yeah, embarrassing, what can I say... I'll start collecting with time stamps next week...

 
Nikolay Demko:

Throw an example of the archive, a script to write how to transfer two bytes.

https://yadi.sk/d/snmT60R43RNUeL file AUDCAD_3DC.rar 247 Mb

Here are the ticks for 3+ years (from 2014 till 2017 October 28.) for AUDCAD, a tool, already processed many times by Alexander, for 3 DC, one of which was quoted 4 digits and its ticks have ended on 26.02.2016. The ticks were taken from http://advancetools.net/index.php/instrumenty/tikovye-ob-emy/istoriya-tikov, unzipped and merged into 3 solid files. Didn't do any checks. The size of 2 of the 3 .csv files is over 2GB.

The resource does not explicitly state, but according to my information, Igor Gerasko should be thanked for these tics.

AUDCAD_3DC.rar
AUDCAD_3DC.rar
  • yadi.sk
View and download from Yandex.Disk
 
Dmitriy Skub:

Nikolai, here is the ruble/dollar archive from the exchange.

Format:

Date Time in msec Bid Ask Last Volume


There are a lot of doubles in the archive by time, I don't understand what the gimmick is. Like this

2017.09.21 11:59:11.601,59843,59862,60100,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60120,150,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60099,10,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60025,2,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60085,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60089,7,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60230,2,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60599,30,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60600,3,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60600,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60394,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60300,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60361,2,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60362,44,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59874,10,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59873,10,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59876,10,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59875,10,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59862,3,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59862,3,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59872,10,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59862,3,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60000,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59950,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60025,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,60025,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59878,10,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59877,10,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59950,1,BUY,1505995151601 0
2017.09.21 11:59:11.601,59843,59862,59880,100,BUY,1505995151601 0




Also, the archive is not sorted by time, the timings are random. Sometimes they are a minute ahead, sometimes even more.

Hence the question: Process data? so that they measured only timing, not paying attention to what goes from each transaction.

I think we have to check for complete duplicates, not only by time but also by price, volume, direction.

 
Nikolay Demko:

There's a lot of time duplication in the archive, I don't know what the big deal is. It's like this.

Also, the archive is not sorted by time, the timings are all over the place. Sometimes a minute ahead, sometimes even more.

Hence the question: should the data be processed? so that they measure only timing, without paying attention to the fact that the recording comes from each deal.

This is not a duplicate; it's just a certain market volume which is spread among several Limit orders. The prices are different there. It should be sorted by time - can you specify the time where the sorting is broken?

This is a question for Alexander, by the way. There are several increments at one and the same time (if you follow your logic) - and how should we calculate?

 
Dmitriy Skub:

These are not duplicates - just some market volume smeared among several limit orders. The prices are different there. It should be sorted by time - can you specify the time where the sorting is broken?

This is a question for Alexander, by the way. You obtain several increments at one and the same time (if you follow your logic) - and how should we calculate them?


Pardon me for interfering. Imho, fair to count as on the exchange, in the system "netting": [average] price of position = cost of all transactions / volume of all transactions.

Where transaction value = transaction volume * exchange rate. For example, if you bought 1.2 lots of EURUSD at 1.2025, then the transaction value = 120,000 * 1.2025 = $144,300.

 
Dmitriy Skub:

These are not duplicates - just some market volume smeared among several limit orders. The prices are different there. It should be sorted by time - can you specify the time where the sorting is broken?

This is a question for Alexander, by the way. There are several increments at one and the same time (if you follow your logic) - and how should we calculate?

That's why I switched to my time scale, to avoid such situations. In this case - I don't know. Feynman considered deltaT -->0. But to =0, such situation, alas, in the theory does not exist.
 
Nikolay Demko:

There's a lot of time duplication in the archive, I don't know what the big deal is. It's like this.

Also, the archive is not sorted by time, the timings are all over the place. Sometimes a minute ahead, sometimes even more.

Hence the question: Process data? so that they measured only timing, not paying attention to what goes from each transaction.

I think we still have to check for complete doubles, not only by time but also by price, volume, direction.

Come on, Nikolai. I see it's not quick - next week I'll collect the ticks and check it myself. But if you're seriously interested, it will be interesting to see your results.
 
Alexander_K2:
Oh, come on, Nikolai. I can see it's not a quick matter - I'll get the tics together next week and check it out myself. But, if you are seriously interested - it will be interesting to see your results.
And Alexander, does stock data really suit you? I thought you were only talking about forex...
 
Vladimir:
Do stock market data really suit you, Alexander?
I don't know... I have an NDD account (I used to misspell ECN). I am confused by these names. I asked my manager about it - it is NDD. They say that they do this directly from exchanges. I get quotes Ask and Bid. There are no quotations for Last. But, one can trade some stuff like rice, sugar, coffee. I have not sorted it out yet.
Reason: