The most effective way to store Tick Data

[Audai Louri]

Hello Everyone,

I'm trying to store tickdata and not sure what would be the most effective way to store this data. The Tick Data I want to store will be up to 1 minute of data and based on this I would like to do some calculations therefore I will require to access that data at any millisecond time interval.

But I'm not sure if I should store it in an

1) Array

2) File

3) Database

4) Better method i'm not aware of?

Any advice in the direction I should head would be highly appreciated.

Thank you in advance all.

Audai,

[Tobias Johannes Zimmer]

Audai Louri:

Hello Everyone,

I'm trying to store tickdata and not sure what would be the most effective way to store this data. The Tick Data I want to store will be up to 1 minute of data and based on this I would like to do some calculations therefore I will require to access that data at any millisecond time interval.

But I'm not sure if I should store it in an

1) Array

2) File

3) Database

4) Better method i'm not aware of?

Any advice in the direction I should head would be highly appreciated.

Thank you in advance all.

Audai,

Wouldn't harm if you tried the array with milsec fields and watch out for Ticks that get lost in high frequency hours.

[Raymond Ogechukwu Omeh]

Depends... on the size of tick data, why you're storing it and how frequently you need to access this data.

[Audai Louri]

Raymond Ogechukwu Omeh #:
Depends... on the size of tick data, why you're storing it and how frequently you need to access this data.

Thank you Raymond.

Honestly if I'm storing tick data in milliseconds I assume its over 60,000 chars and I know that the array can only hold 40,000 chars max.

So there is an option of creating multiple arrays, as receiving data will be faster from RAM rather than file or DB.

The short of it I'm storing data in milliseconds, or the closest I can get to storing in milliseconds, and requesting them per tick.

[Audai Louri]

pennyhunter #:
Wouldn't harm if you tried the array with milsec fields and watch out for Ticks that get lost in high frequency hours.

Hi Pennyhunter,

That was one of my concerns, if I set the timer to 1 milisec that tickdata can be lost.

But I believe array is the fastest option there.

My only concern is that arrays can only hold 40,000 chars.

[Tobias Johannes Zimmer]

Audai Louri #:

My only concern is that arrays can only hold 40,000 chars.

Would you care to back that up with a source?

[Audai Louri]

pennyhunter #:
Would you care to back that up with a source?

void OnTick()
  {
   double test[70000];
  }

Its really more of a warning

[Tobias Johannes Zimmer]

Audai Louri #:

Its really more of a warning

Oh... thanks for the concern, I might have to reevaluate the scalability of a current project...

It would be nice to know if there are possibilities to not have that limitations... Local variable as in opposed to global?

[Alexandre Borela]

Audai Louri #:

Hi Pennyhunter,

That was one of my concerns, if I set the timer to 1 milisec that tickdata can be lost.

But I believe array is the fastest option there.

My only concern is that arrays can only hold 40,000 chars.

Dynamic arrays can have 2147483647 elements.

https://www.mql5.com/en/docs/array/arrayresize

[Audai Louri]

Alexandre Borela #:

Dynamic arrays can have 2147483647 elements.

https://www.mql5.com/en/docs/array/arrayresize

Thanks Alexandar,

I decided to go with the file storage option. Was to worried about the warning I got.

[Tobias Johannes Zimmer]

Have you tried global variables just to be sure?

Audai Louri #:

Hi Pennyhunter,

That was one of my concerns, if I set the timer to 1 milisec that tickdata can be lost.

Also did you look into how in MQL5/Indicators/Examples the calculations are limited to the last bar?