HELP: processing High Frequency data

Hello haqiqi.afshin,

please take a look at this post on the Portuguese section of the forum. It's directly related to this situation. The only problem: it's in Portuguese.

Maybe it's the situation you're facing and you might come to the same conclusions as those on the related topic.

I hope it helps.

I use this to record ticks (well the actual code records ticks from several symbols at the same time).

So you will loose only a few ticks every when the file writes once in an hour (or less)

struct OneTick
  {
   MqlTick           tick;
  };
//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
struct TickArray
  {
   OneTick           ticks[20000];
  };
//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
struct TickContainer
  {
   TickArray         allticks;
   int               ticksrecorded;
  };
//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
class CMultiTick
  {

public:
   int               file_handle;
   TickContainer     bticks;

   CMultiTick() 
     {
      file_handle = INVALID_HANDLE;
     }

   void CMultiTick:: ~CMultiTick()
     {
      SetFile(INVALID_HANDLE);
     }

   void  CMultiTick::AddTick(const MqlTick &theTick)
     {
      AddTick(bticks,theTick);
     }

   void  CMultiTick::AddTick(TickContainer &container,const MqlTick &theTick)
     {
         container.allticks.ticks[container.ticksrecorded].tick=theTick;
         if (container.ticksrecorded==20000) 
            WriteTicksToFile(); 
         else
            container.ticksrecorded++;
     }

   void  CMultiTick::WriteTicksToFile()
     {        
         if (file_handle!=INVALID_HANDLE)
         {
              int anz = bticks.ticksrecorded;
              if (anz>0)
              {
                 FileWriteStruct(file_handle,bticks.allticks,sizeof(OneTick)*anz);
                  bticks.ticksrecorded=0;
                  Print(anz," ticks written to file. ",sizeof(OneTick)*anz);
              }
         }
    }



   void  CMultiTick::SetFile(int fhandle)
     {
      if(file_handle!=INVALID_HANDLE)
         FileClose(file_handle);
      file_handle=fhandle;
     }
  };

I know but I checked that I got all ticks to make synthetic bars which had the same value than the ones from the broker.

Anyway I recorded 21 symbols at the same time and it worked quite well for me.

Hi Malacane 

thank you.I read whole posts in that topic , it helped a lot (thanks google translate ;)  ) . I hoped to find a solution to get all ticks perfectly. but apparently Its not easy. I use meta trader for stock market that is centralized . I compared volumes indicator "tick volume " with exchange "tick volume" data and they were equal. does this indicator count every ticks? . if so this indicator does not loose any tick !!!

 or maybe it get the data separately .if there is not a way to get whole tick by OnTick Event handler I,m going to find a way to modify data by whatever volumes indicator uses for its source. 

 thanks so much.  

I think that was not clear. let me say this way : when new order arrives or someone modify or cancel his order the quote changes when a market order arrives quote changes too.
 So I want to filter ticks resulted by market order . I tried to compare every ticks with tick before it . if ask and bid were equal then its a market order. it works but looses some data .(ticks that are saved by last code and are market order)

 datetime dtm = TimeTradeServer();
 string tick[7];
  int i=0;

  string tickfile_handle;
   string tickfilename = Symbol()+"_tick_"+TimeToString(dtm,TIME_DATE)+".txt";
MqlTick befortick;

int OnInit()
  {
   tickfile_handle=FileOpen(tickfilename,FILE_WRITE|FILE_TXT|FILE_READ);
   SymbolInfoTick(Symbol(),befortick);
   return(INIT_SUCCEEDED);
  }

void OnTick()
  {

   MqlTick last_tick;
   if(SymbolInfoTick(Symbol(),last_tick))
     {
        if(last_tick.ask==befortick.ask && last_tick.bid==befortick.bid)
        {
          datetime stm = TimeTradeServer();
  
          i=i+1;
  
          tick[0]=i;
          tick[1]=TimeToString(stm,TIME_SECONDS);
          tick[2]=TimeToString(last_tick.time,TIME_SECONDS);
          tick[3]=last_tick.last;
          tick[4]=last_tick.volume;
          tick[5]=last_tick.ask;
          tick[6]=last_tick.bid;
  


    
          FileSeek(tickfile_handle,0,SEEK_END);
          FileWriteArray(tickfile_handle,tick,0,WHOLE_ARRAY);

        }  
     }


      befortick=last_tick;

  }