Data Collection script Need some help

New comment
 

Hi I found this script on this website somewhere a while ago and decided to give it a try now. The problem is it overwrites older data once it gets to a 101 lines. I dont know how to code in MQL but looking at the code I see length = 100 and the for loop will only iterate that many times since i = length - 1. I would like to remove this limitation and have the file grow indefinitely. But I have a feeling its limited like that for a reason. Probably a memory bottleneck that will occur without it?

Again I have no knowledge of MQL but I would assume its something along the lines of creating the file handle in init(), closing it in deinit(), and writing to the file without a for loop in the start(). I tried this but being that I dont know what Im doing I failed.

Can somebody please be so kind as to just modify this code for me?

#property indicator_chart_window
extern int length = 100;   // The amount of bars sent to be processed
double ExtMap[];           // Chart buffer
string nameData;

int init()
{
   nameData = Symbol()+".csv";         // name of the data file to be sent
   return(0);
}

int deinit()
{

   return(0);
}
int start()
{
   static int old_bars = 0;   // remember the amount of bars already known   
   if (old_bars != Bars)      // if a new bar is received 
   {
        int handle = FileOpen(nameData, FILE_CSV|FILE_WRITE,';');
        if(handle < 1)
        {
          Comment("Creation of "+nameData+" failed. Error #", GetLastError());
          return(0);
        }
        FileWrite(handle, ServerAddress(), Symbol(), Period());                  // heading
        FileWrite(handle, "DATE","TIME","HIGH","LOW","CLOSE","OPEN","VOLUME");   // heading
        int i;
        for (i=length-1; i>=0; i--)
        {
          FileWrite(handle, TimeToStr(Time[i], TIME_DATE), TimeToStr(Time[i], TIME_SECONDS),
                            High[i], Low[i], Close[i], Open[i], Volume[i]);
        }
        FileClose(handle);
        Comment("File "+nameData+" has been created. "+TimeToStr(TimeCurrent(), TIME_SECONDS) );
        return(0);                            
   }      
   old_bars = Bars;              // remember how many bars are known
   return(0);
}
 

Please use this to post code . . . it makes it easier to read.

 
parliament718:

Hi I found this script on this website somewhere a while ago and decided to give it a try now. The problem is it overwrites older data once it gets to a 101 lines. I dont know how to code in MQL but looking at the code I see length = 100 and the for loop will only iterate that many times since i = length - 1. I would like to remove this limitation and have the file grow indefinitely. But I have a feeling its limited like that for a reason. Probably a memory bottleneck that will occur without it?

Again I have no knowledge of MQL but I would assume its something along the lines of creating the file handle in init(), closing it in deinit(), and writing to the file without a for loop in the start(). I tried this but being that I dont know what Im doing I failed.

Can somebody please be so kind as to just modify this code for me?


"extern int" length = 100;

This value you can change on your chart So you can choose yourself this value because it is "extern" means an input parameter of your program

 
deVries:


"extern int" length = 100;

This value you can change on your chart So you can choose yourself this value because it is "extern" means an input parameter of your program

Hi deVries, thanks for your input but doesnt that script create a new file from scratch every time and then writes "length"-many lines every time? If I were to increase the value of length it would just severely slow it down if it had to write say 10,000 lines every time it runs. Wouldnt a better solution be to just add ONE line of current data to the existing file every time it runs rather than recreating the entire file from scratch and writing 10,000 lines of lookback data. Furthermore, I mentioned I would like the file to grow indefinitely and setting even a very high value to length will eventually make the file start to overwrite itself when it reaches that threshold.

Thank you in advance to anyone who can make this modification for me.

 
The attached script should be what your looking for. If you want to just run it once then remove the if(!IsStopped()) and the Sleep function. I only added them so it would check every 30 seconds if there is a new bar. It checks to see if the file already exists and if it does then it adds a line at the end.
Files:
datavscript.mq4  2 kb
 

Greatly appreciated heelflip! Ill check this when the market opens since I wont be getting any new bars right now =p.
One thing I noticed though is it still writes a 100 bars of lookback data the first time it runs, which is certainly not a problem as long as it doesnt start overwriting when new bars come in, but I noticed the very first line in the file is out of the sequence for some reason:

Line 1: 2011.11.25;21:30:00...

Line 2: 2011.11.24;21:15:00...

Line 3 :2011.11.24;21:30:00...

......

Line 99: 2011.11.25;21:30:00... same as first line

Line 100: 2011.11.25;21:45:00...

Also, can you please explain what you mean by

If you want to just run it once then remove the if(!IsStopped()) and the Sleep function. Do you mean if I just want to get the last 100 bars and stop there?

Again, I really appreciate your help. I dont know why this specific script is not easily available and I had to get you to actually write it. it really seems like the easiest way to start a data collection for outside testing, unless im wrong?

Thanks again.

 

I forgot to go to the end of the file before adding in a new line hence the out of sequence. I've fixed that now. Attached is a single data and because I felt like it I made it into a multi-currency version as well. The new versions check the last entry of the file to make sure that the last time is different to the current time.

Scripts normally only run through the coding once before being removed. What the if(!IsStopped()) does is loop it until you stop it. This means it's always gathering data. The Sleep is the there to put a break in between runs otherwise once it gets to the end it will start straight back at the beginning.

So we'll just have to wait till markets open to see if it records new bars automatically.

 
And the normal script
Files:
datacscript.mq4  3 kb
 
heelflip, it doesnt seem to be writing new bars to the file. Its not overwriting anything either though so that's a good start =]
Post here if you come up with anything. Personally, Im completely fine with manually running the script on the desired currency/frequency, if you can get just that one to work it would be great!

Thanks
 

Ok i've worked out most if not all the bugs which I've found, I think the problem was that you may of have the initial value too high. I've fixed that now. You can turn off the print statments, there were mainly for checking that it was updating right. When I run it, the is no repeats of bars, bars which revert back to 1970 (happened when i tried to read 10000 days :(), bars skipped (check automatically and updates since the last bar recorded so even if you stop running the script it will record those bars), changed the update period to every hh:mm:30 on the server time (makes sure there is a new bar) and this self-adjusts to compensate for longer or shorter processing times.

Hope it works for you,

Heelflip43

 

Ok I've fixed it now the single script now. I can't think of why that is, it's working for all the currencies and time frames on mine. Any how the single one is attached.

If I were to open all the time frames i.e. M1-D1 for all the currencies which I trade (currently 6), that's 42 windows which I would have to open where as with the multi I only need 1. Plus I learned a thing or two about how to make multi-currency programs, which was the main reason to do it.

Files:
datauscript.mq4  4 kb
12
New comment