reading second row of a csv file

 

Hello

 I have this code that i have been working, it find and opend the file(s) fine but I cant seem to extract the correct data, it is for pivot point data:

bool  pivot(){
        bool found=false;
        int i,hd,err,pos,time,stime;
        string url,sym,request,fn,prom,out,prefix,res;

        sym = Symbol();

   if(week==0 && month==0){if(DayOfWeek()==0){
      time = iTime(sym,PERIOD_D1,0);
      }
//+------------------------------------------------------------------+      
        if(DayOfWeek()==1){
   if(TimeToStr(TimeCurrent(),TIME_MINUTES)>="22:04"){
        time = iTime(sym,PERIOD_D1,0);} 
   if(TimeToStr(TimeCurrent(),TIME_MINUTES)<"22:04"){
        time = iTime(sym,PERIOD_D1,1);} 
   if(Symbol()=="COTTON" || Symbol()=="ESTX50" || Symbol()=="SUGAR" && TimeToStr(TimeCurrent(),TIME_MINUTES)<"22:04"){
      time = iTime(sym,PERIOD_D1,d);} 
        }
//+------------------------------------------------------------------+      
      if(DayOfWeek()>=2){
   if(TimeToStr(TimeCurrent(),TIME_MINUTES)>="22:04"){
        time = iTime(sym,PERIOD_D1,0);}  
   if(TimeToStr(TimeCurrent(),TIME_MINUTES)<"22:04"){
        time = iTime(sym,PERIOD_D1,1);}
        }       
//+------------------------------------------------------------------+   
   prefix = StringSubstr(TimeToStr(time),0,10)+"_"+"daily_";}
//+------------------------------------------------------------------+ 
  if(month==0 && week==1){
   if(DayOfWeek()==1){time = iTime(sym,PERIOD_D1,0);}
   if(DayOfWeek()==2){time = iTime(sym,PERIOD_D1,1);}
   if(DayOfWeek()==3){time = iTime(sym,PERIOD_D1,2);}
   if(DayOfWeek()==4){time = iTime(sym,PERIOD_D1,3);}
   if(DayOfWeek()==5){time = iTime(sym,PERIOD_D1,4);}
   prefix = StringSubstr(TimeToStr(time),0,10)+"_"+"weekly_";}
//+------------------------------------------------------------------+ 
   if(month==1 && week==0){
      time = iTime(sym,PERIOD_MN1,0)+(2*3600);
         prefix = StringSubstr(TimeToStr(time),0,10)+"_"+"monthly_";}
//+------------------------------------------------------------------+    
   fn = prefix + sym + ".csv";
   hd = FileOpen(fn,FILE_CSV|FILE_READ,",");

                prom = FileReadString(hd);
                stime= StrToInteger(StringSubstr(prom,0,10));

        
   FileClose(hd);       

                for(i=0;i<8;i++){
                        pos = StringFind(prom,",",0);
                        res = StringSubstr(prom,0,pos);
                        prom = StringSubstr(prom,pos+1);
                                double fPiv = NormalizeDouble(StrToDouble(res),Digits);
                                switch(i){
                                        case 1 : pivot[3] = fPiv; break;
                                        case 2 : pivot[2] = fPiv; break;
                                        case 3 : pivot[1] = fPiv; break;
                                        case 5 : pivot[4] = fPiv; break;
                                        case 6 : pivot[5] = fPiv; break;
                        
       }
   }      
}

Attached is the file that I am trying to read.

Suggestions welcome.

Thanks in advance

 

Antony 

 

Basically I just need to read the data from the seconf row of a csv file, the data is price data.

Antony 

 

Something like this?

int handle=FileOpen(FileName,FILE_CSV|FILE_READ,'@'); //<-- use a delimiter which does not exist in your file
if(handle>0){                                         //    MQL4 does separate with \r\n (https://book.mql4.com/functions/files)
   int line2read=2;                                   
   int line=0;
   string myline;
   while(!FileIsEnding(handle)){        
      myline=FileReadString(handle);      
      line++;     
      if(line==line2read)break;      
   }
   FileClose(handle);

I couldn't find out whether it is possible to use '\r\n' as the delimiter in the FileOpen function or not. I did some tests and got: "wrong delimiter for FileOpen as CSV function"

Btw. "Max file size is 4 MB (mq4, ex4, mq5, ex5, mqh, mqt, zip, txt)."
Either you tried it to upload as .csv or it is bigger than 4 MB (what I don't expect). 

 

@kronin. 

\r\n is not delimiter. It's a control character to represent a newline. \r is carriage return, and \n is new line feed. In mql4 FileWrite (https://docs.mql4.com/files/FileWrite), the \n\r is added automatically. 

\n = LF (Line Feed) used in UNIX,   \r = CR (Carriage Return) used in Mac,   \n\r = CR + LF used in Windows 

Here's the reference https://en.wikipedia.org/wiki/Newline  . 

There's well known bug in FileIsEnding() that 's not yet fixed. It does not return true when file is ending. So instead using FileIsEnding(), we better use FileSeek() < FileSize(), which is more reliable. Test the code below, if you like.

BTW, thanks kronin, for helping/replying in the forum. 


@tonyjms2005

Do you want to read the column or the row ?. The column is the vertical one while the row is the horizontal one. 

I ask that because you didn't attach the csv files. Just open the csv file in MetaEditor, and copy paste here, some of the content of your .csv file.  

Please read again the docs about FileReadString (https://docs.mql4.com/files/FileReadString). It says, "...  For text files, the string will be read before the delimiter ...". Your delimiter is ',' (= comma). So after the delimiter, FileReadString won't read anything.

To solve this, just keep reading the file. That because after first reading, the file pointer is at the delimiter and ready to read the next string.

For example, say you have 2 rows and 4 columns of data like this

1.234562.345673.45678 4.56789 
9.876548.765437.65432 8.76543

 

Your codes should be like this. To check if your codes is correct or not, just print the result, like my example below.

  FileDelete ("test.txt");
  int hnd,pos;
  string txt;

  hnd = FileOpen ("test.txt", FILE_CSV|FILE_WRITE, ",");
  if (hnd > 0)
     {
     FileWrite (hnd, 1.23456, 2.34567,3.45678,4.56789);
     FileWrite (hnd, 9.23456, 8.34567,7.45678,6.56789);
     FileClose (hnd);
     }

  hnd = FileOpen ("test.txt", FILE_CSV|FILE_READ, ",");
  if (hnd > 0)                                           //--- read the file if only hnd is > 0
     {
     //Print ("File size ",FileSize(hnd));
     while (FileTell (hnd)< FileSize(hnd))
     //while (FileIsEnding (hnd) == false)
       {
       pos ++;
       //Print ("Loop number ",pos);  //--- read row number
       //Print ("start file pointer at ", FileTell (hnd),". File size ",FileSize(hnd));

       txt = FileReadString (hnd);   //--- read first column
       //Print (txt);
     
       txt = FileReadString (hnd);   //--- read second column
       //Print (txt);
       
       txt = FileReadString (hnd);   //--- read third column
       //Print (txt);
       
       txt = FileReadString (hnd);   //--- read fourth column
       //Print (txt);
       
       //Print ("End file pointer at ", FileTell (hnd),". File size ",FileSize(hnd));   
       if (FileIsEnding (hnd) == false)
          Print ("File is not ending. More to come");
       }
     
     FileClose (hnd);                                      //--- and so close the file if only hnd is > 0
 
phi.nuts:

@kronin. 

\r\n is not delimiter. It's a control character to represent a newline. \r is carriage return, and \n is new line feed. In mql4 FileWrite (https://docs.mql4.com/files/FileWrite), the \n\r is added automatically. 

\n = LF (Line Feed) used in UNIX,   \r = CR (Carriage Return) used in Mac,   \n\r = CR + LF used in Windows 

Here's the reference https://en.wikipedia.org/wiki/Newline  . 

There's well known bug in FileIsEnding() that 's not yet fixed. It does not return true when file is ending. So instead using FileIsEnding(), we better use FileSeek() < FileSize(), which is more reliable. Test the code below, if you like.

BTW, thanks kronin, for helping/replying in the forum. 


Thanks phi.nuts
I seem to have missed the FileIsEnding bug (public bug tracking list? :-) ). So my code is dangerous as it does an infinite loop, if the file has less lines than the line number we are looking for. I will try yours.

I know about carriage return and line feed, but don't know how to us it as THE delimiter in FILEOPEN()

Book
"The specified symbols (";" and "\r\n") are used to separate entries in MQL4." This is next to figure 148

I read this as "if no delimiter is found in a  row, \r\n does separate it".  (English is not my first language)


Last but not least, I have started with the book and carefully follow this forum about 6 months ago. I thought it is time to give something back (Have had holidays this week. So not that frequent from next week on)!
Thank you guys for all the great answers, articles, code snippets....

 
phi.nuts:

@kronin. 

\r\n is not delimiter. It's a control character to represent a newline. \r is carriage return, and \n is new line feed. In mql4 FileWrite (https://docs.mql4.com/files/FileWrite), the \n\r is added automatically. 

\n = LF (Line Feed) used in UNIX,   \r = CR (Carriage Return) used in Mac,   \n\r = CR + LF used in Windows 

Here's the reference https://en.wikipedia.org/wiki/Newline  . 

There's well known bug in FileIsEnding() that 's not yet fixed. It does not return true when file is ending. So instead using FileIsEnding(), we better use FileSeek() < FileSize(), which is more reliable. Test the code below, if you like.

BTW, thanks kronin, for helping/replying in the forum. 


@tonyjms2005

Do you want to read the column or the row ?. The column is the vertical one while the row is the horizontal one. 

I ask that because you didn't attach the csv files. Just open the csv file in MetaEditor, and copy paste here, some of the content of your .csv file.  

Please read again the docs about FileReadString (https://docs.mql4.com/files/FileReadString). It says, "...  For text files, the string will be read before the delimiter ...". Your delimiter is ',' (= comma). So after the delimiter, FileReadString won't read anything.

To solve this, just keep reading the file. That because after first reading, the file pointer is at the delimiter and ready to read the next string.

For example, say you have 2 rows and 4 columns of data like this

1.234562.345673.45678 4.56789 
9.876548.765437.65432 8.76543

 

Your codes should be like this. To check if your codes is correct or not, just print the result, like my example below.

 

 

Hi

Thank you for thequick replies. I am trying to read the data on the SECOND ROW, so the example table above I would ignore the Row starting with 1.23456 and read the row starting with 9.87654

 You see the first row has invalid data for use on the chart so we must ignore that.

Pivot  S1  S2  S3  R1  R2  R3
1718.36 1704.66 1694.67 1680.97 1728.35 1742.05 1752.04
             
             

Thank you again

 

Antony 

 

Like this? Not anymore reading the full line, but split it into the values and save it in the pivot array:

string FileName="test.csv";
FileDelete("test.csv");

int handle = FileOpen ("test.csv", FILE_CSV|FILE_WRITE, ",");
if (handle>0){
   FileWrite(handle,"Pivot","S1","S2","S3","R1","R2","R3");
   FileWrite(handle,1718.36,1704.66,1694.67,1680.97,1728.35,1742.05,1752.04);
   FileClose(handle);
}

int line2read=2;
int values=7;    
string pivot[2,7];                                    //    [lines,values]
                                                      //    Arrays cannot initialize with vars, only the first dimension can be resized
handle=FileOpen(FileName,FILE_CSV|FILE_READ,',');     
if(handle>0){                                         
   int line=0;
   string myline;
   while(FileTell(handle)< FileSize(handle)){         //<-- FileIsEnding is buggy and does never return true
      for(int i=0;i<values;i++){                      //    Use FileTell and FileSize instead (https://forum.mql4.com/52640)
         pivot[line,i]=FileReadString(handle);
      }   
      line++;                                         
      if(line==line2read)break;      
   }
}   
FileClose(handle);
for(int j=0;j<values;j++)Print("Value: " + pivot[line2read-1,j]); //-1 as line 2 is at pos 1 in the Array


 The disadvantage is, you MUST know how many values the rows include. Therefore, maybe someone has a better option.

 

Hi

Thank you, I may have to play around with that as the Print output returns nothing:

pivot[line2read-1,j
 
tonyjms2005:

Basically I just need to read the data from the seconf row of a csv file, the data is price data.

Antony 


Great idea...
 

one needs to be careful when reading files with MQL functions. FileReadString() has a bug, it reads at most 4095 characters. it returns and suddenly you don't know if you are in the middle of a line or not. if a line is longer all the codes published at this forum fail. an error safe version of reading lines is a bit more complex. you might say "4095, never ever i will hit this limit". that's ok, but there are times when line lengths are unknown/undefined. if there's interest i will post it.

another point: using a separation character under the precondition that your file doesn't contain it is not a very good idea.

one more: FileIsEnding() is not buggy. FileIsEnding()/FileLineIsEnding() return the correct result *after* the read operation that "steps over" the limit. so, setting a file pointer to the end of file and calling FileIsEnding() will return FALSE. but calling FileRead* at the end of file and then FileIsEnding() will return the correct result, TRUE. the same is valid for lines.

 
paulepanke:

1. one needs to be careful when reading files with MQL functions. FileReadString() has a bug, it reads at most 4095 characters. it returns and suddenly you don't know if you are in the middle of a line or not. if a line is longer all the codes published at this forum fail. an error safe version of reading lines is a bit more complex. you might say "4095, never ever i will hit this limit". that's ok, but there are times when line lengths are unknown/undefined. if there's interest i will post it.

2. another point: using a separation character under the precondition that your file doesn't contain it is not a very good idea.

3. one more: FileIsEnding() is not buggy. FileIsEnding()/FileLineIsEnding() return the correct result *after* the read operation that "steps over" the limit. so, setting a file pointer to the end of file and calling FileIsEnding() will return FALSE. but calling FileRead* at the end of file and then FileIsEnding() will return the correct result, TRUE. the same is valid for lines.

 

1. Yes, I am interessted.
2. I know, but how can I use CRLF as the separator if I want to read the full line (not separated by anything)? I struggled with that and then gave up. 
3. Could you give me an example please? 

Many thanks 

Reason: