File operation and Array bug??

[[Deleted]]

Hello,

I have come across a wierd bug where my data does not seem to load correctly into an array. Any help would be much appreciated. The attached code and csv faile are trimmed verision of what I am working with, but the error is still persistant in this example. What I am trying to do is load a CSV file that is 17 columns wide by 360 rows into an array called Trainingdata. If I limit the amount of iterations of my loop to less the 175 the data loads into the Array fine, but the first 15 rows are all zeros and I get a 4002 error. If i increase the iterations to 360 then the advisor breaks down and loses the first 100-150 rows of the Array and I wind up with a 4002 erro, which at that point disable the full verision of my advisor. The really wierd thing is it will print from a random point each time the EA is loaded and run to the end of the Array.

Any thoughts or help would be much appreciated.

I wasn't able to load the CSV file as well, but any CSV thats 360 x 17 should produce the same results. My CSV are a combination of +/- numbers both integer and doubles. There are no String, Dates or colors. If it makes it easier I can add the CSV file as a txt file later

#include <stderror.mqh>
#include <stdlib.mqh>

#property copyright "Copyright © 2010, Chris Freeman"

  
//---------------------------------------------------------------- 1a --



int nn_layer = 4;
int nn_input = 16; // Number of input neurones
int nn_hidden1 = 8; // Number of neurones on the first hidden layer
int nn_hidden2 = 4; // number on the second hidden layer
int nn_output = 1; // number of outputs


static double      trainingData[][17];  // IMPORTANT! size = nn_input + nn_output


int maxTraining = 10;  // 
double targetMSE = 0.01; // the Mean-Square Error of the neurones we should get at most (you will understand this lower in the code)

int    ann; // This var will be the identifier of the neuronal network.
   
//==========================================================================================================
int deinit() {
  
   return(0);
}
//===========================================================================================================
int init() {
   double MSE;
   
   Print("=================================== START EXECUTION ================================");
   

   ArrayResize(trainingData,1);
   
   Print("##### INIT #####");

   
   Print("##### REGISTER DATA #####");
   
    int Handle;
    int filechk;
    string file="ann_test_input2.csv";                          
   static    double Table[17];                       
                                                
    

    Handle = FileOpen(file, FILE_READ|FILE_CSV ,","); 
     if(Handle==-1)              
       Alert("An error while opening the file. "); 
       for(int j=1; j<360; j++)                               // <------total of 360 rows in the csv FILE 
                                             
    {
    Table[1] = FileReadNumber(Handle);
    Table[2] = FileReadNumber(Handle);
    Table[3] = FileReadNumber(Handle);
    Table[4] = FileReadNumber(Handle);
    Table[5] = FileReadNumber(Handle);
    Table[6] = FileReadNumber(Handle);
    Table[7] = FileReadNumber(Handle);
    Table[8] = FileReadNumber(Handle);
    Table[9] = FileReadNumber(Handle);
    Table[10] = FileReadNumber(Handle);
    Table[11] = FileReadNumber(Handle);
    Table[12] = FileReadNumber(Handle);
    Table[13] = FileReadNumber(Handle);
    Table[14] = FileReadNumber(Handle);
    Table[15] = FileReadNumber(Handle);
    Table[16] = FileReadNumber(Handle);
    Table[0] = FileReadNumber(Handle);   //output variable
    


     
    prepareData("train",Table[1],Table[2],Table[3],Table[4],Table[5],Table[6],Table[7],
                        Table[8],Table[9],Table[10],Table[11],Table[12],Table[13],Table[14],
                        Table[15],Table[17],Table[0]); //---- add other array elements
                                                                          
    
        }
    
      if (filechk<0)
         {
         Alert("error");
         FileClose(Handle);
         return;
         }
      Alert("worked");
      FileClose(Handle); 
   
   printDataArray();
   
   
   Print("##### TRAINING #####");

   
Print("the last error code was = ", GetLastError());
      
   Print("##### RUNNING #####");
  
   
                                      

   
   Print("=================================== END EXECUTION ================================");


   return(0);
}

int start() 
{
   return(0);
}

//===============================================================================================
void printDataArray() {
   int i,j;
   int bufferSize = ArraySize(trainingData)/(17)-1;
   string lineBuffer = "";
   for (i=0;i<bufferSize;i++) {
      for (j=0;j<(17);j++) {
         lineBuffer = StringConcatenate(lineBuffer, trainingData[i][j], ",");
      }
      debug("DataArray["+i+"]", lineBuffer);
      lineBuffer = "";
   }
}
//================================================================================================
void prepareData(string action, double var_1, double var_2, double var_3, double var_4, double var_5, double var_6,
                                double var_7, double var_8, double var_9, double var_10, double var_11, double var_12,
                                double var_13, double var_14, double var_15, double var_16, double output) 
                                                                             
{
   int i;
   static   double inputVector[];
   static   double outputVector[];
   // we resize the arrays to the right size
   ArrayResize(inputVector,16);
   ArrayResize(outputVector,1);
   
   inputVector[0] = var_1;
   inputVector[1] = var_2;
   inputVector[2] = var_3;
   inputVector[3] = var_4;
   inputVector[4] = var_5;
   inputVector[5] = var_6;
   inputVector[6] = var_7;
   inputVector[7] = var_8;
   inputVector[8] = var_9;
   inputVector[9] = var_10;
   inputVector[10] = var_11;
   inputVector[11] = var_12;
   inputVector[12] = var_13;
   inputVector[13] = var_14;
   inputVector[14] = var_15;
   inputVector[15] = var_16;
   
   outputVector[0] = output;
   
   if (action == "train") {
      addTrainingData(inputVector,outputVector);
   }
   if (action == "compute") {
      for(i=0;i<16;i++)
       compute(inputVector);
   }
}

//================================================================================================
void addTrainingData(double inputArray[], double outputArray[]) {
   int j;
   int bufferSize = ArraySize(trainingData)/(17)-1;
    
  
   for (j=0;j<16;j++) {
      trainingData[bufferSize][j] = inputArray[j];
   }
   for (j=0;j<1;j++) {
      trainingData[bufferSize][16+j] = outputArray[j];
   }
   
   ArrayResize(trainingData,bufferSize+2);
}


/*************************
** teach()
** Get all the trainign data and use them to train the neurones one time.
** In order to properly train the neurones, you need to run this function many time,
** until the Mean-Square Error get low enough.
*************************/
double teach() {
   int i,j;
   double MSE;
static   double inputVector2[];
static   double outputVector2[];
   
   ArrayResize(inputVector2,16);
   ArrayResize(outputVector2,1);
   int call;
   int bufferSize = ArraySize(trainingData)/(17)-1;
   for (i=0;i<bufferSize;i++) {
      for (j=0;j<16;j++) {
         inputVector2[j] = trainingData[i][j];
       }
         outputVector2[0] = trainingData[i][16];
      //f2M_train() is showing the neurones only one example at a time.
                
      //call = f2M_train(ann, inputVector2, outputVector2);
      
   }
   // Once we have show them an example, we check if how good they are by checking their MSE. If it's low, they learn good!
  
   return(0);
}

//=========================================================================================

double compute(double inputVector[]) {
   int j;
   int out;
   double output;
   ArrayResize(inputVector,16);
   
   //RE-ENABLE
   //out = f2M_run(ann, inputVector);
   //output = f2M_get_output(ann, 0);
   
         
   //debug("Calclated output as ",output);
   return(output);
}

//========================================================================================
void debug(string a, string b) {
   Print(a+" ==> "+b);
}

//========================================================================================
int Fun_Error(int Error)                        // 
  {
   switch(Error)
     {                                          //             
      case  4: Alert("Trade server is busy. Trying once again");
         Sleep(3000);                           // 
         return(1);                             // 
      case 135:Alert("Price changed. Trying once again");
         RefreshRates();                        // 
         return(1);                             // 
      case 136:Alert("No prices. Waiting for a new tick.");
         while(RefreshRates()==false)           //
            Sleep(1);                           //
         return(1);                             //
      case 137:Alert("Broker is busy. Trying once again.");
         Sleep(3000);                           //
         return(1);                             // 
      case 146:Alert("Trading subsystem is busy. Trying once again");
         Sleep(500);                            // 
         return(1);                             // 
                                                // 
      case  2: Alert("Common error");
         return(0);                             // 
                                 //
                                   // 
      case 133:Alert("Trading forbidden");
         return(0);                             // 
      case 134:Alert("Not enough money to execute operation");
         return(0);                             // 
      default: Alert("Error occurred",Error); //    
         return(0);                             // 
     }
  }
//-------------------------------------------------------------- 11 --
                           // 

[gordon]

Buggy code... 4002 means you are trying to access indexes not in the array.

ERR_ARRAY_INDEX_OUT_OF_RANGE

4002

Array index is out of range.

Try to pinpoint the problem using Print() statements before looping on arrays using ArrayRange() to print array dimension sizes. Also note that ALL arrays in MQL4 are static (even arrays defined in functions, regardless of if they were declared static or not), so this might be a possible cause of your bug.

[[Deleted]]

There is a post discussing Excel file write/read.

https://www.mql5.com/en/forum/107448/page2

and fx1 provides some functions especially for write/read datas to/from excel files,which makes this kind of issue to be easy.

http://www.fx1.net/excellink.php