Handle large set of data

[[Deleted]]

Hello, can you suggest decision how to manage large data and trade with Metatrader 4. I have an EA that records in csv file in Metaeditor/File folder quotes and some variables of symbols in Market Watch. However when recording data becomes very, very large - above 1 million of rows, and if EA just records it is ok, but of there is an open order EA must read from file and take decisions, and then my pc work with 100% of processor power all the time! Does anybody has similar problem and to know how to solve it? Thanks.

[Marco vd Heijden]

Yes the solution is to stop recording.

Only hold the last (most important) values in Globalvars there is no need to save the obsolete data.

[nicholish en]

iCurrency:
Hello, can you suggest decision how to manage large data and trade with Metatrader 4. I have an EA that records in csv file in Metaeditor/File folder quotes and some variables of symbols in Market Watch. However when recording data becomes very, very large - above 1 million of rows, and if EA just records it is ok, but of there is an open order EA must read from file and take decisions, and then my pc work with 100% of processor power all the time! Does anybody has similar problem and to know how to solve it? Thanks.

Derive a class from CObject and override its Load and Save methods to load the row into your new object. Then derive a class from CList to store your collection of data, and override its Load Save methods to read your CSV. Here is a post I just made about using the collection class' Load/Save methods. https://www.mql5.com/en/forum/213003

#property strict
#include <Arrays\List.mqh>
//+------------------------------------------------------------------+
class MyObj : public CObject
{
public:
   string symbol;
   double bid;
   double ask;
};
//+------------------------------------------------------------------+
void OnStart()
{
//---
   CList list;
   for(int i=0;i<1000000;i++)
   {
      MyObj *obj = new MyObj;
      obj.symbol = Symbol();
      obj.bid = Bid;
      obj.ask = Ask;
      list.Add(obj);
   }
   
   string symbol;
   double bid,ask;
   uint st = GetTickCount();
   int cnt=0;
   for(MyObj *obj = list.GetFirstNode();obj!=NULL;obj = list.GetNextNode())
   {
      symbol = obj.symbol;
      bid    = obj.bid;
      ask    = obj.ask;
      cnt++;
   }
   uint ed = GetTickCount();
   Print("It took ",ed-st," ms to iterate over ",cnt," objects");   
}
//+------------------------------------------------------------------+

[[Deleted]]

Thanks.