How to modify or delete a row in .CSV file.

Fabio Brondo  

Hi all!
I'm coding my very first trading bot, but i'm stuck on this point since a while, even if i've looked to the guide deeply...
I have coded a bot that i sending telegram messages every time that i'm doing an action on MT4, and is also saving the operations to a csv file like this one:

Order ID;Order Type;Open Price;Open Time;Symbol;Lots;Stop Loss;Take Profit;Message ID

166816632;buy;1.20339;2023.02.06 13:49:38;GBPUSD+;0.01;0.00000;0.00000;648
166835123;buy limit;1.00000;2023.02.06 15:29:12;GBPUSD+;0.01;0.00000;0.00000;649
166816632;buy;1.20339;2023.02.06 13:49:38;GBPUSD+;0.01;0.00000;0.00000;650
166836511;buy;1.20568;2023.02.06 15:35:10;GBPUSD+;0.01;0.00000;0.00000;651
166836537;buy limit;1.00000;2023.02.06 15:35:17;GBPUSD+;0.01;0.00000;0.00000;652
166836757;buy limit;1.00000;2023.02.06 15:36:05;GBPUSD+;0.01;0.00000;0.00000;654
166836830;buy limit;90.00000;2023.02.06 15:36:23;SGDJPY+;0.01;0.00000;0.00000;655
166837990;buy limit;1.00000;2023.02.06 15:41:14;GBPUSD+;0.01;0.00000;0.00000;658

What I would like to do, is:

Once the order is closed or deleted, it must be deleted also on the file.

Every time i'm updating the SL/TP or the entry price of a planned order, it must be updated too.

I'm trying to do that using the Order ID as a reference but currently i'm unable.

Can someone please help me?

Thank you in advance for your time!


Fabio

Fernando Carreiro  
Fabio Brondo:I'm coding my very first trading bot, but i'm stuck on this point since a while, even if i've looked to the guide deeply... I have coded a bot that i sending telegram messages every time that i'm doing an action on MT4, and is also saving the operations to a csv file like this one: What I would like to do, is: Once the order is closed or deleted, it must be deleted also on the file.Every time i'm updating the SL/TP or the entry price of a planned order, it must be updated too.I'm trying to do that using the Order ID as a reference but currently i'm unable. Can someone please help me? Thank you in advance for your time! Fabio

CSV files are sequencial non-structured files, so you cannot change lines directly.

You have to read in the entire file contents, converting it into structured data, make the changes and then write out the entire file again.

Or, if you already have the data available internally then just rewrite the entire file again without needing to read it in.

Another option, is to create a structured file format that can be accessed randomly and modified ass needed instead of sequentially.

You can also create a hybrid CSV file that has a proper fixed record size and can be read as a CSV by adding extra white-space to align everything.

Fabio Brondo  
Fernando Carreiro #:

CSV files are sequencial non-structured files, so you cannot change lines directly.

You have to read in the entire file contents, converting it into structured data, make the changes and then write out the entire file again.

Or, if you already have the data available internally then just rewrite the entire file again without needing to read it in.

Another option, is to create a structured file format that can be accessed randomly and modified ass needed instead of sequentially.

You can also create a hybrid CSV file that has a proper fixed record size and can be read as a CSV by adding extra white-space to align everything.

Hi @Fernando Carreiro, thank you for your feedback.
I've used the CSV format since i've found a lot of examples and was easier to adapt the example code to my scope.

Do you have any suggestion on wich would be the best file format to use for what i need to do?

Thank you

Fernando Carreiro  
Fabio Brondo #: Hi @Fernando Carreiro, thank you for your feedback. I've used the CSV format since i've found a lot of examples and was easier to adapt the example code to my scope. Do you have any suggestion on wich would be the best file format to use for what i need to do? Thank you

I have no ideia how or why you are using the CSV file, so I cannot make any suggestions without a more detailed explanation.

Fabio Brondo  
Fernando Carreiro #:

I have no ideia how or why you are using the CSV file, so I cannot make any suggestions without a more detailed explanation.

Ok, here's the idea of the project:

First of all create a bot that sends on telegram each operation that is done on MT4 (DONE).

Than is saving in parallel all the operations on a file (TBD wich kind of):

  • New order (SAVE)
  • Modified order SL/TP/Entry price (UPDATE)
  • Deleted or closed order (DELETE THE ROW)

The final scope of this file is to be read from another BOT (slave) that places automatically the orders on MT4 (To be coded once this one is finished).

Fernando Carreiro  
Fabio Brondo #:Ok, here's the idea of the project: First of all create a bot that sends on telegram each operation that is done on MT4 (DONE). Than is saving in parallel all the operations on a file (TBD wich kind of):

  • New order (SAVE)
  • Modified order SL/TP/Entry price (UPDATE)
  • Deleted or closed order (DELETE THE ROW)

The final scope of this file is to be read from another BOT (slave) that places automatically the orders on MT4 (To be coded once this one is finished).

If that is case, then write the operations sequential to the file instead of the trade's status. If another bot is going to read it as if it was a "Trade Copier", then the file should be sequencial and have the operations to be carried out.

In other words it would be similar to what you already have in the Experts and Journal logs. No line ever gets deleted or changed, and all the operations follow a chronological sequence.

Lorentzos Roussos  
Fabio Brondo #:

Ok, here's the idea of the project:

First of all create a bot that sends on telegram each operation that is done on MT4 (DONE).

Than is saving in parallel all the operations on a file (TBD wich kind of):

  • New order (SAVE)
  • Modified order SL/TP/Entry price (UPDATE)
  • Deleted or closed order (DELETE THE ROW)

The final scope of this file is to be read from another BOT (slave) that places automatically the orders on MT4 (To be coded once this one is finished).

As @Fernando Carreiro said it depends , and his suggestion is excellent , but if you don't need the closed trades anymore you can write to a binary file .

Reading and writing will be faster , you maintain the structure in memory and update it when needed .

It looks hard but it really is not .

What you need is :

  1. Adding and removing trades from a collection
  2. Finding trades in a collection
  3. Saving and loading the collection

It comes down to just storing binary data of your base members of your custom object , you know what you are storing and how much so you can trust it.

What are some specs of your trades that you need ?, i can show you an example based on that .

Fabio Brondo  
Lorentzos Roussos #:

As @Fernando Carreiro said it depends , and his suggestion is excellent , but if you don't need the closed trades anymore you can write to a binary file .

Reading and writing will be faster , you maintain the structure in memory and update it when needed .

It looks hard but it really is not .

What you need is :

  1. Adding and removing trades from a collection
  2. Finding trades in a collection
  3. Saving and loading the collection

It comes down to just storing binary data of your base members of your custom object , you know what you are storing and how much so you can trust it.

What are some specs of your trades that you need ?, i can show you an example based on that .

Hi @Lorentzos Roussos!
Thank you for your feedback :)
The specs that I need are:


Lot size, TP value, SL Value, Price, Message ID,Random number*

Currently I'm starting to think about a Json file to do (i think) what you and Fernando are suggesting
Could this one be the right path?

* Since order number is different from account to account, and it means that the system could potentially place multiple orders if checking the shared json file is not finding the order already opened with the order ID. So i want to use a random number that is different for every order.


Lorentzos Roussos  
Fabio Brondo #:

Hi @Lorentzos Roussos!
Thank you for your feedback :)
The specs that I need are:


Lot size, TP value, SL Value, Price, Message ID,Random number*

Currently I'm starting to think about a Json file to do (i think) what you and Fernando are suggesting
Could this one be the right path?

* Since order number is different from account to account, and it means that the system could potentially place multiple orders if checking the shared json file is not finding the order already opened with the order ID. So i want to use a random number that is different for every order.


Hi , try this , i did not understand the use of the random number .

#property version   "1.00"
input int magic=235;//magic #
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
//first you create a custom object for your trade
  class myTrade
  {
  public:
  //then you declare the variables the trade has
    //the position id in the terminal 
      ulong position_id;
    //the message id in the telegram
      long message_id;
    //the lots 
      double lots;
    //the open price sl tp
      double op,sl,tp;
    //the open time 
      datetime open_time;
    //a flag indicating the trade must be removed 
      bool for_removal;
    //i did not understand why you need the random number 
      /*
      so lets setup the basics :
      -what happens when you create this object to the values
      */
      myTrade(void){reset();}
 void reset(){
      position_id=0;
      message_id=-1;
      lots=0.0;
      op=0.0;
      sl=0.0;
      tp=0.0;
      open_time=0;
      for_removal=false;
      }
      //then you'll need to setup a trade after you open it 
 void setup(ulong _position_id,
             long _message_id,
           double _lots,
           double _op,double _sl,double _tp,
           datetime _time){
      //so in your sequence you open a trade , then send the telegram bot a message and then set it up in your collection
        position_id=_position_id;
        message_id=_message_id;
        lots=_lots;
        op=_op;
        sl=_sl;
        tp=_tp;
        open_time=_time;
      }
      //then you'll need to save a trade in a file 
 void save(int file_handle){
      FileWriteLong(file_handle,((long)position_id));
      FileWriteLong(file_handle,message_id);
      FileWriteDouble(file_handle,lots);
      FileWriteDouble(file_handle,op);
      FileWriteDouble(file_handle,sl);
      FileWriteDouble(file_handle,tp);
      FileWriteLong(file_handle,((long)open_time));
      FileWriteInteger(file_handle,((int)for_removal),INT_VALUE);
      }   
      //then you'll need to load a trade from a file 
 void load(int file_handle){
      position_id=(ulong)FileReadLong(file_handle);
      message_id=(long)FileReadLong(file_handle);
      lots=(double)FileReadDouble(file_handle);
      op=(double)FileReadDouble(file_handle);
      sl=(double)FileReadDouble(file_handle);
      tp=(double)FileReadDouble(file_handle);
      open_time=(datetime)FileReadLong(file_handle);
      for_removal=(bool)FileReadInteger(file_handle,INT_VALUE);    
      //you can add any checks here , if the trade exists etc 
      }  
      //you can add additional functions for the trades here like getting the profit etc 
  };
/* now you have the myTrade object ready you need the collection */
struct myTradesCollection{
//what does it have ? trades
  myTrade trades[];
  //what happens when you create it or remove it ?
     myTradesCollection(void){reset();}
    ~myTradesCollection(void){reset();}
void reset(){
     ArrayFree(trades);
     }
  //what else do you need ? to add a new trade to the collection
 int add_trade(ulong _position_id,
               long _message_id,
               double _lots,
               double _op,double _sl,double _tp,
               datetime _time){
     int new_size=ArraySize(trades)+1;//the new size of your collection
     ArrayResize(trades,new_size,0);
     trades[new_size-1].setup(_position_id,_message_id,_lots,_op,_sl,_tp,_time);
     //return the exact location of the new trade
     return(new_size-1);
     }
  //what else ? removal of a trade from the collection 
 int remove_trade(int ix){//you are sending the location in the collection here 
     //calc the new the collection size  
       int new_size=ArraySize(trades)-1;
     //the new size is also the location of the last trade in your collection (total-1)
     //so you move the last trade from the last slot to the ix position
       trades[ix]=trades[new_size];
     //and then you shrink the collection 
       if(new_size>0){ArrayResize(trades,new_size,0);}
       else{ArrayFree(trades);}
     //and you return the new size of the collection
     return(new_size);
     }
     //finally save the collection
bool save(string folder,string filename){
     //first create the path 
       string location=filename;
       if(StringLen(folder)>0){location=folder+"\\"+filename;}
     //if the file exists already delete it 
       if(FileIsExist(location)){FileDelete(location);}
     //create the new file 
       int f=FileOpen(location,FILE_WRITE|FILE_BIN);
       //if its opened
       if(f!=INVALID_HANDLE){
       //write the # of trades 
         FileWriteInteger(f,ArraySize(trades),INT_VALUE);
       //then loop in each trade and call save for this file 
         for(int i=0;i<ArraySize(trades);i++){trades[i].save(f);}
       //done
       FileClose(f);
       return(true);
       } 
     return(false);
     }
     //and loading the file 
bool load(string folder,string filename){
     //first reset 
       reset();
     //create the path 
       string location=filename;
       if(StringLen(folder)>0){location=folder+"\\"+filename;}
     //if the file exists open it
       if(FileIsExist(location)){
       int f=FileOpen(location,FILE_READ|FILE_BIN);
       if(f!=INVALID_HANDLE){
       //read the # of trades 
         int total=(int)FileReadInteger(f,INT_VALUE);
         if(total>0){
         //size the collection 
           ArrayResize(trades,total,0);
         //and loop and load each trade 
           for(int i=0;i<total;i++){trades[i].load(f);}
         }
       FileClose(f);
       return(true);
       }} 
     return(false);  
     }
     //you can add additional functions here like monitoring of trades getting the total profit etc
};
//declare the collection
myTradesCollection TC;
//now you have a folder for the system 
  string SystemFolder="BrondoEA";
//and a filename for the system
  string Filename="";
int OnInit()
  {
//---
  //upon start you create the filename 
  //you can have as many variations as you like based on the system 
  //but , if you are only saving by magic # you will also need to add the symbol of each trade in the structure
    Filename=_Symbol+IntegerToString(magic)+".txt";
  bool loaded=TC.load(SystemFolder,Filename);
  //now whenever a change occurs in your system you call save at the end of the check loop 
//---
   return(INIT_SUCCEEDED);
  }

If you also need to add strings into the myTrade object i'm attaching an include for having text in classes that can easily be saved and loaded 

Files:
Reason: