How to filter a CSV file with a large number of records?

New comment
 
I downloaded a 10-year data file in CSV format with over 3 million records. 

I want to import this CSV file into MetaTrader 5 as a custom symbol.

However, before importing the data, I need to filter records from the CSV file. In other words, I want the CSV file to only contain data within specific hours.

How can I quickly prepare this CSV file with large records before importing it as a custom symbol  into MetaTrader?
 
Since the CSV file has over 3 million records, it couldn't be loaded in Excel completely. 

Do I need to code a script in Python, or can this task be done quickly in MQL5 ?
 
B.Moreno #: Since the CSV file has over 3 million records, it couldn't be loaded in Excel completely. Do I need to code a script in Python, or can this task be done quickly in MQL5 ?

I see three options, but there are probably more:

  • Import it into a database application, apply a filter selection to the data, before exporting the final results.
  • Code a utility, in whatever language you want, to read the data and filter it, and export it.
  • Code a MQL5 script to read the data, filter it, and create the custom symbol directly.
 
1. Write a python script and import 'pandas' package.

2. If you cannot use python, use Ron's CSV Editor or CSVEd programs

 
Thank you for both of your guidance.
 
I don't know how large your CSV file is, but I know EditPad Lite can open relatively large files (megabytes) without locking up like Notepad does.  Failing that, truly massive files (gigabytes) can easily be opened for editing/trimming with HxD Hex Editor, although you'll have to carefully find the desired trim points as everything will be run together (no lines) in the editor.  Since CSV files are simply text files where rows are separated by lines (carriage return characters) and columns by commas, one doesn't need to use spreadsheet software to make basic edits.
New comment
Reason: