Import CSV Bar Data Simply NOT Working

[windowshopr]

I've read a few articles, and still have yet to find a working solution for this, while also trying not to deviate too far from what the documentation says to do.

I'm trying to create my own custom SPY symbol (different than the default SP500m data provided by the demo server).

For ease (and THIS may be the problem, but not sure yet), I'm importing the same Symbol Specifications as the SP500m provided by MQL5 demo server.

Once that is done, I navigate to the "Bars" tab (to import 1 minute bar data), browse for my file (which is in the format the documentation asks for it to be in, like so:

) and I see this parsed result:

Cool! Or so I thought.

I click OK, and the bars show up in the previous window/screen properly, like so:

BUT, as soon as I click OK there, it all "disappears". I can't visualize my newly created symbol on the chart:

When I try to run an optimization on an EA, it immediately crashes saying that there is no history for the date range I've selected (when I know darn well there IS data in the dataset I just imported for that date range):

And when I go back into the Symbols window and find my created symbol again, it's not showing any of the bar data that I just imported:

When I click "Request" for a smaller date range that I know to be in the dataset, it doesn't return anything/0 bars.

What is happening here? Is the dataset too big at 456,376 rows of 1 minute data? I see no error messages or feedback telling me WHY it's failing to import the bar data, which according to the documentation appears to be in proper format. Is it because I'm trying to replicate the SP500m symbol settings that don't translate over to this dataset?

I've tried restarting MT5 several times now, still nothing. Others have had similar problems but many of their solutions were fixing their formatting of the data, the order of the data, or something else I read like "synchronization issues", whatever that means...

I hope I've provided enough detail here to have someone give me some insight. It's frustrating when you try to follow the documentation and it doesn't work, worse when there's no feedback.

Thanks!

[Jonan]

Did you ever get a fix for this? After 3 years of no issues with importing data, last week I've had the same issues of bar data disappearing after i import the data???

[Julian Cramer]

Hi there, 

I resolved the issue by deleting the first file inside 

C:\Users\[windows account]\AppData\Roaming\MetaQuotes\Terminal\[instance id]\bases\Custom

-> History -> <Your custom symbol> . Then delete the file 1970.hcc.  

[devmanojsoni]

hello there, i can confirm you that it is not working

when i try to import a csv file of 2048 rows, it is able to import it without any issue

but when i try to import a csv file of more than 2048 rows (4236 rows), it is NOT able it import it 

I have attached the WORKING & the NOT WORKING files, please check

[devmanojsoni]

Hi there, I changed all the datatypes to MetaTrader 5 supported datatypes, filled null values to Zero. But still not working beyond 2048 Rows:

Here's the code to change to MetaTrader 5 format using Python & Pandas:

import pandas as pd

import csv

# Read the csv file

df = pd.read_csv('NOT_WORKING.csv')

# Check the data types of each column

print(df.dtypes)

print(df.isnull().sum())

df.fillna(0, inplace=True)

# Convert the date column to datetime data type

df['Date'] = pd.to_datetime(df['Date'], format='%Y.%m.%d %H:%M:%S')

# Convert datetime data type to MetaTrader 5 format

df['Date'] = df['Date'].dt.strftime('%Y.%m.%d %H:%M')

# Change data types of columns

df['Open'] = df['Open'].astype(float)

df['High'] = df['High'].astype(float)

df['Low'] = df['Low'].astype(float)

df['Close'] = df['Close'].astype(float)

df['Tick Volume'] = df['Tick Volume'].astype(int)

df['Volume'] = df['Volume'].astype(int)

df['Spread'] = df['Spread'].astype(float)

# Write the dataframe back to a csv file

df.to_csv('UPDATED_DATETIME_DATATYPE_AND_OTHER_DATA_TYPES.csv', index=False)

[devmanojsoni]

Can any one please confirm that is there any restriction added to the new build, build 3662, version 5, 26 mar 2023?

Or team doesn't care to fix this  issue ?

[Frank Grannell]

devmanojsoni #:

Can any one please confirm that is there any restriction added to the new build, build 3662, version 5, 26 mar 2023?

Or team doesn't care to fix this  issue ?

Did you managed to find a solution?
I have the same issue.

[Carlo Salpietro Damiano]

I 

windowshopr:

I've read a few articles, and still have yet to find a working solution for this, while also trying not to deviate too far from what the documentation says to do.

I'm trying to create my own custom SPY symbol (different than the default SP500m data provided by the demo server).

For ease (and THIS may be the problem, but not sure yet), I'm importing the same Symbol Specifications as the SP500m provided by MQL5 demo server.

Once that is done, I navigate to the "Bars" tab (to import 1 minute bar data), browse for my file (which is in the format the documentation asks for it to be in, like so:

) and I see this parsed result:

Cool! Or so I thought.

I click OK, and the bars show up in the previous window/screen properly, like so:

BUT, as soon as I click OK there, it all "disappears". I can't visualize my newly created symbol on the chart:

When I try to run an optimization on an EA, it immediately crashes saying that there is no history for the date range I've selected (when I know darn well there IS data in the dataset I just imported for that date range):

And when I go back into the Symbols window and find my created symbol again, it's not showing any of the bar data that I just imported:

When I click "Request" for a smaller date range that I know to be in the dataset, it doesn't return anything/0 bars.

What is happening here? Is the dataset too big at 456,376 rows of 1 minute data? I see no error messages or feedback telling me WHY it's failing to import the bar data, which according to the documentation appears to be in proper format. Is it because I'm trying to replicate the SP500m symbol settings that don't translate over to this dataset?

I've tried restarting MT5 several times now, still nothing. Others have had similar problems but many of their solutions were fixing their formatting of the data, the order of the data, or something else I read like "synchronization issues", whatever that means...

I hope I've provided enough detail here to have someone give me some insight. It's frustrating when you try to follow the documentation and it doesn't work, worse when there's no feedback.

Thanks!

Hello, I did exaclty the same and it's not working for me either. I have a csv file of 189206 rows.  

After I completed the importing process without any issue I get this error when I try to backtest:

Tester Gold: no history data from 2023.01.01 00:00 to 2023.09.01 00:00

It's strange that they haven't solved yet.

Has anyone found any expedient?

[rupert65]

I have the same issue, would be really valuable to me if anyone knew how to fix it.

[rudysemail]

Removing the 1970.hcc file worked for me. I just did 14,500 candles. They have to be accurate though otherwise MT5 will drop them. This is what my python script looks like to ensure that

    # Set open prices to equal the former close. Then drop nan row.
    ohlc_df.open = ohlc_df.close.shift(1)
    ohlc_df = ohlc_df[1:]

    # Make sure high is highest and low is lowest 
    ohlc_df.high = ohlc_df[['open', 'high', 'low', 'close']].values.max(axis=1)
    ohlc_df.low = ohlc_df[['open', 'high', 'low', 'close']].values.min(axis=1)
    ohlc_df = ohlc_df.drop_duplicates(subset=['open', 'high', 'low', 'close'])

    assert len(ohlc_df) > 0, "\n~~~ Resample error. ~~~"

    return ohlc_df.dropna()

[Alexandru Casian]

Julian Cramer #:

Thanks, deleting 1970.hcc is fixing the problem even now. Just remember to restart the terminal after deleting the file.