The world is changing, I don't how to import and convert the data now! - page 2

 

Currently,  this is the critical bottleneck I met. 

How to import the DukasCopy M1 data into the latest version of MT4 without weekend data, convert it to other Periods, and get backtest quality improved over 90%?  

 
Hi Jollydragon,

 

To answer your questions: 

> This is just a short-term solution.

 This solution would be still effective if you used it a year from now.  MT4 don't often change the logic surrounding trading/back-testing so unless there's some major changes and/or bug fixes, there is no major impetus to update your (back-testing) MT4 build. 

Besides, how long can we still get support on TickStory v1.5 in future? 

How long would you work for free?  It's a tough question to answer but suffice to say that we have been proudly supporting the trading community for free for a number of years and continue to do so. 

> I am new to Forex and havn't worked out a good strategy yet. Before I achieve it, I'm afraid I can only make use of all free sources I can get.  

I understand what you are saying, however I don't see the connection with Tickstory.  If you can use the software for free right now, what exactly is the down-side?

 

Anyway, just thought I'd throw in a different perspective since the topic title implied there was no hope ;) 

Regards. 

 
tickstory:

 

Dear Sir, appreciate your support anyway!

 
You're welcome, Jollydragon - good luck!
 
jollydragon:

 I downloaded EurUsd M1 data from DukasCopy.

If using the script, CSV2FXT, to import and convert the data, the backtest quality is 'na' and there are weekend data in charts.
Then, how to import the DukasCopy M1 data into the latest version of MT4 without weekend, convert it to other Periods, and get backtest quality improved over 90%?   

Going back to your original post:

1) The DK data I've downloaded (for the Majors) does not have GMT weekend data apart from  a couple of hours at the end of each sunday. What symbol & timezone are you using ?

2) The quality reported is just a number embeded in the tick data file (FXT). It's nothing special. I suspect MQ have changed the way the on-screen quality is computed, and the computed result is displayed rather than the one directly from the FXT file.

Only with external tick data, the terminal doesnt get a chance to compute the 'quality'  - so n/a is displayed.

Just ignore the quality setting, the one in Birts scripts is hardcoded to 99 anyway.  

 
ydrol:

Going back to your original post:

1) The DK data I've downloaded (for the Majors) does not have GMT weekend data apart from  a couple of hours at the end of each sunday. What symbol & timezone are you using ?

2) The quality reported is just a number embeded in the tick data file (FXT). It's nothing special. I suspect MQ have changed the way the on-screen quality is computed, and the computed result is displayed rather than the one directly from the FXT file.

Only with external tick data, the terminal doesnt get a chance to compute the 'quality'  - so n/a is displayed.

Just ignore the quality setting, the one in Birts scripts is hardcoded to 99 anyway.  

 


 

 

Dear Ydrol,

1) I downloaded EurUsd M1 data from Dukascopy website and found the weekend data after converting to H1. 

2) Yes, I'm confused seeing the quality is 99.9% as finding there is only 1 tick for M5, M15 with TickStory.

3) After converting by Birts script, the quality is 'n/a' and I don't know if I can use the data and script.  

 

Hi,

 I downloaded EurUsd M1 data from Dukascopy website and found the weekend data after converting to H1. 

1. Dukascopy do not have weekend GMT tick data to download. Apart from the last couple of hours on Sunday. Tickstory will tell you the same.  

 jollydragonYes, I'm confused seeing the quality is 99.9%  

2. The quality is just a number injected into the tick file (FXT file) by whomever created the ticks. MT4 may compute this from the History when creating fake ticks, but for FXT generated outside of MT4 (eg TDS, Tickstory, whatever) the examples I've seen are hardcoded so they just gives people a warm feeling to see it. What is important is that :

- you have all of the ticks in the FXT (./tester/history/EURUSD1_0.FXT) ,

- the 'mini' candles in the FXT corresponds to to candles your history files (./history/<BrokerServer>/EURUSD*.HST). (Tickstory, TDS will do this)

- And that your backtest runs and picks up all of the ticks (check the tick count).  

- you don't connect to your broker and receive live ticks on the MT4 install that you are using for back testing.  - Untick/Clear Tools->Options->Server->Keep Personal Settings and Data at startup.

 

Think about this. Why is TickStory quality always 99.9% , and TDS quality (was) always 99% (not 99.9%) even when

a) they use the same data,

b) there are occasional holes in Dukascopy data?

I'll tell you :) Because, unless using MT4 modelled ticks, the quality is simply a number injected by whomever created the tick file. Birt is happy to mention this on his blog. Ask Tickstory  exactly how is the quality computed? I'm making assumptions on how they always hit 99.9% based on what I've seen in the FXT file format, and on Birt's blog and responses in TickStory forums. The assumption is there is no intelligence or computation behind the quality. It's simply a flag that visually lets you know that MT4 is reading the 'real' tick file and not its modelled one.

jollydragon as finding there is only 1 tick for M5, M15 with TickStory.

If you are only seeing one tick for TickStory contact their support forum. Also see point 1. Also check you are running EA with every tick option.

jollydragon After converting by Birts script, the quality is 'n/a' and I don't know if I can use the data and script. 

 Ask Birt on his forum. But if you run a back test, and can see the final tick count is significantly bigger than MT4 fake tick count, then you are getting your ticks from the external tick data.

The quality inside the FXT file is exactly the same. Always 99 for Birt, Always 99.9 for Tickstory. What I think has changed is that the terminal displays, on screen, the quality it has computed internally, rather than the quality it has loaded from the FXT file (after creating it).

The quality you want to see is really just a  'dumb' number, inserted into the FXT file, for both Tickstory and TDS. That might change one day to account for gaps in the source data.

MQ have changed the terminal to display the quality it computes when creating the FXT, rather than the quality it extracted from the FXT. (I suspect)

Because TDS, TickStory, etc prevent the terminal generating creating a new FXT, the terminal doesn't compute an internal quality, and displays n/a.

 But the tick data is still the same and still as good (or bad) as it was before.

That is my understanding after writing my own scripts to download tickdata and generate FXT(tick), HST(history) files. 

UPDATE1:  I've just verified the Modelling quality behaviour. 

If I create a new demo account and download history via History Center , and run an EA using 'Every Tick' Model - I get 'Modelling quality = 25%'

If I then make the same MT4 generated tick file read only (so MT cant overwrite it, same technique used by TDS, TickStory etc), and run the EA again with no other changes, I get 'Modelling quality = n/a' - but everything else works the same.

This is using MT's same tick file that got 25% quality, just seconds before. Nothing to do with TickStory, TDS, or DukasCopy. 

TickStory 1.6 has probably gone to some extra lengths to make the terminal display their 99.9% number. But 'n/a' is just fine if the ticks are there, and match the history file.

UPDATE2:

Base on screenshot below it is simply the TickStory launcher that has problems with new builds of MT, because they hardcoded the MT build numbers into their launcher. (A guess) I wouldn't be surprised if a small tweak could get TS 1.5 launching for new MT. The quality will be n/a :)

 
ydrol:

Hi,


Dear Ydrol,

 

Thanks for your comment. What is TDS? Is it another free software for data downloading, importing and converting? Where to download?

 
tickstory:
You're welcome, Jollydragon - good luck!


Please refer to the screenshot below. 

After revert MT4 and empty the folders as mentioned on TickStory forum, I can back test successfully yesterday.

However, the error comes out again today. 

 

 

PS I see Birt has already answered a week ago re TDS, on his forum - and even said what TDS is? And tickstory have already answered you on thier forum regarding their launcher issue?  Why are you asking all the same questions again here? It doesnt look good!  

I'm interested in backtesting and data generation, but it seems you are just asking the same questions all over the place? And you are not even interested in tick data? just H1?  TickStory and TickDataSuite are for getting ticks

If you want higher timeframes just delete all your history, download M1 data, and import into CSV.  Forget Tickstory, Forget TDS Just let MT make up ticks, as you don't seem to care about ticks much. Of course your back testing wont be accurate at all until you realise you need ticks.

PPS Your TS issue is probably because MT updated itself. Please discuss with TS on their forum. Oh looks like you started already! Menu->Help->About Let TS know on their own forum if their instructions to prevent updates are wrong.

From Birt  "If you want 90% backtesting, simply use the Metaquotes download center, it’s decent if you’re not looking for high precision." Decide if you want tick data or not.  

Reason: