Altering volume in datasets and the resultant affects on back-testing results
Guys, please forgive the following monologue:
As part of trying to prove the robustness of a bot and ensure it is not too 'fitted' to the original 1min dataset it was optimised against, I have been testing it on different data sources. In doing so I have come across a volume related problem.
The 1min dataset I've been provided with, whilst appropriately formatted in all other ways to import into MT4, has large numbers in the volume column. I am near positive that it is market depth data rather than the required MT4-style market info tick data.
Now, the bot itself does not use volume in its calculation, it makes its decisions based purely on the price action. However, from searching this forum I can see that MT4 uses the tick volume data in its processes. The problem this is causing is two-fold.
Hi Raptor. Thanks, I'll have a look at that.
Yes, I'm aware there is no market depth in MT4. I should clarify: The secondary/external data-feed that I've been given (to compare to my FXCM MT4 data), came directly from a London-based prop house. They don't use MT4. That's why I don't think the volume is tick count. I doubt FTSE ticks up/down 350 times in a 1 minute candle (for example) in any environment, MT4 or other. Hence my comment at the end about feed type. I'm happy to be wrong though.
I could ask the technical guy in the city of course but it's one of those situations where you don't want to pester someone having already had them do you a favour. D'oh.
The main question is still valid in any case I suppose: Why are there such significant differences in test results when the volume is altered? Tests 2 and 4 both have volume column populated. The difference is in the actual values. What is it that causes the change in results?
By way of an update (in case anyone is actually reading this), the whole tick volume interpolation MT4 does is a pain in the butt.
A friend of mine created a linux command to replace volume in a .csv dataset (to do the something similar to the EA created by Gordon and listed in a previous .fxt size problem thread on https://www.mql5.com/en/forum/124870), so that we could run the same test over and over on the same dataset but with different blanket volume figures in. Obviously, as described above, I'd done this manually with a setting of '4' for each 1min candle and gotten horrible results.
So I now have results for:
- the original huge volume figures
- no volume at all (but with residual numbers coming from somewhere)
- 4
- 8
- 50
-25
- 30
After 4 and 8 not reproducing the original tests results, 50 did. So I then started to wonder if it's necessary for there to be a 'tick' for every point that a candle is tall (so 25 ticks for a 25pt tall candle) in order for MT4 to use the full extremities in each case. And that by having less it can affect trades. Just a wild theory but seeing as 4 and 8 have different results and that a 1pt tall 1min candle surely doesn't need more than 4 ticks, I can only imagine it's the larger candles that somehow need 'filling'. So I reduced 50 to 25, tested again and can see results being affected very slightly. So at least I know 'full resolution' for FTSE 2007-2012 inclusive, is in between the two. A test at 30 seemed to be enough to reproduce the original results closely enough. Which is understandable since there can't be many 30+pts sized 1min candles in FTSE and frankly I can take a risk on missing out the extremities of those few that are that 'may' affect the odd trade.
To be honest I have no idea if I can trust these results or not. An idea as to technically how and why MT4 uses this tick volume would go some way to helping. If tick volume is indeed somehow linked to the number of candle points then it's feasible I could have something written in linux to account for this so that spurious volume data can be wiped out (from datasets that contain non-tick type volume), and replaced with something representative. Ho hum.
Additionally the question has to be asked, was Gordon's EA any good with its setting at 4?
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Guys, please forgive the following monologue:
As part of trying to prove the robustness of a bot and ensure it is not too 'fitted' to the original 1min dataset it was optimised against, I have been testing it on different data sources. In doing so I have come across a volume related problem.
The 1min dataset I've been provided with, whilst appropriately formatted in all other ways to import into MT4, has large numbers in the volume column. I am near positive that it is market depth data rather than the required MT4-style market info tick data.
Now, the bot itself does not use volume in its calculation, it makes its decisions based purely on the price action. However, from searching this forum I can see that MT4 uses the tick volume data in its processes. The problem this is causing is two-fold.
Firstly the .fxt file is huge (over 6gb), which curtails testing date ranges. Fine, I can work around this by testing smaller ranges in isolation. The real issue is that since this market depth volume is effectively useless to MT4 (as a representation of tick volume), I do not know if I can trust the profit/loss outcome of the testing. As a way of investigation I reimported the data without the volume box ticked* and re-ran the test (note that the .fxt was only 0.15gb this time). The difference in the profit/loss figures was significant. The same number of trades had fired, the draw-down was the same and the graph was near identical but the profit was circa 30% less. With this I decided to use a trick I'd seen on here, that is to simply alter the volume in every record to '4' and run the test once more. Again I got the same number of trades and the graph was recognisably similar but this time the profit had been truly decimated and the draw-down increased 50%!
*to confuse matters, when un-ticking the volume box in the import window some smaller numbers remained in that column. It's difficult to know what these numbers are or whether MT4 used them (given that I'd chosen the option not to import volume). I'm guessing not since the .fxt was so small in that test.
To simplify/clarify the tests I've run:
1. original dataset - provided a solid baseline, dataset provided by FXCM
2. new dataset with market depth included as volume - closest to the baseline in all ways, incl in terms of profit (although still reduced)
3. new dataset without volume - same outcome as above except profit reduced by 30% (whilst remembering that smaller numbers were appearing in the volume column for some reason)
4. new dataset with volume set to '4' - same trades/graph with profit decimated
So I'm still trying to work out why the profit/loss figures are so severely affected by differing volume entries whilst most other things (including win % and the graph) remain relatively true to one another. The first test seemed representative in all ways using spurious market depth volume and yet profit/loss was unrecognisable with the blanket '4' volume entries. This seems backward!
Having searched the forum, I can't find any answers to the odd things that are happening. It could well be that the new data came from a different type of feed (DD vs ECN etc) but I don't think that is the cause of the issues described.
Any ideas gratefully received as I'm totally lost.
Trev.