Optimizing Strategy Tester Backtest Performance
I am running a large amount of back tests. I have been keeping a few laptops busy most of the time day in and day out. I have found a few issues that hopefully someone here can help me with. I run a EA test with M5 charts for 1 year timeframe and the tests are taking about 20 minutes per setting using the "Control Points" modeling method. often I want to optimize more than one variable at a time as they are inter related. What I find so far is that when I am able to increase the timeframe to M15 the test typically runs about 1 to 2 minutes or 10 to 20 times as fast even though the bar length is only 3 times the size. Likewise when I run the test with H1 charts it runs in about 5 seconds per instance per year. As I increase to 6 years in H1 charts the test runs about 4 times slower (5 seconds for 1 year, 2 minutes for 6 years) and increasing to 16 years slows the test to 20 minutes per instance. It seems the test time increases exponentially with the size (Number of bars) in the test sample.
Is there something I can do to get a larger history sample in RAM so it will not bog down the test engine when I run a longer test? My laptops have 8 to 12 GB of RAM and those with 12 GB are not running faster.
If it is taking that long even with Control Points, then the EA has some major coding issues. It is most probably doing everything repeatedly on every single tick, instead of separating things into priorities.
For example, updating indicator values should be updated once per bar and not on every tick!
Also many calculations and Array values and function returns can be cached in variables instead of calculated and accessed and returned on every single tick.
If it is taking that long even with Control Points, then the EA has some major coding issues. It is most probably doing everything repeatedly on every single tick, instead of separating things into priorities.
For example, updating indicator values should be updated once per bar and not on every tick!
Also many calculations and Array values and function returns can be cached in variables instead of calculated and accessed and returned on every single tick.
I think have all my values calculated one time per bar with exception of the new tick being compared to the trigger prices on tick. That is how the code was specified. Exits are by SL and TP with exception of a close all open trades function that I trigger on the 8300th minute of the week.
My question is really regarding the fact that increasing the data from one year to 15 years is making the EA back test about 200 times as slow where there is only 15 times as much data to test.
Is this because the system tester is paging through the data? I think so.
Is there a way to load the data into RAM so it gets looked at efficiently rather than by referring to slower memory sources like potentially the hard disk being referenced thousands of times?
I have no Idea as I did not post it! I don't even know how it got there! I will delete it! I must of have been accidental but don't know how on earth it happened!
This is very strange! That image is not even mine! I don't have any trades on any of my accounts today nor do I have any account with that balance!
This is intriguing me!
As you can imagine, something is wrong as my original post to you was only in text and had no image. The system must of messed up and switch contents with someone else.
My original post was and I quote:
As can be seen in the following screenshot, if I do a search it reports it correctly but somehow the system has placed an image there on my post instead:
Is this a shameless plug for BBSO Genuity Scalper, Fernando?
I don't even know the existence of BBSO. I did not post it and the website's search correctly reports my original post! That is why I have already PM two Moderators to look into it, but I don't know who is online right now to look into it.
I am assuming that the site has switched the contents with someone's post for some obscure "buggy" reason!
I promise, I did not post that image!
I'm just pulling your leg Fernando.
And the OP is a consummate gentlemen with their response 
I'll agree it is a highly unusual glitch.
I'm just pulling your leg Fernando.
And the OP is a consummate gentlemen with their response
I'll agree it is a highly unusual glitch.
Thanks for pointing out the source of the problem. The blog post of the author of that EA did definitely post that image on his blog. What I don't understand is how the site pushed it onto my post.
Suggestions? Should delete it or wait until a moderator has been able to look into it?
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I am running a large amount of back tests. I have been keeping a few laptops busy most of the time day in and day out. I have found a few issues that hopefully someone here can help me with. I run a EA test with M5 charts for 1 year timeframe and the tests are taking about 20 minutes per setting using the "Control Points" modeling method. often I want to optimize more than one variable at a time as they are inter related. What I find so far is that when I am able to increase the timeframe to M15 the test typically runs about 1 to 2 minutes or 10 to 20 times as fast even though the bar length is only 3 times the size. Likewise when I run the test with H1 charts it runs in about 5 seconds per instance per year. As I increase to 6 years in H1 charts the test runs about 4 times slower (5 seconds for 1 year, 2 minutes for 6 years) and increasing to 16 years slows the test to 20 minutes per instance. It seems the test time increases exponentially with the size (Number of bars) in the test sample.
Is there something I can do to get a larger history sample in RAM so it will not bog down the test engine when I run a longer test? My laptops have 8 to 12 GB of RAM and those with 12 GB are not running faster.