Back testing has slowed down, help!
As far as I know, running out of RAM is worse than maxing out the CPU. When the CPU hits 100%, things might slow down, but the system usually keeps running.
If you run out of RAM, your screen can freeze completely and the system may become unresponsive.
With the amount of RAM you have, you'd rarely run into memory-related freezing — unless you’re doing some crazy multitasking or there's a memory leak in play.i9 14900 28 core intel
128g DDR5 ram
3080 graphics card (if that even matters)
Jeff
As a general rule, I delete my tick data and history after every few new versions of mt4/5 and also after forced cumulative, windows updates. Both of these events often break or fragment the data files. It may not break them, however, these events almost always makes accessing the data, noticably slower. And it is always possible that windows may defragment some during automatic harddrive maintenance/garbage collection.
But also the 14900 is 1 cpu that has a bad rep and many of them have an inherent instability problem. you only have to do a google search aand find that intel and microsoft had several lawsuites regarding both 13 and 14gen cpu manufacturing flaws found after selling their stocks; against them during last year. They were unsuccessful, but then, goliath does often win battles whatever the battle ground.
you could alwasy delete the cache files in your Agent(s) folders, however, that will also force you to lose your place in any tests that you had stopped/paused. You could also move these files to a different folder and then delete the original files, and then copy the files back to their original location. This could defragment them; assuming that you have enuf space on your harddrive without having to break the files into pices and place them all over your drive in smaller pieces. In which case a defragment with the defragger will almost certain to make a noticable speed increase; however, this process may take "over night" to defragment these files.
As a general rule, I delete my tick data and history after every few new versions of mt4/5 and also after forced cumulative, windows updates. Both of these events often break or fragment the data files. It may not break them, however, these events almost always makes accessing the data, noticably slower. And it is always possible that windows may defragment some during automatic harddrive maintenance/garbage collection.
But also the 14900 is 1 cpu that has a bad rep and many of them have an inherent instability problem. you only have to do a google search aand find that intel and microsoft had several lawsuites regarding both 13 and 14gen cpu manufacturing flaws found after selling their stocks; against them during last year. They were unsuccessful, but then, goliath does often win battles whatever the battle ground.
you could alwasy delete the cache files in your Agent(s) folders, however, that will also force you to lose your place in any tests that you had stopped/paused. You could also move these files to a different folder and then delete the original files, and then copy the files back to their original location. This could defragment them; assuming that you have enuf space on your harddrive without having to break the files into pices and place them all over your drive in smaller pieces. In which case a defragment with the defragger will almost certain to make a noticable speed increase; however, this process may take "over night" to defragment these files.
Rad that is exactly the type of idea i was looking for TY! I will try that and let you know. I think I have 1.2 tb left on my drive or something similar its at roughly 40%
I though maybe my cores were over heating or what not, but they are not.
Jeff
With the higher end of 13 and 14gen cpus, they are much more prone to throttling than previous gen cpus, so just because there is no "obvious" overheating, does NOT mean that they are not "heating up". That is why the throttle in the first place, so that they dont get that hot that they may "overheat". I recommend that you do a trial of a little, free program called "ParkControl". It simply forces the cpus to not park and has option to stop or limit throttling. If your cpu dont get any hotter, then, you will also notice a speed increase as well as higher clock rates. If they do get to an obvious "overheated", then, you will know that you need to either lower your temps by decreasing your power, or getting better heatsink and/or extra fans. Note that heating up, may not be noticable due to throttling, and this also has result of lowering the clock rates, including memory clocks, thereby slowing the whole system down. This could be your issue. In this instance i think it is more likely a defrag or corrupted files issue, however, this is something you can try; to get your system into more optimal shape.
i9 14900 28 core intel
128g DDR5 ram
3080 graphics card (if that even matters)
The Intel i9-14900 CPU has 24 cores and 32 threads. 128GB of RAM is a good amount for this usage scenario. You didn't say what type of storage you're using for Windows, MetaTrader 5, and any system paging files (preferably a fast NVMe rather than a SATA SSD, or worse yet, a HDD). Graphics card only matters if your EA makes use of OpenCL calculations.
Regarding Michael's thoughts on throttling and overheating, note that "ParkControl" can make changes that significantly increase the amount of power your computer uses at idle (where it spends most of its time when on) if you don't know what you're doing. I would recommend using HWiNFO64 (free) to observe your CPU core speeds and temperature sensors during backtesting sessions before making changes. On the Summary page, clicking in the little CPU window with the core bars toggles it between actual and effective core clock speeds. "Effective" speed takes load and throttling into account (helpful to see how load/speed is being distributed between cores). The numbers flash red beside cores that are thermal-throttling.
Note that Intel has released microcode updates for this CPU that can cause significant performance losses on certain boards/configurations. Thermal paste goes bad too. You might want to run some benchmarks just to make sure your system scores in the ballpark of where it should be scoring with that CPU. For example, an adequately cooled, properly configured Intel i9-14900 should be scoring around 34k and an i9-14900K around 38k in Cinebench R23.
The Intel i9-14900 CPU has 24 cores and 32 threads. 128GB of RAM is a good amount for this usage scenario. You didn't say what type of storage you're using for Windows, MetaTrader 5, and any system paging files (preferably a fast NVMe rather than a SATA SSD, or worse yet, a HDD). Graphics card only matters if your EA makes use of OpenCL calculations.
Regarding Michael's thoughts on throttling and overheating, note that "ParkControl" can make changes that significantly increase the amount of power your computer uses at idle (where it spends most of its time when on) if you don't know what you're doing. I would recommend using HWiNFO64 (free) to observe your CPU core speeds and temperature sensors during backtesting sessions before making changes. On the Summary page, clicking in the little CPU window with the core bars toggles it between actual and effective core clock speeds. "Effective" speed takes load and throttling into account (helpful to see how load/speed is being distributed between cores). The numbers flash red beside cores that are thermal-throttling.
Note that Intel has released microcode updates for this CPU that can cause significant performance losses on certain boards/configurations. Thermal paste goes bad too. You might want to run some benchmarks just to make sure your system scores in the ballpark of where it should be scoring with that CPU. For example, an adequately cooled, properly configured Intel i9-14900 should be scoring around 34k and an i9-14900K around 38k in Cinebench R23.
Those benchmarking and temp monitoring softwares create the problems that they purport to avoid. Dont waste your time or put your system under such stresses. As i alluded to previously, if ParkControl creates higher temps, then, you know you have an issue that without ParkControl, you didnt know about previously. If extra obvious heat is detected, then, you need more air and ventilation. This will result in both better temps and higher clocks and cpu frequencies.
Those benchmarking and temp monitoring softwares create the problems that they purport to avoid. Dont waste your time or put your system under such stresses. As i alluded to previously, if ParkControl creates higher temps, then, you know you have an issue that without ParkControl, you didnt know about previously. If extra obvious heat is detected, then, you need more air and ventilation. This will result in both better temps and higher clocks and cpu frequencies.
I went easy on you the first time because one doesn't know what they don't know. But since you contradicted my redirect, I will now respond directly. As an experienced system builder, computer repair technician, and overclocker, I could not disagree more strongly: You have absolutely no idea what you're talking about when it comes to CPU power management or throttling! The ONLY thing ParkControl would be good for in this context would be correcting incorrect Power Profile settings that are sending processes to the E-cores (these settings are not only hidden by default, but they can only be accessed through the also hidden Control Panel's Power Options applet otherwise as the newfangled options in the Settings app are superfluous). The rest of its settings will just cater to the placebo effect while potentially increasing your idle system power consumption with no measurable improvement in performance (except maybe latency, which is really only helpful to audio engineers doing realtime DSP/mixing/multitrack recording through their PC, where loads are low enough that the CPU can incorrectly think it's idle and slow down/park cores, causing little glitches with some interfaces).
With the higher end of 13 and 14gen cpus, they are much more prone to throttling than previous gen cpus, so just because there is no "obvious" overheating, does NOT mean that they are not "heating up". That is why the throttle in the first place, so that they dont get that hot that they may "overheat". I recommend that you do a trial of a little, free program called "ParkControl". It simply forces the cpus to not park and has option to stop or limit throttling. If your cpu dont get any hotter, then, you will also notice a speed increase as well as higher clock rates. If they do get to an obvious "overheated", then, you will know that you need to either lower your temps by decreasing your power, or getting better heatsink and/or extra fans. Note that heating up, may not be noticable due to throttling, and this also has result of lowering the clock rates, including memory clocks, thereby slowing the whole system down. This could be your issue. In this instance i think it is more likely a defrag or corrupted files issue, however, this is something you can try; to get your system into more optimal shape.
- ParkControl cannot stop thermal throttling from happening. If it could, it would quickly damage the CPU (but it can't, because it lacks the settings to modify the CPU's built-in thermal protection parameters. If you like living dangerously, Intel XTU can change these—not recommended!).
- ParkControl cannot change the Intel Turbo Boost parameters that control sub-throttle temperature clock boosts. If it could, it would quickly cause system instability (but it can't, because it lacks the settings to modify the CPU's built-in thermal speed boost parameters. Once again, Intel XTU can change these, but if you're an overclocker, it is recommended to change these in the UEFI instead).
- ParkControl cannot stop the CPU from throttling back due to dips in voltage. If it could, it would quickly cause the system to freeze (but it can't, because it lacks the appropriate CPU-specific setting to disable this feature).
- ParkControl cannot stop the CPU from scaling back frequency due to hitting the PL1, PL2, or IccMax limits. If it could, it would be a CPU unlocking tool, but it lacks any such settings.
- ParkControl cannot stop the CPU from scaling back frequency due to the number of cores loaded. If it could, it would be an overclocking tool, but it lacks any such settings. You probably don't even know what those settings look like (because you would've never posted this if you were familiar with any of these).
Apart from these five things (which ParkControl has no control over), the Intel CPUs do not throttle their speed back arbitrarily. There is nothing that ParkControl can do to for the scenario you mentioned, making the CPU "stop throttling" so it will "heat up". The core parking settings it can adjust ONLY affect the CPU's idle response and thresholds; they do not control its loaded behavior. And the frequency scaling settings it can adjust are normally always at 100% on desktop processors by default, having nothing to do with thermal throttling (although, if somehow changed manually, they could reduce the CPU's performance from defaults).
In my previous post, I recommended two pieces of software. One which will allow the user to see what the CPU is actually doing in realtime, as well as observe various system temperatures and identify the source of any throttling that might be happening. The other will run the CPU through a quick calculation and generate a score which can be used to compare your system's performance with others. If other people are getting a score of 34k out of their i9-14900 and you get 21k, something is wrong with your system that warrants further investigation. But if it's on par, and the MetaTester loads are being distributed evenly (and to the P-cores first), then we can rule out CPU issues and focus on what MetaTrader 5 is doing.
Your suggestions to try clearing the agent cache folder and defragmenting were good.
wow. i was not contradicting you in my previous response, so there was no need for your obvious emotional response.
I am system builder and in windows systems engineer level employment, going on 17 years now, so i do know what i am chatting about; and you had no idea of my experience so i can accept your ignorance of that fact, therefore I will not respond in similar manner as you have done.
I never said ParkControl can do any of the things that you have claimed that I had said. Using ParkControl just removes some of the limitations that the power profiles make which may be limiting the cpu and memory in a backtest/optimisation task. And as i said if it gets obviiously hotter, then, you need more air, or if nothing happens, then, good, then, using ParkControl will only be good in that event.
I only read part of your response because i dont have time to waste on such things.
And I appolgise to the op for not directing any response to the starter of the thread as all comments should be.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
i9 14900 28 core intel
128g DDR5 ram
3080 graphics card (if that even matters)
So i have been doing ALOT of back testing with my new EA. It has been getting slower and slower and now it is common for it to lock up and freeze my entire computer when performing fast genetic optimization in ticks or real ticks for more then 1 year at a time. At first I thought I was just getting impatient, but then it started to freeze the computer.
I though maybe my cores were over heating or what not, but they are not.
Today, my buddy wanted me to help install said EA on his account, I have some stable settings I use for my broker but I told him we should check the settings with his broker since it likely will have slightly different performance with different data etc etc
So i downloaded a new instance of MT5 using his broker, and low and behold it is back testing super fast again. I can run fast genetic optimization runs in real ticks 2-3 years at time with very fast results.
I am suspecting maybe there is some sort of folder that is getting filled up with previous back test data on the original instance of MT5. Any ideas why this is happening or how to correct it?
Thank you so much in advance for any help
Jeff