there is meanwhile a growing community for advanced cycle analysis of financial time series that I decided to open a new thread to bundle these discussions a bit.
Next to the wellknown thread Fast Fourier Transform - Cycle Extraction we have meanwhile also a lot of Goertzel Browser implementations predominantly published in the Advanced Elite thread Requests & Ideas. According to Dennis Meyers' paper “MESA vs. Goertzel” the Goertzel browser seems even more important than Ehlers' MESA Cycle Analysis.
In the following you find the links to the various Goertzel Browser versions predominantly provided by mladen:
Goertzel Browser end-point based on Crodzilla’s Original version
Goertzel Browser v1
Goertzel Browser v2
Goertzel Browser v2.1
Goertzel Browser v2 end-point
Goertzel Browser v2.1 end-point
Goertzel Browser v3
Goertzel Browser v3 mtf
Goertzel Browser v4
Goertzel Browser v5.5
Goertzel Browser v5.51
Goertzel Browser v5.52
Goertzel Browser v5.53
In all these versions the compound wave of the analyzed cycles was still not satisfying considering its trend prediction capability. In several publications the implementation of a cycle significance test was promising a solution to this problem, mentioning as most prominent test the Bartels Test for Measuring Cycle Significance developed by Julius Bartels. See e.g.
- Article “Cycle Analysis Explained” by Alex Bernal
- Book “Decoding the Hidden Market Rhythm” by Lars von Thienen
- Description Cycle Trends
Bartels created several different tests, so please emphasize “Measuring Cycle Significance” when searching in Google. The Bartles Test allows a significance ranking of analyzed dominant cycles in time series (e.g. generated with Goertzel, Hurst, etc.) . The most important paper with a practical calculation description was provided by Charles E. Armstrong in his paper “Cycle Analysis – A Case Study”. But beware: there is a typo in Table III; instead of A**2 – B**2 the correct calc. is A**2 + B**2. Another good description of the Bartels Test can be found in “Analyzing and Forecasting Future Prices” from Antony F. Herbst.
I will provide a first implementation of the Bartels Test based on the Goertzel Browser 5.5 later in this thread.
Empirical Mode Decomposition (EMD)
I discovered a great page about an MQL5 indicator having even the capability to replace the Goertzel Browser:
Introduction to the Empirical Mode Decomposition (EMD) Method
The EMD algorithm can be used for various purposes in financial time series analysis:
- calculating the instantaneous significant cycle frequency spectrum
- creating a smoothed moving average
- detrending and smoothing
It would be great to also have an mql4 version of the EMD algo, e.g. also an EMD Browser - but this is right now beyond my capabilities.
Log Periodic Cycle Analysis
A real predictive approach to evaluate the change points of bubble or crash regimes due to the log-periodic behavior of time series is provided by Didier Sornette.
An actual application is here: The Anatomy Of A Pre-Crash Bubble ;
The basic paper of his model is „Critical Market Crashes“ ;
A great proprietary software implementation under the name Extreme Hurst is available from Parallax Financial Research
An mql4 version of the indi would be gorgeous !
Scientific more detailed papers are here:
- Detection of Crashes and Rebounds in Major Equity Markets ;
- Dragon-Kings, Black Swans and the Prediction of Crises ;
- The Illusion of the Perpetual Money Machine ;
- Clariﬁcations to Questions and Criticisms on the Johansen-Ledoit-Sornette Financial Bubble Model;
- Everything You Always Wanted to Know about Log Periodic Power Laws for Bubble Modelling but Were Afraid to Ask;
When the new metatrader 4 start to live normally, there will be no problem at all to use EMD in that new metarader 4 - since it is going t be capable of using classes (without classes capability it would be a Sisyphus job to convert it). So it is just a matter of some patience and it will be available for metatrader 4 too
PS: thanks for the thread (and everything posted in it). I enjoy it
Goertzel Browser with Bartels Cycle Significance Test filter
Now it’s coming, the modified Goertzel Browser v5.5 with the Bartels Significant Cycle test.
Here are some details and explanations of the mod:
- I added the Centered MA choice as a smoothing possibility for the original time series before applying Goertzel cycle extraction. Reason is pretty simple: achieving noise reduction. So far in the Goertzel Bowser versions the Hodrick Prescott Filter was only applied for detrending the series. The Centered MA has by definition no lag, but as a tribute it is repainting a bit over the latest few bars. This seems to be acceptable/neglectable compared to the considered cycle window/lengths. Choice 1 is the Hodrick Prescott Filter (I didn’t succeed to use the built-in HP algo, so you have to add the attached HP indi), as choice 2 I added the Absolutly No Lag LWMA of mladen (you have to add it as well to the indi folder). It seems the repainting period is less in the ZeroLag MA. Results are impressive: the composite cycle is much clearer than the original one.
- In the DPO detrending function I added the Absolutly No Lag LWMA next to the HP algo. Further I added a special detrending method using a linear regression of the log transformed smoothed time series data according to “Practical Implementation of Neural Network based time series”. I personally like this approach.
- Bartels test was added to filter the Goertzel waves according to their significance. Implementation was done using the following conditions:
o Bartels test is applied to every extracted Goertzel cycle length
o Native time series data are taken from a window with the significance length of 5 cycle lengths (default). They are smoothed and detrended over the significance length with the linear regression of log transformed smoothed Absolutly No Lag LWMA data as described above.
o Obtained native Bartels values have the following meaning: Value -> 0 significant; value -> 1 not significant. For an easier understanding I introduced percentage levels by inversing the counting and multiplying the values by 100 (as recommended in “Decoding the Hidden Market Rhythm”, p.45). Result: A low value implies a low statistical alignment between cycle and real data and vice versa. Threshold for significance can be selected in the indi (default is 40%).
o If Bartels filter is activated, Goertzel waves below the Bartels significance threshold value are not considered in the composite wave and the table. Remaining Goertzel waves are ranked by Amplitude or Cycle Strength but can be ranked by Bartels value as well. Ranking by Cycle Strength is recommended to select the most dominant waves for the composite wave.
- The table shown in the sub window was modified, adding the Bartels values and indicating whether Amplitudes or Cycle Strengths are selected as proposed in “Decoding the Hidden Market Rhythm”). Cycle Strength is already implemented in the original 5.5 version:
Cycle Strength = Cycle Amplitude/ Cycle Length
IMPORTANT: Next to the Goertzel Browser 5.5Bartels there MUST be further added the attached the indis HP and "absolutely no lag lwma" to indi folder !
Some open issues are existing:
- The MTF capability is not working yet. I couldn't fix it.
- I didn't succeed to use the built-in Hodrick Prescott algo for smoothing the native time series values
- I couldn't embed the "absolutely no lag lwma" algo as function in the Goertzel Browser.
In the image below the Goertzel Browser 5.5Bartels is in the 2nd sub window. It has an HP smoothing, no detrending, Bartels Filter enabled, Cycle Strength enabled and no noise reduction. Similar is the reference Goertzel Browser 5.5 in the 1st sub window.
Enjoy testing. I appreciate further improvements.
A very nice indicator is also Richcap’s R-MESA-Instant_Spectrum v.1.2 of course. You have to add the R-MESA library to the MT4 “libraries” folder to make the first one working. Unfortunately the extracted Goertzel periods are not really confirmed by the extracted MESA periods. Recommendation is to restrict max. periods to 150-200.
Recommendation Settings Goertzel Browser 5.5 Bartels
Hi there, here are some setting recommendations:
- use all filtered significant Goertzel cycles (Bartels value >40%) for the composite wave. Limitations to the most dominant waves are distorting highs and lows in my view.
- try to extend MaxCycleLength to e.g. 300 with WindowSizePast at least 5 x MaxCycleLength (here 1500). This allows you to catch the longer trends as well.
- use always Cycle Strength instead of Amplitude. this allows you a better evaluation of the most influential waves.
Youdaman, Boxter. I had been experimenting with various mtf and period settings to find coincident agreement for entry and exit. I have found that (1) trend change from dominant period from one Goertzel instance backed up by (2) high probability Bartels Goertzel confirmation of direction is better than any other combination I had tried. Great stuff. This would be great to incorporate into an include file and/or dll as solicited here. Perhaps Mladen can get to it when the 574++ dust settles? Thanks!
Hi JahMon, I am currently updating the Bartels version having all indis built in and removing some bugs I identified with the new MT4 editor. I still have problems with mtf functionality. I am not going for a dll right now. Updated version will come soon.
Goertzel Browser 5.6 incl. Bartles Cycle Significance test and centered MA smoothing
Hi all, I think I fixed now everything . Attached is the updated Goertzel Browser 5.6. I selected a new version no. because of the various features compared to version 5.53, despite it doesn't contain the nice "PeakSpan" parameter. New/Fixed features of the Browser are now:
- fixed several coding bugs with new MT4 editor
- MTF is working now
- SampleSize now = BartNoCycles*MaxCycLength+BarToCalculate; WindowSizePast: Must be >= SampleSize
- Automated individual ListOfCyclesID with ChartID
- Smoothing possibility of original time series data with built-in Hodrick Prescott filter or Absolutly No Lag LWMA - no additional indis necessary anymore !
- in detrending function Absolutly No Lag LWMA was added as alternative to HP algo - no additional indi necessary anymore !
- added further detrending method using a linear regression of the log transformed smoothed time series data according to “Practical Implementation of Neural Network based time series” blog. It normally shows no significant change to no-detrending data if used with Bartels.
- Bartels cycle significance test for every extracted Goertzel cycle.
- modified value table adding Bartels values
it is recommended
- to use Bartels value >= 50
- # cycles for significance = 5
- if Bartels applied, use cycles with cycles strengths >1
- use cosine waves only
Practical usage must be still verified. At least the composite wave is still dynamically adapting and not really providing fixed turning point forecasts in my (limited) tests so far. There are proposals to use the Browser with DPO or ROC indicator using half of the dominant cycle period. Dominant cycle is defined as significant cycle with largest cycle strength.
An end-pointed version would be interesting for test purposes. mladen - I removed index buffer from sample values used for Goertzel calc., allowing me to restrict the data to the sample size. This step provided more stable switching between different smoothing and detrending methods.
Fixed Goertzel Browser 5.6
Hi there, I still discovered a bug in the Detrend Regression function - sorry. Attached is the fixed version 5.6.