Hearst index - page 33

New comment
 
Roman.:


How cool it all turns out! :-)

Can these studies be attached (supplemented) to it or can something similar (self-sufficient for the filter) be drawn for the same simple connection to a trading owl as a trend-flat filter?

Here is my signal part of the trend owl using iVAR indicator readings.


If the theoretical studies are confirmed, there will be a whole series of indicators describing chaotic and deterministic effects.

The predictive value of iVAR seems very questionable. Visually, it really isn't any better than MA and clearly suffers from volatility. Strictly speaking, "chaotic" indicators should not react to the volatility, but to the deterministic process and the initial conditions, and they are not the same. That is, chaotic indicators should often begin to change even before the price changes (predict the change), but this is not the case with iVAR.

p.s. Better use polarized fractal efficiency indicator. It shows much more interesting moments (imho).

 
C-4:


If theoretical studies are confirmed, there will be a whole series of indicators describing chaotic and deterministic effects.

The predictive value of iVAR seems highly questionable. Visually, it really is no better than MA and clearly suffers from volatility. Strictly speaking, "chaotic" indicators should not react to volatility, but to deterministic process and initial conditions, which are not the same thing. That is, chaotic indicators should often begin to change even before the price changes (predict the change), but this is not the case with iVAR.

p.s. Better use polarized fractal efficiency indicator. It shows much more interesting moments (imho).


Thank you,C-4.

I'll have a closer look at it, I'll get to grips with it...

 

Good afternoon!

I'm currently learning R/S analysis and writing a program in C# that implements it. I have a few questions. I hope you can help me

1.In his book, Peters transforms the original time series

a) is the difference between price of the first day and price of the second day used as Yt?

b) what should be the constants a, and b?

2. and again from the book:

Here let's say a series of 500 values, I specify an initial n as 20, how do I then increase the number n???

Thanks in advance, I hope for your help.

 

b) Apparently the second book of Peters is meant and it is about eliminating the AR effects of the series. The best solution in this case would be to find the coefficients analytically for each series individually. However, I have never used such transformations, so I cannot say exactly how these coefficients should be found. But do not attach great importance to these transformations, because their meaning is to eliminate extremely insignificant overestimation of RS estimation on series like arch (I may be wrong, I remember very vaguely). Better start by working with normally distributed random walks (get stuck with this method already at this stage).

a) Better use logarithmic returns rather than series differences: ln(Pi/Pi-1).

2) Peters' N increases by one in each run: n++. The number of independent runs must be at least 2. That is, if you have a series of 500 values, divide them by 2, and you will have N varying from 20 to 250 respectively. When n will cover the series with a remainder, e.g. n = 33, 250 / 33 = 7.5757575, take the number of bars for which n will be an integer divisor, e.g. for n = 33 the number of bars will be 231. Then first calculate rs for the segment 1-231, then for the segment 20-250. The arithmetic average for both bars will be the desired RS.

P.S. Basically I can send you my code in C#. There all these transformations are already done, but the code itself is useless, because Peters's statistics doesn't show any regularities, and the results presented in his book are pure profanation (see pages 24-26 of current thread).

 

I'm sorry, but I don't quite understand under point 2)... if you don't mind sending code....

P.S. I looked at the results... don't want to bother you... but still... why is using this algorithm not acceptable to you?

 
I increased n by one (n++)... but it turned out that with n = 46, and 47 the number of periods is the same... and there was some kind of staggered graph...
 
kexs1k:

I'm sorry, but I don't quite understand under point 2)... if you don't mind sending code....

1) P.S. I looked at the results... don't want to bother you... but still... why is using this algorithm not acceptable to you?


kexs1k:
2) I increased n by one (n++)... but it turned out that at n = 46, and 47 the number of periods is the same... and it turns out some kind of stepped graph...


1) For me using this algorithm is not acceptable, because it does NOT WORK!

2) It's hard to explain all the intricacies, better see the code, it's small, I'm giving it right here:

#pragma warning disable 1587
///<summary>
/// <para>
/// Класс WealthLab.Strategies.Herst реализует методы R/S статистики, такие как таблица R/S оценки к периоду R/S расчета
/// в двойном логарифмическом масштабе, U-статистика, V-статистика, Коэффициент Херста (H) и т.д.</para>
/// <para>
/// Используется  два вида расчета: 1. по скользящему окну; 2. по разбиению всех доступных данных на независимые подпериоды
/// длинной N (задается пользователем).
/// </para>
/// <para>
/// При втором методе для использования всех доступных данных в случае, если 
/// имеется остаток (количество наблюдений не делится на размерность периода N без остатка)расчет производится
/// дважды. Первый раз рассчитываются значения для данных таким образом, что неиспользованные данные (остаток)
/// остаются в самом конце, на втором проходе неиспользованные данные (остаток) остаются в самом начале. Т.е.
/// используется метод скользящего окна, с шагом равным величене остатка. Таким образом и при первом и при втором
/// методе рассчета используются все доступные для наблюдения данные.
/// </para>
/// </>
///</summary>
#pragma warning restore 1587
using System;
using System.Collections.Generic;
using System.Linq;
using WealthLab.Indicators;

namespace WealthLab.MyIndicators
{
    /// <summary>
    /// Задает тип рассчета статистики Херста
    /// <param name="NotDependentSubperiods"> Не зависимые подпериоды</param>
    /// <param name="SlidingWindow"> Скользящее окно</param>
    /// </summary>
    public enum HerstCalculation
    {
        NotDependentSubperiods,
        SlidingWindow,
        Modern
    } ;
    public class EmptyStrategy : WealthScript
    {
        protected override void Execute()
        {
            PrintDebug("Run Empty Strategy...");
        }
    }


    public abstract class MyIndicators
    {
        protected WealthScript ws_strategy;
        protected bool InitEnable = false;
        public MyIndicators() : this (null){}
        /// <summary>
        /// Основной конструктор требует текущего окружения WealtLab через класс WealthLab.WealthLabScript
        /// </summary>
        /// <param name="wl_script">Класс типа WealthLab.WealthLabScript содержащий все необходимые данные</param>
        public MyIndicators(WealthScript wl_script)
        {
            if(wl_script == null) return;
            ws_strategy = wl_script;
        }
    }

    public class Herst : MyIndicators
    {
        //Хранит ценовой (аккумулятивный) ряд доходностей
        private List<double> series = new List<double>();
        /// <summary>
        /// Инициализиурет класс Herst текущем окружением WealthScript 
        /// </summary>
        /// <param name="ws"> Текущий класс WealthScript</param>
        
        public Herst() : this(new EmptyStrategy()){}
        public Herst(WealthScript ws) : base(ws) {}
        #region RSAnalysis
        /// <summary>
        /// Возвращает пролагорифмированное по основанию 10 отношение R/S для периода N.
        /// </summary>
        /// <param name="N"> Период расчета R/S</param>
        /// <param name="CalcType"> Тип рассчета</param>
        /// <returns>
        /// Значение RS
        /// </returns>
        /*public static DataSeries operator + (DataSeries ds1, DataSeries ds2){}*/
        public double RS(ref DataSeries data, int N, HerstCalculation CalcType)
        {
            if (N > data.Count) return -1.0;
            switch (CalcType)
            {
                case HerstCalculation.NotDependentSubperiods:
                    return RSAnalysisInLogScaleSubperiods(ref data, N);
                //case HerstCalculation.Modern(N):
                //    return RSAnalysisInLogModern(N);
                default:
                    return RSAnalysisInLogScaleSubperiods(ref data, N);
            }
        }
        
        private double RSAnalysisInLogScaleSubperiods(ref DataSeries series, int N)
        {
            //находим количество не используемых наблюдений для выбранного периода.
            int NotUsingM = series.Count - (int)(Math.Floor((double)series.Count/N))*N;
            //Создаем конвертер данных
            //Для простоты рассчетов конвертируем DataSeries в List<double>
            List<double> first = Convert.DataSeriesToList(ref series);
            //if(NotUsingM != 0){}

            List<double> second = Convert.DataSeriesToList(ref series);
            //1. Удаляем неиспользуемые наблюдения с начала списка
            first.RemoveRange(0, NotUsingM);
            //2. Затем удаляем неиспользуемые наблюдения с конца списка (0 - первый элемент)
            second.RemoveRange(second.Count - NotUsingM, NotUsingM);
            //3. Разбиваем ряд на k равных групп, и для каждой из них подсчитываем R/S размах к периоду затем
            //   находим их среднее значение.
            //Для простоты рассчета индикатора снова конвертируем списки в DataSeries
            DataSeries firstDS = Convert.ListToDataSeries(ref first);
            DataSeries secondDS = Convert.ListToDataSeries(ref second);
            double avrg_r1 = CountR(ref firstDS, N);
            double avrg_r2 = CountR(ref secondDS, N);
            double R = (avrg_r1 + avrg_r2)/2;
            return R;
        }
        private double CountR(ref DataSeries series, int N)
        {
            //DataSeries R = new DataSeries("R");
            double R = 0.0;
            int count = 0;
            for (int i = 0; i < series.Count; i++)
            {
                if ((i + 1) % N == 0)
                {
                    count++;
                    int first_element = i + 1 - N;
                    //находим среднее значение или математическое ожидание периода:
                    double sma = Indicators.SMA.Value(i, series, N);
                    //находим стандартное отклонение
                    double std_dev = Indicators.StdDev.Value(i, series, N, StdDevCalculation.Sample);
                    double acum = 0.0;
                    DataSeries acum_series = new DataSeries("Acum Series");
                    double min = double.MaxValue, max = double.MinValue;
                    for (int k = first_element; k <= i; k++)
                    {
                        acum += series[k] - sma;
                        acum_series.Add(acum, DateTime.Now);
                        if (acum < min) min = acum;
                        if (acum > max) max = acum;
                    }
                    if (std_dev == 0.0) return 0.0;
                    R += Math.Log10((max - min)/std_dev);
                }
            }
            R /= count;
            return R;
        }

        public double RSS(DataSeries series, int bar, int N)
        {
            if(bar < N) return 0.0;
            int first_element = bar + 1 - N;
            //находим среднее значение или математическое ожидание периода:
            double sma = Indicators.SMA.Value(bar, series, N);
            //находим стандартное отклонение
            double std_dev = Indicators.StdDev.Value(bar, series, N, StdDevCalculation.Sample);
            double acum = 0.0;
            DataSeries acum_series = new DataSeries("Acum Series");
            double min = double.MaxValue, max = double.MinValue;
            for (int k = first_element; k <= bar; k++)
            {
                acum += series[k] - sma;
                acum_series.Add(acum, DateTime.Now);
                if (acum < min) min = acum;
                if (acum > max) max = acum;
            }
            if (std_dev == 0.0) return 0.0;
            double RS = Math.Log10((max - min) / std_dev);
            return RS;
        }

        public double CountH(double RS, int Period)
        {
            double n = RS / (Math.Log10(Period));
            return n;
        }
        #endregion
    }
}
You cannot run it directly, because it is designed to work in conjunction with WealthLab. I am attaching appropriate libraries. With some persistence, it is possible to understand the calling methods.
Files:
wld5lib.zip  268 kb
 

As promised, I am publishing an interesting study:

Before you is a study of two time segments of the RTS Index. The first segment (green line) tests from 2006 to 2009, the second from 2009 to 2012. You can see that the RTS has changed fundamentally since 2008. The first thing that catches your eye is that the RTS has become much more efficient: its curve has become much closer to random. The degree of entropy and uncertainty has increased. However, the degree of maximum entropy, which is pure random walk, has not yet been reached. But it must be understood that at the present stage the algorithm tends to overestimate its readings on Pareto distributions and in reality the deterministic component in the modern RTS is even smaller.

The time horizon from 3 to 30 minutes is of interest (the period is 10^N min, where N is plotted on the abscissa axis in the range from 0.5 (10^0.5 = 3 min) to 3.5 (10^3.5 = 3162 min, or 52 hours or 3.7 days)). Note that there is a highly insignificant but statistically distinguishable anti-trend component in this range. It is likely to be even larger in reality, as we have inflated estimates on the Pareto distributions. It did not exist before 2009. The market was extremely trendy on all time horizons available for analysis. What is this? Perhaps we are seeing the effects of HFT algorithms that were not so widely represented before the 2008 crisis. It is not for nothing that scalpers on RTS have been saying for a long time that it is almost impossible to operate - robots do not allow trading the usual formations.

We can assume that such different results of the two segments are caused by halving the statistical sample. But the total amount of data is large and amounts to about 1 000 000 minute bars from 2006 to 2012 which gives 500 000 data points for each interval. For small periods this is tens of thousands of sub-periods, which means that the reduction in statistical significance is extremely small and cannot give such a spread of results (which is evident from the flatness of the curves). However, there is one more variant: from 2006 to 2009, the RTS market was more Pareto and tailed, now the market is more liquid and closer to the normal distribution. This is a strong counterargument, which is why I am now working hard on volatility estimation to smooth out the effects of such distributions and effectively counter the random Pareto-type wiggle.

 
C-4:


This is more about the method itself. If your method gives 0.5 on Norm. Random exactly 0,5 and 0,47-0,48 on currencies - it means that we should seriously understand your methodology. Theoretically the markets should not divide into trendy and anti-trendy. At Peters all markets studied had H above 0.5.

Have you tried to randomly mix up the series data? In theory it should destroy correlations and bring estimates of H maximally close to 0.5 (it's a question about checking the correctness of the calculation method).

Regarding anti-rewarding - after all most of currency majors, as far as I remember, have AC of neighboring returns less than 0, and if we assume that the series is a generalized Brownian motion, then C ~ 2^(H-1) - 1, i.e. H must be less than 0.5...

 

What a bloody mess!!! It's just nonsense!

Right now I'm working on mixing the data. That's what I did at first: shuffled all the RTS bars, ran a test on the shuffled ones, ended up with RS skewing even more! But it can't be like that. If RS only shows the fact that the distribution is non-normal, then the estimate should not change: after all, statistically both distributions are identical. If RS catches complex correlations, then they should be broken by data mixing, and RS should not be distinguishable from random walk. Instead the third option stubbornly falls out. Perhaps it is the complete dimensional incomparability of the RTS bars at different periods. The volatilities of 2006 are too different from recent years. As a result, the large 2006 bars creeping into the moderate range of recent years make the outliers seem to be even larger. My algorithm is written in such a way that it should effectively handle such outliers, but apparently something is breaking through the defenses.

Okay, based on the following, let us present the most negative scenario: The algorithm only catches the fact that the distribution is non-normal. There is no deterministic component in the market series. Let us check the following way: if rise of price is more likely to be followed by rise (and vice versa, if fall is followed by fall), then the graph will be flatter no matter how you turn it around. The number of reversals for N bars period will be less and therefore the number of zag-zag lines will also be less. We describe the dependence as follows: we divide the number of bars by the number of zig-zag lines and get the average number of bars per ZZ line. The distance covered is not important, so the type of distribution should not play a role. We draw a similar chart in double logarithmic scale. We take the last EURUSD history of 1 million bars as a testing tool and test it. Then let's generate a random walk on the same EURUSD volumes. It will not be normally distributed. Now compare two charts, if they are identical, it means that the algorithm only catches non-normal distribution and is of no use. So, let's have a look:

Whoa, whoa. Our worst fears have come true. Both graphs are identical. Both are Pareto distributed and indistinguishable from a random walk, but one is a true market and the other a simple generator based on real volumes.

A final check: again we take these two series and calculate the distance for them. If the indicator reacts only to non-normality of the distribution, then the swing of the normal random value should show strictly 0.5, the swing of the real eurusd will be above 0.5 (because the distribution is not normal), the swing of the generated random distribution will be almost indistinguishable from the real eurusd as seen in the chart above. Test and watch:

Holy shit!!! Did you see that!? The random EURUSD is just the tiniest bit higher than the normal random ramp. Its slope is almost equal to 0.5. The random value has remained unchanged as expected. But EURUSD has seriously underestimated(!) its range! At the same time, the Hurst index of eurusd is as much as 0.53, i.e. the currency is trending, but we see only the beginning of its trend. Its trend gathers strength only after 70 days(!), and before that it is a strongly pullback market. This means that the EURUSD has a gigantic horizon, we only see the very beginning of the prologue.

Apparently it is the non-normality of the distribution that hides all the deterministic effects. However, the main abnormality is not so much in the strange volatility, but in the price increments (bars) themselves. It is a very complex and deep mechanism and there is no way to work with it at random.

s.s. though maybe it's all wrong conclusions because of wrong data and methods. hh.

1...262728293031323334353637383940...46
New comment
Reason: