How to fix a bug in Darch code? - Articles, Library comments

lve0200 2017.01.06 16:58 #31

Hi All,

OK, i've got everything to run.

1: make sure, you have darch version 0.10.0 !! How to do this, see post #29

2: make sure, you have enough historical data on the symbol to want. Open a chart of this symbol in MT4 and control how much historical data you have load.

3: @ Vladimir. IMHO there is still a bug in your code: (both Versions)

if(RGetBool(hR,"first")) lim = limit;  <=== lim = limit
      // Resize arrays
      ArrayResize(o, lim);
      ArrayResize(hi, lim);
      ArrayResize(lo, lim);
      ArrayResize(clo, lim);
      // Fill the array with new data
      for(i = 0; i < lim; i++)    <===  i < lim, BUT!!
        {
         o[i]  = Open[i+1];  <== i+1
         hi[i] = High[i+1];
         lo[i] = Low[i+1];
         clo[i]= Close[i+1];

This leads to a boundary error! If limit is 5000 i = 0 .. 4999 , i+1 = 5000 hence 0 .. 5000 = 5001

I thinbk, the correct statement must be:

if(RGetBool(hR,"first")) lim = limit -1;

Rgds

hk

How can I make MetaTrader 4 Build 600 array questions

lve0200 2017.01.07 13:17 #32

@ Vladimir,

your code works OK! sorry for my mistake, as in the previous post. However, there are issues around this array. May be, its a good idea, to put an additional line in the code, which checks the following condition: Bars > limit > 500. If this condition is not true, the whole thing crashs. You can put the EA in the strategy tester under the conditons: 1) put the thing on "OnTick()", because OnTimer doesn't work in the tester AND 2) there seems to be a limit for the array. The input parameter "limit" must be set to 1000 for testing. Beware that the tester maintains his own set of input parameters. So, if you change the defaults for the input parameters at compile time, they ARE NOT automatically forwarded to the tester!! (Only the very first time). The tester will continue with the old values!!

Now, that i was able to run the thing in the tester, I pumped the whole 2016 through it [DAX30]. The only code change I made was setting the epoch for the SAE training to 300, because the original value looked a bit too lightweight to me. Now the real work of fine tuning the trading can begin. Obviously the TP/SL value is crucial. The TP/SL must coincide with the time frame, the volatility of the instrument and the network. Here are the results of the SAE with:

epoch 300, TP/SL = 50/25, Whole year 2016, limit = 1000, all other parms as in the original code. Its not really that great ... Don't worry about the 3467 chart errors; they are always there, even in my own EAs, and they seam to be w/out impact.

For details, look at the attached file.

Best Regards

hk

Files:

e_DNSAE.zip 472 kb

Lot size in strategy For loop in an Indicator does not work

lve0200 2017.01.07 16:01 #33

OK,

for the RBM, some more work was necessary. The RBM kept crashing for all limit inputs < 1200. Don't know why. I changed a few 500/501 values in the R files to 401 and like that I got it working for limit = 1000. I set the epoch from 5 to 25.

The result is not any better, than the SAE :-(

I realised that the networks crash after apporax. 3 Months of input (check the date/time entries of the result files. Looks like a memory leak somewhere.

Find the result file attached.

Rgds

hk

Files:

StrategyTester2.zip 32 kb

StrategyTester2.gif 10 kb

Questions on OOP in Backtesting/Optimization Asctrend not sending Emails

Vladimir Perervenko 2017.01.12 09:00 #34

lve0200:

Всем привет,

Хорошо, у меня есть все, чтобы бежать.

1: убедитесь, у вас есть Darch версия 0.10.0 !! Как это сделать, смотрите пост # 29

2: убедитесь, что у вас есть достаточное количество исторических данных по символу хотеть. Открыть диаграмму этого символа в МТ4 и контролировать, сколько исторических данных вы имеете нагрузку.

3: @ Владимир. ИМХО есть еще ошибка в коде: (обе версии)

if (RGetBool(hR, "first" )) lim = limit;  <=== lim = limit
       // Resize arrays
       ArrayResize (o, lim);
       ArrayResize (hi, lim);
       ArrayResize (lo, lim);
       ArrayResize (clo, lim);
       // Fill the array with new data
       for (i = 0 ; i < lim; i++)    <===  i < lim, BUT!!
        {
         o[i]  = Open [i+ 1 ];  <== i+ 1
         hi[i] = High [i+ 1 ];
         lo[i] = Low [i+ 1 ];
         clo[i]= Close [i+ 1 ];

Это приводит к ошибки проверки границ данных! Если предел равен 5000 г = 0 .. 4999, я + 1 = 5000, следовательно, 0 .. 5000 = 5001

Я thinbk, правильное утверждение должно быть:

если (RGetBool (чч, "первый")) Нт = предел -1;

Rgds

Нк

Нет ошибки.

Лимит = 5000. i=0-4999, но i+1=1...5000. Почему так? Котировки мы берем начиная с первого бара ( сформировавшегося), а массив заполняем с нулевого бара.

Для обучения глубокой нейросети необходимо как можно больше данных на этапе претренинга. На этапе тюнинга можно кардинально уменьшить количество примеров, предоставив только наиболее важные. Например по 2 бара с каждой стороны вершины зигзага.

Отсечение последних 500 баров ( можно уменьшить до 300) - эти данные нужны для тестирования после обучения. Нейросеть не должна их видеть предварительно.

Я рад, что Вы изучаете эту тему. Готов ответить на Ваши вопросы.

Я сейчас готовлю статью по darch ( v 0.12.0)/ Она сильно отличается от 0.10.0. Огромное количество вариантов настроек. Без освоения 0.10.0 будет сложно понять новую версию

Удачи

Library for easy and Library for easy and

Vladimir Perervenko 2017.01.12 09:10 #35

lve0200:

в порядке,

для УКР, еще несколько работ было необходимо. УКР терпело крах для всех предельных входов <1200. Не знаю, почему. Я изменил несколько 500/501 значения в R файлы 401 и нравится, что я получил это работает на пределе = 1000. Я установил эпоху от 5 до 25 лет.

Результат не лучше, чем :-( SAE

Я понял, что аварии сети после apporax. 3 месяца ввода (проверьте записи даты / времени результата файлов. Похоже, утечка памяти где-то.

Найдите файл результата прилагается.

Rgds

Нк

Привет lve0200

Я не запускаю эксперт в тестере. Теряется самая важная возможность эксперта ( Динамическое переобучение ). Запускаю на демо счете.

Я писал, что SAE дает более стабильные результаты чем RBM.

Утечку памяти я не наблюдал.

Удачи

Library for easy and

lve0200 2017.01.24 08:41 #36

Hi Vladimir,

1) if I train the network with 500 candles, and K ~ 15, it never re-trains. K remains stable or becomes better. I therefore use the tester for quick analysis. The tester speeds up the view on the behaviour of the EA, enormously! If you run only on Demo accounts, it takes forever and you may not have a representative trade behaviour, because you may have caught a trend or a side wise move, only and when market behavour changes, your findings are not valid, anymore.

2) I am not sure, if the Zigzag is a good target for output. I believe that the Zigzag in unpredicatable, because it is only a book-keeper which shows you the last Top/Bottom in the past, and with some delay. There are some warnings out there, which say "Never use ZigZag for trading decisions" it is just a post mortem analytical tool!

3) The way you constructed the SW is, that the time period of the chart is the "tick" of the network. One new candle in the chart = 1 new tick for the network. Like this the network (TTR) can only caluculate indicators >= of this time period. In your SW, Indicators are always calculated on the same time period, equal to the chart time period. It may be desirable to let the MT4 EA calculate the indicators on different time periods and upload the vectors into R, as you do with candles.

4) Your SW immediately re-news a trade, if SL or TP was hit; this is an absurd use of SL/TP. I changed the structure of your SW, so that trades terminated by SL or TP are only re-newed on the next Signal, when the trading direction has changed.

Best Regards

hk

Machine learning in trading: NonLag MA ver.7.1 slows Problem reading data from

Vladimir Perervenko 2017.01.24 19:48 #37

lve0200:

Hi Vladimir,

1) if I train the network with 500 candles, and K ~ 15, it never re-trains. K remains stable or becomes better. I therefore use the tester for quick analysis. The tester speeds up the view on the behaviour of the EA, enormously! If you run only on Demo accounts, it takes forever and you may not have a representative trade behaviour, because you may have caught a trend or a side wise move, only and when market behavour changes, your findings are not valid, anymore.

2) I am not sure, if the Zigzag is a good target for output. I believe that the Zigzag in unpredicatable, because it is only a book-keeper which shows you the last Top/Bottom in the past, and with some delay. There are some warnings out there, which say "Never use ZigZag for trading decisions" it is just a post mortem analytical tool!

3) The way you constructed the SW is, that the time period of the chart is the "tick" of the network. One new candle in the chart = 1 new tick for the network. Like this the network (TTR) can only caluculate indicators >= of this time period. In your SW, Indicators are always calculated on the same time period, equal to the chart time period. It may be desirable to let the MT4 EA calculate the indicators on different time periods and upload the vectors into R, as you do with candles.

4) Your SW immediately re-news a trade, if SL or TP was hit; this is an absurd use of SL/TP. I changed the structure of your SW, so that trades terminated by SL or TP are only re-newed on the next Signal, when the trading direction has changed.

Best Regards

hk

Прежде всего я рад, что Вам пригодился пример в статье. Несколько мыслей по :

1. Для обучения нейросети 500 баров абсолютно недостаточно. Минимум 2000 баров. Тестировать можно на 300 баров. Эксперт в тестере МТ4 может работать только с onTick().

Мы сейчас разработали шлюз R к МТ5. Там тестер более продвинутый.

Во всем остальном исследуйте, экспериментируйте.

Желаю удачи.

История баров начиная с Triangular moving average (TMA) My newbie system for

JulInParis 2017.03.28 17:21 #38

Hi Vladimir,

When trying to execute your code step by step, I came across the following error when executing these two lines ( I'm using R Revolution 3.2.3 ) :

1)

if (soft) { y <- DT$train$y %>% classvec2classmat()}

R : "object 'soft' not found"

...and R is right : only the object actFun$soft exists at this point ! 

Should we replace the lines referring to 'soft' by something like 'actFun$soft' ?

2) 

y <- DT$train$y %>% classvec2classmat()

R : can't find classvec2classmat

This function is available in the kohonen package but I'm not sure if I should add this package to the project...

Best regards.

Julien

Error -1 array out of range Examples: Meta COT Project

Vladimir Perervenko 2017.03.29 12:28 #39

JulInParis:

Hi Vladimir,

When trying to execute your code step by step, I came across the following error when executing these two lines ( I'm using R Revolution 3.2.3 ) :

1)

if (soft) { y <- DT$train$y %>% classvec2classmat()}

1. Неверно. Переменная определяется в эксперте

input bool   soft          = true; //softmax?
....
RAssignBool(hR, "soft", soft);

Эта переменная должна быть в глобальном окружении.

Vladimir Perervenko 2017.03.29 12:33 #40

JulInParis:

Hi Vladimir,

When trying to execute your code step by step, I came across the following error when executing these two lines ( I'm using R Revolution 3.2.3 ) :

1)

if (soft) { y <- DT$train$y %>% classvec2classmat()}

2. Верно. Просто перепишите

y <- DT$train$y %>% kohonen::classvec2classmat()

Пакет "kohonen" должен быть инсталлирован.

Удачи

Discussion of article "Deep neural network with Stacked RBM. Self-training, self-control" - page 4