Backtesting a Neural Network - page 2

[Marco vd Heijden]

Adrijus:

Yes, I do understand the the sentence you marked in bold is really debatable. I have not heard about a term "memory size" before however, I fully understand what you mean by that as it makes perfect sense. Somehow I made an impression that the more training data, the better and never thought about this concept of memory size.

Trying not to deviate from the topic too much, I am using a Feed-forward Neural Network with it's inputs as Tapped delay line hence, overall known as Time delay neural network. It uses Levenberg-Marquardt algorithm for quicker convergence and Bayesian Regularization to reduce overfitting. Also it stops training when validation error stops decreasing. As I understand this TDNN is really susceptible to memory size. The way it constructed right now is that data[] is being converted into a matrix of numberOfVariables*timeDelays by numberOfPatterns. Then it is being divided into training and validation matrices and testing part is disregarded and training part is fed through NN. That was all under assumption that all of the data is available at the very beginning (how naive).

All that being said I will have to have take your advise for the sake of better generalization without increasing NN size where it would make the training unbearably slow and even then it might not generalize well enough. In terms of data being outdated I though about it myself however, I was going to backtest with all data before limiting it to see what kind of difference it would make (maybe just my curiosity). I am also concerned about the amount of data my broker provides on lets say EUR/USD 1H which is only about 1.5 years so if choose longer period of time for more patterns that might cause some issues.

Regards,

Adrijus

1.5 years might still be too much.

Maybe 5 blocks of D1 will speak to you, and that is, less then a week.

That could be your short term thing, and then you can multiply that by 4, so 4*5  D1 to meet 'natural' standards and look at the same but a larger picture which can be your medium thing,

And then some seasonal print 2*6 MN1 or 3*4 MN1 and etc nut anything beyond that might have reliability issues.

[Ernst Van Der Merwe]

Stanislav Korotky:
Your code is buggy. You should set timeToStart only once (when it's uninitialized yet), otherwise it's changing on every tick/bar and if-statement triggers always. Also timeToStart should be double.

No, it's not. When a static variable is immediately assigned a value, it doesn't change. Also timeToStart shouldn't be a double. The code is working fine.

int barsToWait=20;
//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
//---
   static int timeToStart=(int)Time[0]+_Period*60*barsToWait;
   
   if((int)Time[0]<timeToStart)
     { 
      static datetime prevTime;
      if(prevTime!=Time[0])
        {
         Print((string)Time[0]);
         prevTime=Time[0];
        }
      return;
     }
   else
      printf("Time to start=%s",(string)Time[0]);
//---


  }