Limit in covariance matrix library?

Lorentzos Roussos 2023.12.22 18:07

Is there a size limit on the covariance matrix library ?

948 features x 16000 samples

  Print("Starting eigenveck");
           matrix original;
                  original.Init(ArraySize(samples),ArraySize(samples[0].features));   
                  //fill
                  for(int i=0;i<ArraySize(samples);i++){
                  for(int j=0;j<ArraySize(samples[0].features);j++){
                  original[i][j]=samples[i].features[j];                  
                  }
                  } 
             Print("Filled original matrix");
             matrix covariance_matrix=original.Cov(false);
             Print("Created covariance matrix");

2023.12.22 19:59:48.289 SVMNET_optimizer_01 (EURUSD,H1) Total features(948) Total Outcomes(966)
2023.12.22 19:59:48.400 SVMNET_optimizer_01 (EURUSD,H1) Total samples 16144
2023.12.22 19:59:48.400 SVMNET_optimizer_01 (EURUSD,H1) Starting eigenveck
2023.12.22 19:59:48.465 SVMNET_optimizer_01 (EURUSD,H1) Filled original matrix
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1) Access violation at 0x00007FF66F790B0E write to 0x0000024959388000
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)    crash -->  00007FF66F790B0E C5FDE7A1E0000000  vmovntdq   [rcx+0xE0], ymm4
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B16 4881C100010000    add        rcx, 0x100
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B1D 4881C200010000    add        rdx, 0x100
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B24 4981E800010000    sub        r8, 0x100
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B2B 4981F800010000    cmp        r8, 0x100
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B32 0F8378FFFFFF      jnb        0x00007FF66F790AB0
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1) 
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B38 4D8D481F          lea        r9, [r8+0x1F]
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B3C 4983E1E0          and        r9, 0xFFFFFFFFFFFFFFE0
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B40 4D8BD9            mov        r11, r9
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1)               00007FF66F790B43 49C1EB05          shr        r11, 0x05
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1) 
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1) 00: 0x00007FF66F790B0E
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1) 01: 0x00007FF66EF0B5E1
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1) 02: 0x0000000000E98740
2023.12.22 19:59:50.252 SVMNET_optimizer_01 (EURUSD,H1) 03: 0x00007FF66EF5189E

Alain Verleyen 2023.12.22 21:09 #1

Is it possible to have a source code to reproduce the issue ?

Lorentzos Roussos 2023.12.22 23:28 #2

Alain Verleyen #:
Is it possible to have a source code to reproduce the issue ?

yes

int OnInit()
  {
  EventSetMillisecondTimer(44); 
   return(INIT_SUCCEEDED);
  }
  
void OnTimer(){
EventKillTimer();
matrix original;
//original.Init(160,10);//works
original.Init(16000,1000);//fails
for(int r=0;r<original.Rows();r++){
   for(int c=0;c<original.Cols();c++){
      original[r][c]=((double)MathRand())/((double)32767.0);
      }
   }
Print("Values filled");
matrix covariance_matrix=original.Cov(false);
Print("DONE");
ExpertRemove();
}

void OnDeinit(const int reason)
  {
  }

void OnTick()
  {
  }

Alain Verleyen 2023.12.23 02:45 #3

Lorentzos Roussos #:

yes

Confirmed and reported to MQ.

Lorentzos Roussos 2023.12.23 20:05 #4

Alain Verleyen #:
Confirmed and reported to MQ.

thanks

Slava 2024.01.23 12:05 #5

Alain Verleyen #:
Confirmed and reported to MQ.

Fixed. Thank You

New comment