How to distribute calculations over multiple cores?

[Lorentzos Roussos]

If i have 8 cores in the cpu and i want to distribute calculations but not in the tester environment do i open 8 charts and have them work on their distributed packs of calculations ? 

[Alain Verleyen]

<Please edit your topic title, try to always use a meaningful title when you create a topic>.

[Fernando Carreiro]

Lorentzos Roussos: If i have 8 cores in the cpu and i want to distribute calculations but not in the tester environment do i open 8 charts and have them work on their distributed packs of calculations ? 

I think given that MetaTrader is multi-threaded, it will be the operating system that decides which threads are placed on which cores.

Some may be dispersed over the cores and some may be on the same core, because other applications will also be doing the same.

[Lorentzos Roussos]

Fernando Carreiro #:

I think given that MetaTrader is multi-threaded, it will be the operating system that decides which threads are placed on which cores.

Some may be dispersed over the cores and some may be on the same core, because other applications will also be doing the same.

But the chart utilizes one thread from the cpu ? is that what it means that eas run on their thread ?

[Fernando Carreiro]

Lorentzos Roussos #: But the chart utilizes one thread from the cpu ? is that what it means that eas run on their thread ?

Yes! Most PC cores normally handle two simultaneous threads per core, but some only handle one thread per core.

But the threads get switched in and out so that threads from each application get a chance at being executed.

[Lorentzos Roussos]

Fernando Carreiro #:

Yes! Most PC cores normally handle two simultaneous threads per core, but some only handle one thread per core.

But the threads get switched in and out so that threads from each application get a chance at being executed.

Aha so my limitation is that the execution in the ea is linear and its my job to make it parallel (not on one ea) ?

[Fernando Carreiro]

Lorentzos Roussos #: Aha so my limitation is that the execution in the ea is linear and its my job to make it parallel (not on one ea) ?

Yes, each EA runs linearly on it's own thread. MetaTrader is multi-threaded, but EAs are not. MQL programs are single threaded.

You can try offload some of the calculations using OpenCL, which can use the GPU on your graphics card or even the other CPU cores. But you will have to use a "different language" if you want to work with OpenCL.

[Lorentzos Roussos]

Fernando Carreiro #:

Yes, each EA runs linearly on it's own thread. MetaTrader is multi-threaded, but EAs are not. MQL programs are single threaded.

You can try offload some of the calculations using OpenCL, which can use the GPU on your graphics card or even the other CPU cores. But you will have to use a "different language" if you want to work with OpenCL.

ow , i thought it was only for GPUs . 

Another question , if i use the timer and it's called per millisecond and i don't gate the calculations based on whether or not the ea is busy won't i eventually force a "spillover" to other cores ?

something like this , (draft) , but the calcs here are meaningless (and it definately completes the calculation below 1 millisecond )

#property version   "1.00"
input bool usebusy=true;//block if busy
input int sizeofvectors=1000;//size of vectors

bool SystemBusy=false;
int OnInit()
  {
  SystemBusy=false;
  EventSetMillisecondTimer(1);
  return(INIT_SUCCEEDED);
  }
void OnTimer(){
  if(!usebusy||!SystemBusy){
    SystemBusy=true;
    
    vector randoms,activations,derivations;
    randoms.Init(sizeofvectors);
    activations.Init(sizeofvectors);
    derivations.Init(sizeofvectors);
    for(int i=0;i<sizeofvectors;i++){
       randoms[i]=((double)MathRand())/32767.0;
       }
    randoms.Activation(activations,AF_SIGMOID);
    randoms.Derivative(derivations,AF_SIGMOID);
    
    SystemBusy=false;
    }
  }
void OnDeinit(const int reason)
  {
  }
void OnTick()
  {
  }

[Fernando Carreiro]

Lorentzos Roussos #: ow , i thought it was only for GPUs . Another question , if i use the timer and it's called per millisecond and i don't gate the calculations based on whether or not the ea is busy won't i eventually force a "spillover" to other cores ? something like this , (draft) , but the calcs here are meaningless (and it definately completes the calculation below 1 millisecond )

No! As I stated, MQL programs are single-threaded and run linearly. Each event is placed in a queue and processed one at a time.

If you do not return from the OnTimer(), then no other event is processed until it returns, be they new tick or a new chart event or other timer events.

Events will just accumulate in the queue and wait to be processed.

[Lorentzos Roussos]

Fernando Carreiro #:

No! As I stated, MQL programs are single-threaded and run linearly. Each event is placed in a queue and processed one at a time.

If you do not return from the OnTimer(), then no other event is processed until it returns, be they new tick or a new chart event or other timer events. They will just accumulate in the queue and wait to be processed.

Okay . 

Why are all 4 cores spiking up though here ? i set it to 10million calcs per ms

Would running this on mt4 and comparing be a good benchmark test ?

and the second one is from when i stopped it , marked in red

[Fernando Carreiro]

Lorentzos Roussos #: Okay . Why are all 4 cores spiking up though here ? i set it to 10million calcs per ms

Because MetaTrader is multi-threaded and the operating system will distribute the threads of all applications amongst the CPU's and the threads get switched in an out and not necessarily always on the same CPU.