Discussion of article "Neural networks made easy (Part 5): Multithreaded calculations in OpenCL" - page 4

New comment
 
Is it possible to make an LSTM network with OpenCL? If yes, which version of NeuroNet.mqh is better? With the version from this article I get the error "Error of execution kernel CaclOutputGradient: 0". And with the version from article 7 it seems that OpenCL is not used at all, if you create defNeuronLSTM neurons. And defNeuronLSTMOCL is not implemented.
Нейросети — это просто (Часть 7): Адаптивные методы оптимизации
Нейросети — это просто (Часть 7): Адаптивные методы оптимизации
  • www.mql5.com
В предыдущих статьях мы рассмотрели разные типы нейронов, но всегда использовали метод стохастического градиентного спуска для обучения нейронной сети. Данный метод, наверное, можно назвать базовым и различные его вариации очень часто используются на практике. Тем не менее, он не единственный и существует целый ряд других методов для обучения...
 
Ivan Titov:
Is it possible to make an LSTM network with OpenCL? If yes, which version of NeuroNet.mqh is better? With the version from this article I get the error "Error of execution kernel CaclOutputGradient: 0". And with the version from article 7 it seems that OpenCL is not used at all, if you create defNeuronLSTM neurons. And defNeuronLSTMOCL is not implemented.
At the moment, LSTM has not been translated to OpenCL yet. I plan to do it in the near future.
 
I see, thanks. Please also clarify: recentAverageSmoothingFactor = 10000 - it is set rigidly. Should I change it by the number of values in the training sample?
 
Ivan Titov:
I see, thanks. Please also clarify: recentAverageSmoothingFactor = 10000 - it is set rigidly. Should it be changed by the number of values in the training sample?

If the sample is smaller, you should decrease it. It makes sense to increase it if you want to see the error over the whole sample.
I use a rigid value to evaluate the network performance in different market conditions.

 

Dmitry, I can't understand why the values in the array are not updated. There are no errors - there is no exit from the method by false. Can you check this for yourself?

bool CNeuronBaseOCL::calcOutputGradients(CArrayDouble *Target)
  {
   if(CheckPointer(OpenCL)==POINTER_INVALID || CheckPointer(Target)==POINTER_INVALID)
      return false;
   uint global_work_offset[1]= {0};
   uint global_work_size[1];
   global_work_size[0]=Target.Total();
   for(uint i=0; i<global_work_size[0]; i++)
     {
      double z=Target.At(i);                    //<--The Target array has three values (1,0;0.0;0.0;0.0)
      if(!Gradient.Update(i,Target.At(i)))
         return false;
     }
                                                //Check the values in the Gradient array after the update
   double target[];                             
   if(getGradient(target)<Gradient.Total())     //<--There are three values in the target array (0,0;0.0;0.0;0.0)
      return false;
//---
   Gradient.BufferWrite();
......
 }
 
Aleksei Lesnikov:

Dmitry, I can't understand why the values in the array are not updated. There are no errors - there is no exit from the method by false. Can you check this for yourself?

Alexey, the update method inherits from the CArrayDouble class and writes data to the array, but does not pass it to the GPU buffer.

if(!Gradient.Update(i,Target.At(i)))
         return false;

At the same time, when the getGradient method is called, it reads data from the GPU buffer, which overwrites the changes made earlier.

virtual int       getGradient(double &values[])    {  return Gradient.GetData(values);    }
.......
int CBufferDouble::GetData(double &values[])
  {
   if(!BufferRead())
      return false;
   return ArrayCopy(values,m_data,0,0,m_data_total);
  }
 
Dmitriy Gizlyk:

Alexei, the update method inherits from the CArrayDouble class and writes data to the array, but does not pass it to the GPU buffer.

At the same time, when the getGradient method is called, it reads data from the GPU buffer, which overwrites the changes made earlier.

Indeed, I've made a mistake. Thanks.
 

Thank you for these articles! It's been very interesting following the ideas of this system.

I'm very interested in the LSTM-module, is there a way to use Open-CL for LSTM-Networks?

I've been trying to modify the LSTM-EA to work with Open-CL, but with no success.

Best regards

Pontus

 

Where should I install the .cl file?
 
pennyhunter #:
Where should I install the .cl file?

Hello, you don't need install the .cl file. You just load it to main programm like string


12345
New comment