![]() |
Neuron Net
|
Class of Self-Attention layer GPU calculation. More...
Public Member Functions | |
CNeuronAttentionOCL (void) | |
Constructor. More... | |
~CNeuronAttentionOCL (void) | |
Destructor. More... | |
virtual bool | Init (uint numOutputs, uint myIndex, COpenCLMy *open_cl, uint window, uint units_count, ENUM_OPTIMIZATION optimization_type) |
Method of initialization class. More... | |
virtual bool | calcInputGradients (CNeuronBaseOCL *prevLayer) |
Method to transfer gradients to previous layer. More... | |
virtual int | Type (void) const |
Identificator of class. More... | |
virtual bool | Save (int const file_handle) |
Save method. More... | |
virtual bool | Load (int const file_handle) |
Load method. More... | |
virtual bool | Init (uint numOutputs, uint myIndex, COpenCLMy *open_cl, uint numNeurons, ENUM_OPTIMIZATION optimization_type) |
Method of initialization class. More... | |
virtual void | SetActivationFunction (ENUM_ACTIVATION value) |
Set the type of activation function (ENUM_ACTIVATION) More... | |
virtual int | getOutputIndex (void) |
Get index of output buffer. More... | |
virtual int | getPrevOutIndex (void) |
Get index of previous iteration output buffer. More... | |
virtual int | getGradientIndex (void) |
Get index of gradient buffer. More... | |
virtual int | getWeightsIndex (void) |
Get index of weights matrix buffer. More... | |
virtual int | getDeltaWeightsIndex (void) |
Get index of delta weights matrix buffer (SGD) More... | |
virtual int | getFirstMomentumIndex (void) |
Get index of first momentum matrix buffer (Adam) More... | |
virtual int | getSecondMomentumIndex (void) |
Get index of Second momentum matrix buffer (Adam) More... | |
virtual int | getOutputVal (double &values[]) |
Get values of output buffer. More... | |
virtual int | getOutputVal (CArrayDouble *values) |
Get values of output buffer. More... | |
virtual int | getPrevVal (double &values[]) |
Get values of previous iteration output buffer. More... | |
virtual int | getGradient (double &values[]) |
Get values of gradient buffer. More... | |
virtual int | getWeights (double &values[]) |
Get values of weights matrix buffer. More... | |
virtual int | Neurons (void) |
Get number of neurons in layer. More... | |
virtual int | Activation (void) |
Get type of activation function. More... | |
virtual int | getConnections (void) |
Get number of connections 1 neuron to next layer. More... | |
virtual bool | FeedForward (CObject *SourceObject) |
Dispatch method for defining the subroutine for feed forward process. More... | |
virtual bool | calcHiddenGradients (CObject *TargetObject) |
Dispatch method for defining the subroutine for transferring the gradient to the previous layer. More... | |
virtual bool | calcHiddenGradients (CNeuronBaseOCL *NeuronOCL) |
Method to transfer gradient to previous layer by calling kernel CalcHiddenGradient(). More... | |
virtual bool | UpdateInputWeights (CObject *SourceObject) |
Dispatch method for defining the subroutine for updating weights. More... | |
virtual bool | calcOutputGradients (CArrayDouble *Target) |
Method of output gradients calculation by calling kernel CalcOutputGradient(). More... | |
Protected Member Functions | |
virtual bool | feedForward (CNeuronBaseOCL *prevLayer) |
Feed Forward method. More... | |
virtual bool | updateInputWeights (CNeuronBaseOCL *prevLayer) |
Method for updating weights. More... | |
Protected Attributes | |
CNeuronConvOCL * | Querys |
Convolution layer for Querys. More... | |
CNeuronConvOCL * | Keys |
Convolution layer for Keys. More... | |
CNeuronConvOCL * | Values |
Convolution layer for Values. More... | |
CBufferDouble * | Scores |
Buffer for Scores matrix. More... | |
CNeuronBaseOCL * | AttentionOut |
Layer of Self-Attention Out. More... | |
CNeuronConvOCL * | FF1 |
Convolution layer for first layer of Feed Forward block. More... | |
CNeuronConvOCL * | FF2 |
Convolution layer for second layer of Feed Forward block. More... | |
uint | iWindow |
Window size. More... | |
uint | iUnits |
Number of units. More... | |
COpenCLMy * | OpenCL |
Object for working with OpenCL. More... | |
CBufferDouble * | Output |
Buffer of Output tenzor. More... | |
CBufferDouble * | PrevOutput |
Buffer of previous iteration Output tenzor. More... | |
CBufferDouble * | Weights |
Buffer of weights matrix. More... | |
CBufferDouble * | DeltaWeights |
Buffer of last delta weights matrix (SGD) More... | |
CBufferDouble * | Gradient |
Buffer of gradient tenzor. More... | |
CBufferDouble * | FirstMomentum |
Buffer of first momentum matrix (ADAM) More... | |
CBufferDouble * | SecondMomentum |
Buffer of second momentum matrix (ADAM) More... | |
const double | alpha |
Multiplier to momentum in SGD optimization. More... | |
int | t |
Count of iterations. More... | |
int | m_myIndex |
Index of neuron in layer. More... | |
ENUM_ACTIVATION | activation |
Activation type (ENUM_ACTIVATION) More... | |
ENUM_OPTIMIZATION | optimization |
Optimization method (ENUM_OPTIMIZATION) More... | |
Class of Self-Attention layer GPU calculation.
Detailed description on the link.
Definition at line 3890 of file NeuroNet.mqh.
|
inline |
Constructor.
Definition at line 3908 of file NeuroNet.mqh.
CNeuronAttentionOCL::~CNeuronAttentionOCL | ( | void | ) |
Destructor.
Definition at line 3922 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get type of activation function.
Definition at line 2973 of file NeuroNet.mqh.
|
virtualinherited |
Dispatch method for defining the subroutine for transferring the gradient to the previous layer.
TargetObject | Pointer to the next layer. |
Definition at line 3328 of file NeuroNet.mqh.
|
virtual |
Method to transfer gradients to previous layer.
[in] | prevLayer | Pointer to previous layer. |
Definition at line 4136 of file NeuroNet.mqh.
|
protectedvirtual |
Feed Forward method.
prevLayer | Pointer to previos layer. |
Reimplemented from CNeuronBaseOCL.
Definition at line 4028 of file NeuroNet.mqh.
|
virtualinherited |
Dispatch method for defining the subroutine for feed forward process.
SourceObject | Pointer to the previous layer. |
Definition at line 3151 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get number of connections 1 neuron to next layer.
Definition at line 2974 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get index of delta weights matrix buffer (SGD)
Definition at line 2963 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get index of first momentum matrix buffer (Adam)
Definition at line 2964 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get values of gradient buffer.
[out] | values | Array of data |
Definition at line 2970 of file NeuroNet.mqh.
|
inlinevirtualinherited |
|
inlinevirtualinherited |
|
inlinevirtualinherited |
Get values of output buffer.
[out] | values | Array of data |
Definition at line 2968 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get values of output buffer.
[out] | values | Array of data |
Definition at line 2967 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get index of previous iteration output buffer.
Definition at line 2960 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get values of previous iteration output buffer.
[out] | values | Array of data |
Definition at line 2969 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get index of Second momentum matrix buffer (Adam)
Definition at line 2965 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get values of weights matrix buffer.
[out] | values | Array of data |
Definition at line 2971 of file NeuroNet.mqh.
|
inlinevirtualinherited |
|
virtualinherited |
Method of initialization class.
[in] | numOutputs | Number of connections to next layer. |
[in] | myIndex | Index of neuron in layer. |
[in] | open_cl | Pointer to COpenCLMy object. #param[in] numNeurons Number of neurons in layer |
optimization_type | Optimization type (ENUM_OPTIMIZATION) |
Definition at line 3031 of file NeuroNet.mqh.
|
virtual |
Method of initialization class.
[in] | numOutputs | Number of connections to next layer. |
[in] | myIndex | Index of neuron in layer. |
[in] | open_cl | Pointer to COpenCLMy object. |
[in] | window | Size of in/out window and step. |
[in] | units_countNumber | of neurons. |
[in] | optimization_type | Optimization type (ENUM_OPTIMIZATION) |
Definition at line 3942 of file NeuroNet.mqh.
|
virtual |
Load method.
[in] | file_handle | handle of file |
Reimplemented from CNeuronBaseOCL.
Definition at line 4336 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get number of neurons in layer.
Definition at line 2972 of file NeuroNet.mqh.
|
virtual |
Save method.
[in] | file_handle | handle of file |
Reimplemented from CNeuronBaseOCL.
Definition at line 4308 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Set the type of activation function (ENUM_ACTIVATION)
Definition at line 2957 of file NeuroNet.mqh.
|
inlinevirtual |
Identificator of class.
Reimplemented from CNeuronBaseOCL.
Definition at line 3914 of file NeuroNet.mqh.
|
protectedvirtual |
Method for updating weights.
prevLayer | Pointer to previos layer. |
Reimplemented from CNeuronBaseOCL.
Definition at line 4275 of file NeuroNet.mqh.
|
protectedinherited |
Activation type (ENUM_ACTIVATION)
Definition at line 2943 of file NeuroNet.mqh.
|
protectedinherited |
Multiplier to momentum in SGD optimization.
Definition at line 2939 of file NeuroNet.mqh.
|
protected |
Layer of Self-Attention Out.
Definition at line 3897 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of last delta weights matrix (SGD)
Definition at line 2933 of file NeuroNet.mqh.
|
protected |
Convolution layer for first layer of Feed Forward block.
Definition at line 3898 of file NeuroNet.mqh.
|
protected |
Convolution layer for second layer of Feed Forward block.
Definition at line 3899 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of first momentum matrix (ADAM)
Definition at line 2935 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of gradient tenzor.
Definition at line 2934 of file NeuroNet.mqh.
|
protected |
Number of units.
Definition at line 3902 of file NeuroNet.mqh.
|
protected |
Window size.
Definition at line 3901 of file NeuroNet.mqh.
|
protected |
Convolution layer for Keys.
Definition at line 3894 of file NeuroNet.mqh.
|
protectedinherited |
Index of neuron in layer.
Definition at line 2942 of file NeuroNet.mqh.
|
protectedinherited |
Object for working with OpenCL.
Definition at line 2929 of file NeuroNet.mqh.
|
protectedinherited |
Optimization method (ENUM_OPTIMIZATION)
Definition at line 2944 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of Output tenzor.
Definition at line 2930 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of previous iteration Output tenzor.
Definition at line 2931 of file NeuroNet.mqh.
|
protected |
Convolution layer for Querys.
Definition at line 3893 of file NeuroNet.mqh.
|
protected |
Buffer for Scores matrix.
Definition at line 3896 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of second momentum matrix (ADAM)
Definition at line 2936 of file NeuroNet.mqh.
|
protectedinherited |
Count of iterations.
Definition at line 2940 of file NeuroNet.mqh.
|
protected |
Convolution layer for Values.
Definition at line 3895 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of weights matrix.
Definition at line 2932 of file NeuroNet.mqh.