![]() |
Neuron Net
|
Class of Self-Attention layer GPU calculation. More...
Public Member Functions | |
CNeuronAttentionOCL (void) | |
Constructor. More... | |
~CNeuronAttentionOCL (void) | |
Destructor. More... | |
virtual bool | Init (uint numOutputs, uint myIndex, COpenCLMy *open_cl, uint window, uint units_count, ENUM_OPTIMIZATION optimization_type) |
Method of initialization class. More... | |
virtual bool | calcInputGradients (CNeuronBaseOCL *prevLayer) |
Method to transfer gradients to previous layer. More... | |
virtual int | Type (void) const |
Identificator of class. More... | |
virtual bool | Save (int const file_handle) |
Save method. More... | |
virtual bool | Load (int const file_handle) |
Load method. More... | |
virtual bool | Init (uint numOutputs, uint myIndex, COpenCLMy *open_cl, uint numNeurons, ENUM_OPTIMIZATION optimization_type) |
Method of initialization class. More... | |
virtual void | SetActivationFunction (ENUM_ACTIVATION value) |
Set the type of activation function (ENUM_ACTIVATION) More... | |
virtual int | getOutputIndex (void) |
Get index of output buffer. More... | |
virtual int | getPrevOutIndex (void) |
Get index of previous iteration output buffer. More... | |
virtual int | getGradientIndex (void) |
Get index of gradient buffer. More... | |
virtual int | getWeightsIndex (void) |
Get index of weights matrix buffer. More... | |
virtual int | getDeltaWeightsIndex (void) |
Get index of delta weights matrix buffer (SGD) More... | |
virtual int | getFirstMomentumIndex (void) |
Get index of first momentum matrix buffer (Adam) More... | |
virtual int | getSecondMomentumIndex (void) |
Get index of Second momentum matrix buffer (Adam) More... | |
virtual int | getOutputVal (double &values[]) |
Get values of output buffer. More... | |
virtual int | getOutputVal (CArrayDouble *values) |
Get values of output buffer. More... | |
virtual int | getPrevVal (double &values[]) |
Get values of previous iteration output buffer. More... | |
virtual int | getGradient (double &values[]) |
Get values of gradient buffer. More... | |
virtual int | getWeights (double &values[]) |
Get values of weights matrix buffer. More... | |
virtual int | Neurons (void) |
Get number of neurons in layer. More... | |
virtual int | Activation (void) |
Get type of activation function. More... | |
virtual int | getConnections (void) |
Get number of connections 1 neuron to next layer. More... | |
virtual bool | FeedForward (CObject *SourceObject) |
Dispatch method for defining the subroutine for feed forward process. More... | |
virtual bool | calcHiddenGradients (CObject *TargetObject) |
Dispatch method for defining the subroutine for transferring the gradient to the previous layer. More... | |
virtual bool | calcHiddenGradients (CNeuronBaseOCL *NeuronOCL) |
Method to transfer gradient to previous layer. More... | |
virtual bool | calcOutputGradients (CArrayDouble *Target) |
Method of output gradients calculation. More... | |
virtual bool | UpdateInputWeights (CObject *SourceObject) |
Dispatch method for defining the subroutine for updating weights. More... | |
Protected Member Functions | |
virtual bool | feedForward (CNeuronBaseOCL *prevLayer) |
Feed Forward method. More... | |
virtual bool | updateInputWeights (CNeuronBaseOCL *prevLayer) |
Method for updating weights. More... | |
Protected Attributes | |
CNeuronConvOCL * | Querys |
Convolution layer for Querys. More... | |
CNeuronConvOCL * | Keys |
Convolution layer for Keys. More... | |
CNeuronConvOCL * | Values |
Convolution layer for Values. More... | |
CBufferDouble * | Scores |
Buffer for Scores matrix. More... | |
CNeuronBaseOCL * | AttentionOut |
Layer of Self-Attention Out. More... | |
CNeuronConvOCL * | FF1 |
Convolution layer for first layer of Feed Forward block. More... | |
CNeuronConvOCL * | FF2 |
Convolution layer for second layer of Feed Forward block. More... | |
uint | iWindow |
Window size. More... | |
uint | iUnits |
Number of units. More... | |
COpenCLMy * | OpenCL |
Object for working with OpenCL. More... | |
CBufferDouble * | Output |
Buffer of Output tenzor. More... | |
CBufferDouble * | PrevOutput |
Buffer of previous iteration Output tenzor. More... | |
CBufferDouble * | Weights |
Buffer of weights matrix. More... | |
CBufferDouble * | DeltaWeights |
Buffer of last delta weights matrix (SGD) More... | |
CBufferDouble * | Gradient |
Buffer of gradient tenzor. More... | |
CBufferDouble * | FirstMomentum |
Buffer of first momentum matrix (ADAM) More... | |
CBufferDouble * | SecondMomentum |
Buffer of second momentum matrix (ADAM) More... | |
const double | alpha |
Multiplier to momentum in SGD optimization. More... | |
int | t |
Count of iterations. More... | |
int | m_myIndex |
Index of neuron in layer. More... | |
ENUM_ACTIVATION | activation |
Activation type (ENUM_ACTIVATION) More... | |
ENUM_OPTIMIZATION | optimization |
Optimization method (ENUM_OPTIMIZATION) More... | |
Class of Self-Attention layer GPU calculation.
Detailed description on the link.
Definition at line 3880 of file NeuroNet.mqh.
|
inline |
Constructor.
Definition at line 3898 of file NeuroNet.mqh.
CNeuronAttentionOCL::~CNeuronAttentionOCL | ( | void | ) |
Destructor.
Definition at line 3912 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get type of activation function.
Definition at line 2966 of file NeuroNet.mqh.
|
virtualinherited |
Method to transfer gradient to previous layer.
NeuronOCL | Pointer to next layer. |
Definition at line 3194 of file NeuroNet.mqh.
|
virtualinherited |
Dispatch method for defining the subroutine for transferring the gradient to the previous layer.
TargetObject | Pointer to the next layer. |
Definition at line 3318 of file NeuroNet.mqh.
|
virtual |
Method to transfer gradients to previous layer.
[in] | prevLayer | Pointer to previous layer. |
Definition at line 4126 of file NeuroNet.mqh.
|
virtualinherited |
Method of output gradients calculation.
Target | Traget value |
Definition at line 3227 of file NeuroNet.mqh.
|
protectedvirtual |
Feed Forward method.
prevLayer | Pointer to previos layer. |
Reimplemented from CNeuronBaseOCL.
Definition at line 4018 of file NeuroNet.mqh.
|
virtualinherited |
Dispatch method for defining the subroutine for feed forward process.
SourceObject | Pointer to the previous layer. |
Definition at line 3141 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get number of connections 1 neuron to next layer.
Definition at line 2967 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get index of delta weights matrix buffer (SGD)
Definition at line 2956 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get index of first momentum matrix buffer (Adam)
Definition at line 2957 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get values of gradient buffer.
[out] | values | Array of data |
Definition at line 2963 of file NeuroNet.mqh.
|
inlinevirtualinherited |
|
inlinevirtualinherited |
|
inlinevirtualinherited |
Get values of output buffer.
[out] | values | Array of data |
Definition at line 2961 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get values of output buffer.
[out] | values | Array of data |
Definition at line 2960 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get index of previous iteration output buffer.
Definition at line 2953 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get values of previous iteration output buffer.
[out] | values | Array of data |
Definition at line 2962 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get index of Second momentum matrix buffer (Adam)
Definition at line 2958 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get values of weights matrix buffer.
[out] | values | Array of data |
Definition at line 2964 of file NeuroNet.mqh.
|
inlinevirtualinherited |
|
virtualinherited |
Method of initialization class.
[in] | numOutputs | Number of connections to next layer. |
[in] | myIndex | Index of neuron in layer. |
[in] | open_cl | Pointer to COpenCLMy object. #param[in] numNeurons Number of neurons in layer |
optimization_type | Optimization type (ENUM_OPTIMIZATION) |
Definition at line 3021 of file NeuroNet.mqh.
|
virtual |
Method of initialization class.
[in] | numOutputs | Number of connections to next layer. |
[in] | myIndex | Index of neuron in layer. |
[in] | open_cl | Pointer to COpenCLMy object. |
[in] | window | Size of in/out window and step. |
[in] | units_countNumber | of neurons. |
[in] | optimization_type | Optimization type (ENUM_OPTIMIZATION) |
Definition at line 3932 of file NeuroNet.mqh.
|
virtual |
Load method.
[in] | file_handle | handle of file |
Reimplemented from CNeuronBaseOCL.
Definition at line 4326 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Get number of neurons in layer.
Definition at line 2965 of file NeuroNet.mqh.
|
virtual |
Save method.
[in] | file_handle | handle of file |
Reimplemented from CNeuronBaseOCL.
Definition at line 4298 of file NeuroNet.mqh.
|
inlinevirtualinherited |
Set the type of activation function (ENUM_ACTIVATION)
Definition at line 2950 of file NeuroNet.mqh.
|
inlinevirtual |
Identificator of class.
Reimplemented from CNeuronBaseOCL.
Definition at line 3904 of file NeuroNet.mqh.
|
protectedvirtual |
Method for updating weights.
prevLayer | Pointer to previos layer. |
Reimplemented from CNeuronBaseOCL.
Definition at line 4265 of file NeuroNet.mqh.
|
virtualinherited |
Dispatch method for defining the subroutine for updating weights.
SourceObject | Pointer to previos layer. |
Definition at line 3349 of file NeuroNet.mqh.
|
protectedinherited |
Activation type (ENUM_ACTIVATION)
Definition at line 2936 of file NeuroNet.mqh.
|
protectedinherited |
Multiplier to momentum in SGD optimization.
Definition at line 2932 of file NeuroNet.mqh.
|
protected |
Layer of Self-Attention Out.
Definition at line 3887 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of last delta weights matrix (SGD)
Definition at line 2926 of file NeuroNet.mqh.
|
protected |
Convolution layer for first layer of Feed Forward block.
Definition at line 3888 of file NeuroNet.mqh.
|
protected |
Convolution layer for second layer of Feed Forward block.
Definition at line 3889 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of first momentum matrix (ADAM)
Definition at line 2928 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of gradient tenzor.
Definition at line 2927 of file NeuroNet.mqh.
|
protected |
Number of units.
Definition at line 3892 of file NeuroNet.mqh.
|
protected |
Window size.
Definition at line 3891 of file NeuroNet.mqh.
|
protected |
Convolution layer for Keys.
Definition at line 3884 of file NeuroNet.mqh.
|
protectedinherited |
Index of neuron in layer.
Definition at line 2935 of file NeuroNet.mqh.
|
protectedinherited |
Object for working with OpenCL.
Definition at line 2922 of file NeuroNet.mqh.
|
protectedinherited |
Optimization method (ENUM_OPTIMIZATION)
Definition at line 2937 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of Output tenzor.
Definition at line 2923 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of previous iteration Output tenzor.
Definition at line 2924 of file NeuroNet.mqh.
|
protected |
Convolution layer for Querys.
Definition at line 3883 of file NeuroNet.mqh.
|
protected |
Buffer for Scores matrix.
Definition at line 3886 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of second momentum matrix (ADAM)
Definition at line 2929 of file NeuroNet.mqh.
|
protectedinherited |
Count of iterations.
Definition at line 2933 of file NeuroNet.mqh.
|
protected |
Convolution layer for Values.
Definition at line 3885 of file NeuroNet.mqh.
|
protectedinherited |
Buffer of weights matrix.
Definition at line 2925 of file NeuroNet.mqh.