The Ultimate AI EA Project

New comment
 

Hello everyone. 

I would like to call upon every worthy programmer and trader to a crucial mission. A scientific project of a respectable standard, with the aim of easing the developing MQL programs using every tool available. A good start would be creating a public project then let anyone willing to join the development process collaborate.This project should be open for every kind of program, showing how their best uses can be obtained. References to the codebase will be helpful, plus more uploads of libraries developed in the project. Some of the topics we are going to tackle are:

1. Neuron Networks and Artificial Intelligence in EAs.

2. Remote controlling of expert advisers.

3. Licencing and software encryption methods.

4. Trade result processing and data management.

5. Web functions, Automation and capabilities of MQL5.

6. Working with OpenCL and MQL5 cloud services in optimization.

7. Creating and implementing custom indicator libraries.

8. Creation and use of custom data types.

... more to come.

If this sounds like an interesting idea, please propose any relevant topic and/or issue your opinions on how or where to get started. 

Stay safe.

 

Just about the subject and the topic, and some information about what was done here in past -

----------------

Neural Network

Neural Network: discussion/development threads

  1. Better NN EA development thread with indicators, pdf files and so on.
  2. Better NN EA final thread 
  3. taking NEURAL NETWORKS to the NEXT LEVEL - very interesting thread
  4. Neural Networks thread (good public discussion)
  5. How to build a NN-EA in MT4: usefull thread for developers.
  6. Radial Basis Network (RBN) - As Fit Filter For Price: the thread 

Neural Network: Indicators and systems development

  1. Self-trained MA cross!: development thread for new generation of the indicators
  2. Levenberg-Marquardt algorithm: development thread

Neural Network: EAs

  1. CyberiaTrader EA: discussion thread and EAs' thread.
  2. Self learning expert thread with EAs' files here.
  3. Artificial Intelligence EAs threads: How to "teach" and to use the AI ("neuron") EA thread and Artificial Intelligence  thread
  4. Forex_NN_Expert EA and indicator thread.
  5. SpiNNaker - A Neural Network EA thread. 

Neural Network: The Books

  1. What to read and where to learn about Machine Learning (10 free books) - the post.

The article

CodeBase

 
Sergey Golubev:

Just about the subject and the topic, and some information about what was done here in past -

Thank you Sergy. That's a splendid start on our first topic. Let me hope for more positive feedback.

 

The project ready on MetaEditor5 and anyone is free to join. I will update a couple of starter files soon.

Keep coding!

 
More topics
1. Cloud licensing for products sold out of the mql community market.
2. Self_writing script
( What would you do with this one?

I would add it in my expert, so when I upload it to the market. If a bug is reported on my app, I can simply call the re-generator function through a filtered user telegram API function then look at the code from anywhere, anytime. A library on this would be helpful, with a couple of functions.)

https://www.mql5.com/en/code/260

SelfGenerator
SelfGenerator
  • www.mql5.com
This script generates a file with its own source code (solution of the classical program in MQL5). It may be useful to study programming and algorithms. - Where is the source code? - Just launch the script and you will get it in 'MQL5\Files'...
 
Hello.
This is an update on the Neural Network project started a couple of days ago, which can also be found in the 'public projects' tab of MetaEditor5.

I have done extensive research on methods and approaches that can be used to improve efficiency of the use of 'Artificial Intelligence' to improve trading results. If you are new in this whole topic of 'AI' and Neural Networks, I recomend you watch a couple of introductory videos online, including this one I found on vimeo: https://vimeo.com/19569529
 which is an implemetation of Neural Networks in c++, and will be helpful if you have some previous knowledge of c++, or advanced MQL programming skills using Object Oriented Programming. Check included files for the code of the above video and check out how it works using a c++ ide or compiler(I tested using code_blocks on windows, my linux crashed). 
Note: We are going to majorly use Object Oriented approach (OOP) in creating our Neural Network model, therefore you might want to consider taking a look at how OOP can be used in MQL5. 
I would also like to point out that my first programming language was MQL, but I'm still comfortable coding c++, despite of some minor changes in terminologies and implementations. You might understand the vimeo video, but that was meant to give you a brief preview of what we will be doing here.
You can follow along without any prior knowledge of any other programming language, except MQL. However, prior exposure to OPP will be helpful.

I know some of you might want to suggest that I make an article off this. But I feel that the best way to get a consept implemented is through communial effort. Thus you can join and contribute to the project through the shared project:

Enough with stories.
In MQL, there are restrictions that limit what a programmer can do in their code, but most of these restrictions have work-arounds if you are experienced enough. I'm glad that MetaQuotes have dedicated developers who focus on providing and improvind the standard library, which brings us to the focus of this piece: Data Types

There are a couple of basic data-types in MQL, including the common int, string, long/ulong, char/uchar and boolean(You might think of another). There also exists user-defined data types which are basically classes. The standard library ships Lists.mqh which contains the CList data type, and Arrays which focus on creation and manipulation of ArrayObjects. 

You can find a whole lot of articles explaining what each of these data types do, but I'd like to focus on a library found in the article:

https://www.mql5.com/en/articles/1334

A great thank you to the writer: Vasiliy Sokolov
for the article. His Library 'Dictionary.mqh' has an implemetation of both Arrays and Lists. Combining the strengths of both data-types to achieve possibly higher performance. You can read more in his article. Basically, we will use this library to create our Neural Network models, so please download it from the article link, and probably learn how it works.

There are different variations of Neural Networks, each specialized to its use case. For example: XOR Neuron Models are used to predict input values whose output will either be one value or another. A good example is a boolean prediction. The Neuron output might either be a 1 (representing yes) or a 0 (representing no). A practical approach for an MQL programmer might be to decide whether to trade(yes) or not to trade(no). Better still, to predict a buy (1) or a sell (0). There are many other use cases. But I'd like to outline whatever I've gathered on my journey in the next posst to keep this short.
Please learn how to use the the library 'Dictionary.mqh' as we will use it for this particular implementation.
 Till next time.
#StaySafe 

printf("Follow the developer progress and contribute on %s","Projects>> Ultimate_AI");


Neural Net in C++ Tutorial
Neural Net in C++ Tutorial
  • 2011.02.04
  • vimeo.com
Update: For a newer neural net simulator optimized for image processing, see http://neural2d.net. Update: For a beginner's introduction to the concepts and…
 
I lightly touched on XOR version of Neural Networks in my previous post. Turns out, Neurons have been modified to try and incorporate some form of 'memory' in Neural Networks. Recursive Neural Networks (RNN) simulate 'short-term memory'. This is particularly useful if you are looking to train a model on data that exhibit some form of pattern or trend in the short term. 
Another model, the 'Long Term - Short Term Memory'(LSTM) network model is able to store long term memory data, while being selective on what type of data to forget, and which ones to keep. A slightly simplified model of the LSTM is the Gated Reccurent Units (GRU's). 
LSTM's and GRU's are pretty promising models, and the terminal aim of this discussion should be possibly aimed at developing one of the two (or both) and (possibly) avail it in the codebase for other enthusiasts and programmers who might travel down this path.

The best way to get Neural Network models working and to understand their working mechanisms is through tinkering with them. I searched the forum and the codebase for some working models, but I found the working principles of most of them being less flexible, but the article based on the vimeo video I posted helped me to get started, but it was impossible to get the model to work in MQL, based on the article I mentioned in my previous post. The MQL implementation was difficult to follow along (sucks debugging someone else's code), and the Author seems to have gone 'ghosty'.

Please note that, while basic Neural Networks do not have 'memory' capacities; and treat each input independent or previous inputs; they are still good at what they do. I therefore wish to start by getting a Basic Neural model working first, -which we might end up combining with our final library too- then progress to a GRU or an LSTM network.

The neural model should be flexible and diverse enough to allow the creation of different configurations of the network.

LSTM's and GRU's are quite suited for a trade algo programmer. With the ability to store 'relevant' information on trends, patterns, repeated price movements and many other use cases, they should be the ultimate goal for Neural Networks in MQL, including a possible implementation of auto-optimization of already existing trading systems natively. There is already a bunch of videos that explain how the LSTM's work, so I'll skip that part in this series. 

Some help from you, through the public project will be highly appreciated.

My next update will be on the dev progress, which I'll refrain from posting here, until the project is complete. However, you can always follow along and contribute via the editor.
Peace.

LSTM's : https://www.youtube.com/watch?v=8HyCNIVRbSU
Illustrated Guide to LSTM's and GRU's: A step by step explanation
Illustrated Guide to LSTM's and GRU's: A step by step explanation
  • www.youtube.com
LSTM's and GRU's are widely used in state of the art deep learning models. For those just getting into machine learning and deep learning, this is a guide in...
 

single coder = Y amount of time to accomplish X goals. 
2 or more coder = number of coder multiply Y amount of time to accomplish XYZ.. goals.

good luck.

 

Nice.

Good luck!

 
Mohamad Zulhairi Baba:

single coder = Y amount of time to accomplish X goals. 
2 or more coder = number of coder multiply Y amount of time to accomplish XYZ.. goals.

good luck.

At least you know I'm not lazy.

 
Waldemar_MQL:

Nice.

Good luck!

You too.

123456
New comment
Reason: