Discussing the article: "Developing a multi-currency Expert Advisor (Part 20): Putting in order the conveyor of automatic project optimization stages (I)"

 

Check out the new article: Developing a multi-currency Expert Advisor (Part 20): Putting in order the conveyor of automatic project optimization stages (I).

We have already created quite a few components that help arrange auto optimization. During the creation, we followed the traditional cyclical structure: from creating minimal working code to refactoring and obtaining improved code. It is time to start clearing up our database, which is also a key component in the system we are creating.

In this series of articles, we are trying to create an automatic optimization system that allows finding good combinations of parameters of one trading strategy without human intervention. These combinations will then be combined into one final EA. The objective is set in more detail in part 9 and part 11. The process of such a search itself will be controlled by one EA (optimizing EA), and all data that will need to be saved during its operation is set in the main database.

In the database, we have tables to store information about several classes of objects. Some have a status field that can take values from a fixed set of values ("Queued", "Process", "Done"), but not all classes use this field. More precisely, for now it is used only for optimization tasks (task). Our optimizing EA searches the task table (tasks) for the Queued tasks to select the next task to run. After each task is completed, its status in the database changes to Completed.

Let's try to implement status auto updates not only for tasks, but also for all other classes of objects (jobs, stages, projects) and arrange automatic execution of all necessary stages up to obtaining the final EA, which can work independently without connecting to the database.

Author: Yuriy Bykov

 

Tried to run your example.

I set groupId_ = 1.

But the expert is not created in the initialisation function. Here it comes out:

int OnInit() {
// Set parameters in the capital management class
   CMoney::DepoPart(expectedDrawdown_ / 10.0);
   CMoney::FixedBalance(fixedBalance_);

// Initialisation string with strategy parameter sets
   string strategiesParams = NULL;

// If the selected index of the strategy group from the library is valid, then
   if(groupId_ >= 0 && groupId_ < ArraySize(CGroupsLibrary::s_params)) {
      // Take the initialisation string from the library for the selected group
      strategiesParams = CGroupsLibrary::s_params[groupId_];
   }

// If the strategy group from the library is not specified, abort the operation
   if(strategiesParams == NULL) {
      return INIT_FAILED;
   }

// Prepare the initialisation string for an Expert Advisor with a group of several strategies
   string expertParams = StringFormat(
                            "class CVirtualAdvisor(\n"
                            "    class CVirtualStrategyGroup(\n"
                            "       [\n"
                            "        %s\n"
                            "       ],%f\n"
                            "    ),\n"
                            "    class CVirtualRiskManager(\n"
                            "       %d,%.2f,%d,%.2f,%.2f,%d,%.2f,%.2f,%d,%.2f,%d,%.2f,%.2f"
                            "    )\n"
                            "    ,%d,%s,%d\n"
                            ")",
                            strategiesParams, scale_,
                            rmIsActive_, rmStartBaseBalance_,
                            rmCalcDailyLossLimit_, rmMaxDailyLossLimit_, rmCloseDailyPart_,
                            rmCalcOverallLossLimit_, rmMaxOverallLossLimit_, rmCloseOverallPart_,
                            rmCalcOverallProfitLimit_, rmMaxOverallProfitLimit_, rmMaxOverallProfitDate_,
                            rmMaxRestoreTime_, rmLastVirtualProfitFactor_,
                            magic_, "SimpleVolumes", useOnlyNewBars_
                         );

   PrintFormat(__FUNCTION__" | Expert Params:\n%s", expertParams);

// Create an Expert Advisor that works with virtual positions
   expert = NEW(expertParams);

// If the Expert Advisor is not created, return an error
   if(!expert) return INIT_FAILED;

...
}


If we go deeper, the object is not created in CVirtualFactory():

class CVirtualFactory {
public:
   // Creating an object from the initialisation string
   static CFactorable* Create(string p_params) {
      // Read the object class name
      string className = CFactorable::ReadClassName(p_params);
      
      // Pointer to the object to be created
      CFactorable* object = NULL;

      // Depending on the class name, call the corresponding constructor
      if(className == "CVirtualAdvisor") {
         object = new CVirtualAdvisor(p_params);
...

Can you tell me what is the issue, and how to try your example in action after all?

Thanks.

 

Hello.

It may be that you have not completed the work on creating entries in the parameter library or even all the other necessary steps of automatic optimisation. For example, you can read more about the last step - filling the library - in Part 17.

It is possible, though, that the point is different. Ideally, you don't need a database to run the final EA. I will check the code and reply later.
 
I looked at the code for this article. Indeed, so far we have not reached the last stage in automation, where the final EA that is not linked to the optimisation database is automatically generated. This will be done in a future article. For now, preliminary execution of the first stages of the pipeline is necessary to get the results.
 

Hi Yuriy

I have used Google Translate to get me to Part 20. Google "Google Translate" and put it on a new tab in the browser. It will place an icon in the search bar at the far right. Load the page in its native language and press the icon to select the article language and the one to translate into it. Presto,I am at part 20! It does not do a perfect job but the translation is 99% useful.

I loaded your Archive Source into Excel and added a few columns to sort on to arrange the contents. In addition to sorting in Excel, the spreadsheet can be imported into an OutLook database directly


I am having problems identifying the starting article to establish the SQL database. I tried running Simple Volume Stage 1 and got a flat line which indicates to me that I probably need to backtrack and create another SQL data base. It would be extremely helpful to have a table of the order of executions of the necessary programs to get a working system. Perhaps you could add it to the Archive Source table.

Another tiny request is to use the <> option for include file specifications instead of "". I am keeping your system separate in my Experts and Include directories, #include <!!!! MultiCurrency\VirtualAdvisor.mqh>, so this change will make it easier to add the subdirectory specification/.

Thanks for your input

CapeCoddah

Files:
 

Hello.

About the initial filling of the database with information about the project, stages, works and tasks you can see in parts 13, 18, 19. This is not the main topic, so the information you need will be somewhere closer to the end of the articles. For example, in part 18:

Проверив, что в базу данных теперь корректно попадают результаты проходов, совершаемых с использованием форвард-периода, проведём более приближенный к реальным условиям тест работы автоматической оптимизации. Для этого в чистую базу данных добавим уже два этапа. На первом будет оптимизироваться одиночный экземпляр торговой стратегии, но только на одном символе и периоде (EURGBP H1) на промежутке 2018 - 2023 годов. Форвард-период на этом этапе использоваться не будет. На втором этапе будет оптимизироваться группа из хороших одиночных экземпляров, полученных на первом этапе. Теперь форвард период уже будет использоваться: под него отводится весь 2023 год.

Or in part 19:

Let's create four stages in the database for our project named "First", "Clustering passes from first stage", "Second", and "Second with clustering". For each stage we will create two works for EURGBP and GBPUSD symbols on the H1 timeframe. For the works of the first stage we will create three optimisation tasks with different criteria (complex, maximum profit and custom). For the rest of the works we will create one task each. We will take the period from 2018 to 2023 as the optimisation interval. For each job we will specify the correct values of input parameters.

Or you can wait for the next article, which will be devoted, among other things, to the issue of initial filling of the database with the help of an auxiliary script.

Switching to using the include folder for storing library files is in the plans, but it hasn't come to that yet.

 
Yuriy Bykov storing library in the plans, but so far it has not come to it.

many thanks

 

Hi Yuriy,

Have you submitted the next article or know when it will be published?

 
Hello, most likely in about a month.