Русский Português
preview
Community of Scientists Optimization (CoSO): Practice

Community of Scientists Optimization (CoSO): Practice

MetaTrader 5Trading |
178 0
Andrey Dik
Andrey Dik

Contents

  1. Implementation of the algorithm
  2. Test results
  3. Summary


Implementation of the algorithm

Let us continue the description of the algorithm implementation that we started in the first part of the article. Here we will present the results of experimental testing on specialized test functions.

The AssignFunds function describes the mechanism for distributing available "funds" among researchers in the CoSO algorithm. These resources represent opportunities for learning, moving, or participating in further processes, and their distribution influences the dynamics of the evolution of solutions. The function accepts one parameter: availableFunds - the total amount of funds to be distributed. The process is divided into several stages.

Funds are split into two parts. outsiderFunds is calculated, which is a portion of funds intended for "outsiders". This amount is calculated as a fraction of the total availableFunds, where omegaCurrent (a parameter that controls the share of funds for outsiders) is a coefficient. The remaining funds, stored in existingFunds, are allocated to active researchers.

    Distribution of funds among existing researchers. This part is executed only if a "global report" (globalReport) exists. If its size is greater than zero, totalRank is calculated - the sum of all ranks in arithmetic progression from 1 to reportSize. Essentially, it is the sum of the weighting factors that will be used to proportionally distribute the funds. The funds (existingFunds) are then distributed one at a time: for each fund, a random number is generated multiplied by the totalRank, and this will be used to select the researcher. Iteration over globalReport occurs.

    For each record in globalReport a cumulative cumSum is calculated, to which a "weight" equal to (reportSize - i) is added. This means that researchers with higher rankings in the report are given more weight and therefore have a better chance of receiving funds. If the random number falls within the interval associated with the current researcher (i.e. rnd <= cumSum), then the amount of funds of this researcher is increased by one. After this, the distribution cycle of the current fund is interrupted, and they move on to the distribution of the next fund.

    Creating outsiders. Finally, the CreateOutsiders function is called, passing outsiderFunds. This function uses dedicated "outsider" funds to create new researchers, or in other words, to initiate new research paths.

      AssignFunds thus implements a grant management strategy that rewards well-established researchers (through existingFunds and rank-based allocation) while simultaneously stimulating the emergence of new ideas or agents (through outsiderFunds).

      //————————————————————————————————————————————————————————————————————
      void C_AO_CoSO::AssignFunds (int availableFunds)
      {
        // Funds for outsiders
        int outsiderFunds = (int)(availableFunds * omegaCurrent);
        int existingFunds = availableFunds - outsiderFunds;
      
        int reportSize = ArraySize (globalReport);
      
        // Distribute funds to existing researchers
        if (reportSize > 0)
        {
          int totalRank = reportSize * (reportSize + 1) / 2;
      
          for (int f = 0; f < existingFunds; f++)
          {
            double rnd = u.RNDprobab () * totalRank;
            double cumSum = 0;
      
            for (int i = 0; i < reportSize; i++)
            {
              cumSum += reportSize - i;
              if (rnd <= cumSum)
              {
                researchers [globalReport [i].index].m++;
                break;
              }
            }
          }
        }
      
        // Create outsiders
        CreateOutsiders (outsiderFunds);
      }
      //————————————————————————————————————————————————————————————————————
      

      The CreateOutsiders function describes the process of creating new "researchers" within the CoSO model, using resources allocated specifically for "outsiders". These new researchers represent potential diversity or new ideas introduced into the system. The function accepts one parameter: outsiderFunds - the amount of funds intended for creating outsiders. If outsiderFunds is less than or equal to zero, the function aborts immediately because there are no funds to create outsiders.

      Determining the number of new researchers. maxNew is set: the maximum number of new outsiders that can be created at the current time. This value depends on the current population size. If the population is large (more than 100 elements), then maxNew is equal to 2 (to slow down growth). Otherwise, maxNew is 5. This is a mechanism for controlling population growth. The actual number of new researchers that will be created is determined. It is chosen randomly in the range from one to the minimum value of outsiderFunds and maxNew. This ensures that the number of new researchers does not exceed the available funds and the established limit, and then the amount of funds each new researcher will receive is calculated (simply dividing the total outsiderFunds by newResearchers).

      The cycle of creating new researchers. The function creates newResearchers in a loop a number of times: first, it searches for a "free" space in the existing array of researchers. If free space is found (idx != -1), it is used. If there is no free space, but the current population size is less than the maximum allowed size (maxPopSize), then a new space is added to the end of the array (idx = actualPopSize).

      Expanding the array (if necessary). If there is no free space and "actualPopSize" is already equal to or greater than "maxPopSize", then check to see if the size of the researcher array can be increased. If actualPopSize already reaches maxPopSize, a new maximum size (newMaxSize) is calculated as maxPopSize + 50, but not more than 500. This is another mechanism for limiting population growth. If the absolute limit of 500 is reached, the current iteration is skipped (no new outsiders are created). Otherwise, maxPopSize is updated to newMaxSize. After checks, idx is set to actualPopSize to use the first new location. If for some reason idx is still -1 after all attempts to find or create a place, then creation of the current researcher is skipped.

      Initializing a new researcher. If idx location is successfully found or created, researchers [idx]. alive is set to 'true', making the researcher active, and the amount of funds or "motivation" is set to fundsPerNew, and "strength" or the initial value of the other parameter is initialized to a random number.

      Initialization of coordinates (position in solution space). For each c coordinate ('coords' is the number of dimensions), the position is initialized to a random number in the given range. Then researchers [idx]. x [c] is adjusted for the sampling step using the SeInDiSp function and the "best" position, initially equal to the current one, is also set to researchers [idx]. x [c]. The rate or vector of change is initialized by a random number taken from a normal distribution with given parameters.

      Initialization of journal probabilities (publication journals). For each journal "j" (journalsNum — number of journals), the probability of publication in journal j is initialized to a random number. Then NormalizeProbabilities is called on researchers[idx]. rho to ensure that the sum of all rho for this researcher is equal to one.

      Updating population size. If a new explorer was added to the end of the array, then actualPopSize is updated to idx + 1.

        Thus, CreateOutsiders dynamically adds new researchers to the population, randomly initializing them in the solution space and distributing allocated funds among them. The maxNew and maxPopSize limiting mechanisms serve to control the population size, preventing it from growing indefinitely.

        //————————————————————————————————————————————————————————————————————
        void C_AO_CoSO::CreateOutsiders (int outsiderFunds)
        {
          if (outsiderFunds <= 0) return;
        
          // Limit the number of new outsiders, especially if the population is already large
          int maxNew = (actualPopSize > 100) ? 2 : 5;
          int newResearchers = (int)(u.RNDfromCI (1, MathMin (outsiderFunds, maxNew)));
          int fundsPerNew = outsiderFunds / newResearchers;
        
          for (int i = 0; i < newResearchers; i++)
          {
            // Find free space
            int idx = -1;
            for (int j = 0; j < actualPopSize; j++)
            {
              if (!researchers [j].alive)
              {
                idx = j;
                break;
              }
            }
        
            if (idx == -1 && actualPopSize < maxPopSize)
            {
              idx = actualPopSize;
            }
        
            if (idx == -1) // No space, expand the array
            {
              if (actualPopSize >= maxPopSize)
              {
                // Limit population growth
                int newMaxSize = MathMin (maxPopSize + 50, 500);
                if (newMaxSize == maxPopSize) continue; // Limit reached, skip creation
        
                maxPopSize = newMaxSize;
                ArrayResize (researchers, maxPopSize);
                for (int j = actualPopSize; j < maxPopSize; j++)
                {
                  researchers [j].Init (coords, journalsNum);
                  researchers [j].alive = false;
                }
                idx = actualPopSize;
              }
            }
        
            if (idx == -1) continue; // Failed to create
        
            researchers [idx].alive = true;
            researchers [idx].m = fundsPerNew;
            researchers [idx].s = u.RNDprobab ();
        
            // Random initialization
            for (int c = 0; c < coords; c++)
            {
              researchers [idx].x [c] = u.RNDfromCI (rangeMin [c], rangeMax [c]);
              researchers [idx].x [c] = u.SeInDiSp (researchers [idx].x [c], rangeMin [c], rangeMax [c], rangeStep [c]);
              researchers [idx].b [c] = researchers [idx].x [c];
              researchers [idx].v [c] = u.GaussDistribution (0.0, -0.01, 0.01, 1);
            }
        
            // Initialize journal probabilities
            for (int j = 0; j < journalsNum; j++)
            {
              researchers [idx].rho [j] = u.RNDprobab ();
            }
            NormalizeProbabilities (researchers [idx].rho);
        
            if (idx >= actualPopSize) actualPopSize = idx + 1;
          }
        }
        //————————————————————————————————————————————————————————————————————
        

        The HireResearchers function describes the process of "hiring" new researchers by an existing researcher who has sufficient grant funding. This is a mechanism for creating new elements (solutions) taking into account the characteristics of existing ones, which is a form of reproduction or diversification. The function takes one parameter: the index of the "supervisor" - the researcher who will hire new ones. If the researcher's index has less than or equal to one fund, the function immediately terminates execution. This means that sufficient funds are required for hiring.

        Separation of supervisor funds. The portion of the funds that the supervisor keeps for themselves is calculated. This is the total supervisor funds multiplied by researchers [idx]. s. The funds available for hiring are calculated. These are the remaining funds. The supervisor funds are being updated. If hireFunds is less than or equal to 0, then hiring is not possible and the function aborts.

        Determining the number of researchers to be hired. The maximum number of new researchers that can be hired is set. If the current population is large (more than 100), maxNew is 1. Otherwise, maxNew is 3. This is a mechanism for controlling population growth. The actual number of new researchers that will be created is determined. It is chosen randomly in the range from one to the minimum value of hireFunds and maxNew. The amount of funds each newly hired researcher will receive is calculated (simple division of hireFunds by newCount).

        The cycle of creating new researchers (recruitment). The function then iterates newCount times: first, it searches for a "free" place in the existing array of researchers. Free space is considered to be an index where the active flag is set to "false" and if free space is found, it is used. If there is no free space, but the current population size is less than the maximum allowed size, then a new slot is added to the end of the array. 

        Checking the possibility of hiring (when limits are reached). If there is no free space, and actualPopSize is already equal to or greater than maxPopSize, or actualPopSize is already equal to or greater than the absolute limit of 500, then creation of the current explorer is skipped (the loop goes to the next iteration or terminates). If newIdx is still -1, it means that no space can be found or created for the new explorer, and the current iteration of the loop is skipped.

        Initializing a new researcher. If the location is successfully found, researchers [newIdx]. alive is set to 'true', making the new researcher active, and the new researcher's funds are set to fundsPerNew. The new researcher's "propensity" parameter is initialized to a random number drawn from a Gaussian distribution. The mean for this distribution is taken from researchers[idx]. s (supervisor parameter) and means that the new researcher will be similar to his "supervisor". The value of 's' is limited to the range from "0" to "1".

        Inheriting characteristics from the supervisor. The best position or "knowledge" found by the new researcher is copied directly from the best position of the supervisor and allows new researchers to inherit the "experience".

        Initialization of position (coordinates). For each c coordinate (coords — number of dimensions), the new researcher's position is initialized as the supervisor's position, plus a random perturbation taken from a Gaussian distribution. This means that new researchers appear "near" their supervisor, but not in exactly the same place. The position is limited by the specified limits. Then, researchers [newIdx]. x [c] is adjusted for the sampling step using the SeInDiSp function, the rate or change vector is also randomly initialized from a normal distribution.

        Initialization of journal (publication) probabilities. For each j journal (journalsNum — number of journals), the probability of publication in j journal is initialized to a random number drawn from a Gaussian distribution. The mean for this distribution is taken from the supervisor probability. This allows new researchers to inherit the supervisor's journal preferences, but with some random variation. Negative rho values are truncated to 0. NormalizeProbabilities is then called for researchers[newIdx]. rho to ensure that the sum of all rho for this researcher is equal to one.

        Updating population size. If a new explorer was added to the end of the array (newIdx is equal to or greater than actualPopSize), then actualPopSize is updated to newIdx + 1.

          Thus, HireResearchers allows successful researchers to reproduce, creating new instances that inherit some of their characteristics, but with some random variation. This promotes exploration of neighboring areas in the search space and the spread of successful strategies, while controlling the overall population size.

          //————————————————————————————————————————————————————————————————————
          void C_AO_CoSO::HireResearchers (int idx)
          {
            if (researchers [idx].m <= 1) return;
          
            int keepFunds = (int)(researchers [idx].m * researchers [idx].s);
            int hireFunds = researchers [idx].m - keepFunds;
            researchers [idx].m = keepFunds;
          
            if (hireFunds <= 0) return;
          
            // Limit the number of people hired, especially in case of a large population
            int maxNew = (actualPopSize > 100) ? 1 : 3;
            int newCount = (int)(u.RNDfromCI (1, MathMin (hireFunds, maxNew)));
            int fundsPerNew = hireFunds / newCount;
          
            for (int i = 0; i < newCount; i++)
            {
              // Find free space
              int newIdx = -1;
              for (int j = 0; j < actualPopSize; j++)
              {
                if (!researchers [j].alive)
                {
                  newIdx = j;
                  break;
                }
              }
          
              if (newIdx == -1 && actualPopSize < maxPopSize)
              {
                newIdx = actualPopSize;
              }
          
              if (newIdx == -1) // No space
              {
                if (actualPopSize >= maxPopSize || actualPopSize >= 500) continue; // Skip creation when the limit is reached
              }
          
              if (newIdx == -1) continue; // Unable to find space
          
              researchers [newIdx].alive = true;
              researchers [newIdx].m = fundsPerNew;
              researchers [newIdx].s = u.GaussDistribution (researchers [idx].s, 0, 1, 1);
              if (researchers [newIdx].s < 0) researchers [newIdx].s = 0;
              if (researchers [newIdx].s > 1) researchers [newIdx].s = 1;
          
              // Inherit from a supervisor
              ArrayCopy (researchers [newIdx].b, researchers [idx].b, 0, 0, WHOLE_ARRAY);
          
              // Position near a supervisor
              for (int c = 0; c < coords; c++)
              {
                researchers [newIdx].x [c] = researchers [idx].x [c] + u.GaussDistribution (0.0, -0.01, 0.01, 1);
          
                // Boundary control
                if (researchers [newIdx].x [c] < rangeMin [c]) researchers [newIdx].x [c] = rangeMin [c];
                if (researchers [newIdx].x [c] > rangeMax [c]) researchers [newIdx].x [c] = rangeMax [c];
          
                researchers [newIdx].x [c] = u.SeInDiSp (researchers [newIdx].x [c], rangeMin [c], rangeMax [c], rangeStep [c]);
                researchers [newIdx].v [c] = u.GaussDistribution (0.0, -0.01, 0.01, 1);
              }
          
              // Perturbed journal probabilities
              for (int j = 0; j < journalsNum; j++)
              {
                researchers [newIdx].rho [j] = u.GaussDistribution (researchers [idx].rho [j], 0, 1, 1);
                if (researchers [newIdx].rho [j] < 0) researchers [newIdx].rho [j] = 0;
              }
              NormalizeProbabilities (researchers [newIdx].rho);
          
              if (newIdx >= actualPopSize) actualPopSize = newIdx + 1;
            }
          }
          //————————————————————————————————————————————————————————————————————
          

          The "ComputeStdDev" function is designed to calculate the standard deviation of the objective function values for all active researchers in the population. It is a measure of the dispersion of data around the mean, and in the context of optimization it may indicate the diversity of solutions in the current population.

          Calculating the mean value:
          • The variables "mean" (to accumulate the sum of the "f" values) and "count" (to count the number of active researchers) are initialized.
          • The function iterates over all researchers in the "researchers" array from "0" to actualPopSize - 1.
          • Inside the loop, it checks whether the researchers[i] is "active", since only active researchers should be taken into account in the calculations.
          • If the researcher is active, its 'f' value is added to "mean" and "count" is increased by 1.
          • After the first loop completes, if "count" is still 0 (no active researchers found), the function returns 0.
          • Finally, "mean" is divided by "count" to obtain the average 'f' value across all active researchers.
          Variance calculation:
          • The "variance" variable is initialized to 0.
          • The function again iterates over all the researchers in the "researchers" array.
          • The active researcher condition is tested again.
          • For each active researcher, the square of the difference between their 'f' value and the already calculated "mean" is calculated. This squared difference is added to the "variance".
          • After the second cycle is completed, "variance" is divided by "count". This gives the variance of the 'f' values for active researchers. 
          Calculate and return the standard deviation. The function returns the square root of the calculated "variance". This is the standard deviation.
          //————————————————————————————————————————————————————————————————————
          double C_AO_CoSO::ComputeStdDev ()
          {
            if (actualPopSize == 0) return 0;
          
            double mean = 0;
            int count = 0;
          
            for (int i = 0; i < actualPopSize; i++)
            {
              if (researchers [i].alive)
              {
                mean += researchers [i].f;
                count++;
              }
            }
          
            if (count == 0) return 0;
            mean /= count;
          
            double variance = 0;
            for (int i = 0; i < actualPopSize; i++)
            {
              if (researchers [i].alive)
              {
                variance += MathPow (researchers [i].f - mean, 2);
              }
            }
            variance /= count;
          
            return MathSqrt (variance);
          }
          //————————————————————————————————————————————————————————————————————
          

          The UpdateOmega function dynamically adjusts the omegaCurrent parameter, which is a share of outsiders or probability of choosing an outsider in the algorithm behavior. The purpose of this function is to adaptively control the exploration/exploitation strategy depending on the current state of the explorer population, namely, their dispersion in the search space.

          The function first calls ComputeStdDev() to determine currentSigma — the current standard deviation of the objective function values for active researchers. A high currentSigma means a large variety of solutions, while a low currentSigma means their convergence.

          Case 1: Population convergence (currentSigma < sigma0). If the current standard deviation is less than some predefined sigma0 threshold, this is a sign that researchers are starting to converge or cluster around certain solutions. In this case, the algorithm moves to the stage of exploitation (refinement) of the found solutions. To avoid premature convergence to a local optimum and to encourage further exploration and the search for better solutions, the share of outsiders (omegaCurrent) is increasing. The increase is calculated as (omegaMax - omegaMin) / 2.0 * epsilonPlus. Here omegaMax and omegaMin define the upper and lower bounds for omegaCurrent, and epsilonPlus is a positive coefficient that controls the rate of increase. The goal is to give more weight to elements that may stand out from the general trend or represent completely new approaches.

          Case 2: Population divergence or lack of convergence (currentSigma >= sigma0). If the standard deviation is greater than or equal to the sigma0 threshold, this means that the population is still highly distributed or is in the early stages of exploration. In this case, the share of outsiders (omegaCurrent) is decreasing. The reduction is calculated as (omegaMax - omegaMin) / 2.0 * epsilonMinus, where epsilonMinus is a negative coefficient that controls the rate of reduction. This is done to prevent the algorithm from wasting too many resources on random exploration when the population is already sufficiently diverse, and it can begin to focus on more promising areas.

          Range limit (omegaCurrent). After changing omegaCurrent, the function ensures that its value remains within the specified range, if omegaCurrent falls below omegaMin, it is set to omegaMin, and if it exceeds omegaMax, it is set to omegaMax, preventing the parameter from going beyond reasonable operating limits.

          In general, the UpdateOmega function implements an adaptive strategy. When a population of researchers shows signs of rapid convergence (low standard deviation), the algorithm increases the probability of selecting "outsiders" (or elements that may disrupt the current structure) to help avoid local optima and encourage new search or exploration of a wider space. Conversely, when the population is still widely distributed, the algorithm reduces the emphasis on outliers to focus on more systematic exploration or exploitation of already discovered promising areas. 

          //————————————————————————————————————————————————————————————————————
          void C_AO_CoSO::UpdateOmega ()
          {
            double currentSigma = ComputeStdDev ();
          
            if (currentSigma < sigma0)
            {
              // Increase the proportion of outsiders at convergence
              omegaCurrent += (omegaMax - omegaMin) / 2.0 * epsilonPlus;
            }
            else
            {
              // Reduce the share of outsiders
              omegaCurrent -= (omegaMax - omegaMin) / 2.0 * epsilonMinus;
            }
          
            // Limit the range
            if (omegaCurrent < omegaMin) omegaCurrent = omegaMin;
            if (omegaCurrent > omegaMax) omegaCurrent = omegaMax;
          }
          //————————————————————————————————————————————————————————————————————
          

          The NormalizeProbabilities function is designed to transform an array of numeric values "probs" so that they represent a correct probability distribution. That is, after executing the function, the sum of all elements in the "probs" array will be equal to one. The "sum" variable is initialized to 0. It will be used to accumulate the sum of all elements in the "probs" array. A loop is started that goes through all the elements of this array. Inside the loop, each value of probs[i] is added to the "sum" variable. 

          If the calculated "sum" is positive, a second loop is started, which also goes through all the elements of the "probs" array. Inside this loop, each element of the array is divided by "sum". As a result of this division, each value of "probs[i]" becomes a corresponding fraction of the total sum, and the sum of all elements in the array becomes equal to 1. This is a standard normalization procedure.

          If "sum" is 0 or negative, which is not typically expected for probabilities but is handled by the function, it means there is nothing to normalize the original values by. For example, all initial probabilities were zero. In this case, the function goes to "uniform distribution". "val" is calculated as (1.0 / size). This means that each element in the array will have the same probability so that the total sums to 1. The third loop is started, which goes through all the elements of the "probs" array. Inside this loop, each element is assigned the computed value of "val". Thus, each element of the array becomes equal to (1 / size), and their sum will be 1.

          This function is a general utility method for ensuring that a set of weights or "raw" probabilities can be used as a true probability distribution (where "true" means that the sum of all probabilities is 1). This is important in many algorithms, such as probability-based selection methods, roulette, or Monte Carlo methods, where proper normalization prevents errors and ensures correct statistical behavior. 

          //————————————————————————————————————————————————————————————————————
          void C_AO_CoSO::NormalizeProbabilities (double &probs [])
          {
            double sum = 0;
            int size = ArraySize (probs);
          
            for (int i = 0; i < size; i++)
            {
              sum += probs [i];
            }
          
            if (sum > 0)
            {
              for (int i = 0; i < size; i++)
              {
                probs [i] /= sum;
              }
            }
            else
            {
              // Even distribution
              double val = 1.0 / size;
              for (int i = 0; i < size; i++)
              {
                probs [i] = val;
              }
            }
          }
          //————————————————————————————————————————————————————————————————————
          

          The CompactPopulation function manages the size and composition of the research population in the algorithm. The main goal is to remove the least effective researchers and, if necessary, reduce the population to a manageable size to maintain the efficiency and performance of the algorithm.

          Counting "active" researchers. The aliveCount variable (number of survivors) is initialized to 0. Next, the loop goes through all the researchers in the current population (from 0 to actualPopSize - 1). For each researcher, its status is checked (researchers [i]. alive). If the explorer is "alive" (its 'alive' property is 'true'), the aliveCount is incremented by one.

            Condition for compaction/reduction. After counting the number of living researchers, the function checks whether compaction needs to be performed. This happens if one of two conditions is met: the number of active researchers aliveCount is less than 75% of the current total population size actualPopSize or if the current population size actualPopSize is greater than 200 (even if most of them are alive, the population is considered too large and needs to be reduced).

            The first phase of compaction (removal of "inactive"). If one of the conditions for compaction is met, the process begins. A new index is initialized to 0. This index will point to the next free space at the beginning of the "researchers" array where "active" researchers will move. A loop is started that iterates through all researchers (from 0 to actualPopSize - 1). Inside the loop, if the current explorer is "active", it is checked to see if it is already in the correct place, and if not, it means it needs to be moved forward in the array.

            The researcher is copied to the newIdx position. Usually, in such cases, (researchers [i]. alive = false) is set to mark the original researcher as "deleted" or inactive, even though it has already been copied. This is a cleanup. newIdx is incremented by one to point to the next location for the next active researcher. After this cycle is complete, all "living" researchers will be at the beginning of the "researchers" array. The population size is updated to aliveCount.

            The second phase of compaction (reduction of a large population). After the first phase (where inactives were removed and live ones were shifted), the function checks if actualPopSize still exceeds 150. This is a nested condition - it is met only if the population was already compactified or had many inactive researchers. If actualPopSize is greater than 150, it means that even after removing inactive ones, the population is still too large and needs to be further reduced to the maximum allowed size of 150. To do this, sorting by "fitness" is performed. After this, the researchers with the best "fitness" will be at the top of the array.

            Removing the "worst" researchers. All researchers starting from the 150th element (that is, those that ended up in the "tail" after sorting) are marked as "inactive" (researchers [i]. alive = false). Their 'm' parameter is also reset. Finally, actualPopSize is set to 150, effectively trimming the population to the desired maximum size. This function ensures that the population:
            1. Does not accumulate unused individuals that can take up memory and slow down processing.
            2. It does not grow uncontrollably, which can also negatively impact productivity and efficiency, and lead to early paralysis.
            3. Maintains a certain level of quality by weeding out the worst individuals when the population size becomes critical.
            //————————————————————————————————————————————————————————————————————
            void C_AO_CoSO::CompactPopulation ()
            {
              // Count living researchers
              int aliveCount = 0;
              for (int i = 0; i < actualPopSize; i++)
              {
                if (researchers [i].alive) aliveCount++;
              }
            
              // If there are too many dead, compactify
              if (aliveCount < actualPopSize * 0.75 || actualPopSize > 200)
              {
                int newIdx = 0;
                for (int i = 0; i < actualPopSize; i++)
                {
                  if (researchers [i].alive)
                  {
                    if (i != newIdx)
                    {
                      // Copy the living researcher to a new location
                      researchers [newIdx] = researchers [i];
                      researchers [i].alive = false;
                    }
                    newIdx++;
                  }
                }
                actualPopSize = aliveCount;
            
                // If the population is still too large, limit it
                if (actualPopSize > 150)
                {
                  // Sort by 'fitness' and keep the best
                  for (int i = 0; i < actualPopSize - 1; i++)
                  {
                    for (int j = i + 1; j < actualPopSize; j++)
                    {
                      if (researchers [i].f < researchers [j].f)
                      {
                        S_Researcher temp = researchers [i];
                        researchers [i] = researchers [j];
                        researchers [j] = temp;
                      }
                    }
                  }
            
                  // Kill the worst
                  for (int i = 150; i < actualPopSize; i++)
                  {
                    researchers [i].alive = false;
                    researchers [i].m = 0;
                  }
                  actualPopSize = 150;
                }
              }
            }
            //————————————————————————————————————————————————————————————————————
            

            The remaining Revision method in the CoSO algorithm starts a loop that iterates through all individuals in the 'a' array, from index 0 to "aSize - 1." Inside the loop, for each a[i] individual, the if (a [i]. f > fB) condition is checked. If the value of the objective function of the current individual turns out to be greater than the current global best value, fB is updated to the new best value: fB = a[i]. f. The index of this best individual is saved. 

            After the loop that has iterated over all individuals in 'a' array completes, the if (bestIND != -1) condition is checked. This condition is 'true' if at least one individual was found whose objective function was better than the previous value of fB. If a new best individual has been found (bestIND is not equal to -1), the ArrayCopy function is called, which copies the 'c' parameters of the found best individual into the global cB array.

            The main purpose of this method is to maintain the current state of the globally best solution found by the algorithm at the given moment. In evolutionary algorithms like CoSO, the concept of "global best" is constantly monitored and updated as better solutions are found. This "global best" result is then used to guide the search. Essentially, this function is an implementation of the "remember the best solution" step at each iteration of the algorithm.

            //————————————————————————————————————————————————————————————————————
            void C_AO_CoSO::Revision ()
            {
              int bestIND = -1;
              int aSize = ArraySize (a);
            
              for (int i = 0; i < aSize; i++)
              {
                if (a [i].f > fB)
                {
                  fB = a [i].f;
                  bestIND = i;
                }
              }
            
              if (bestIND != -1)
              {
                ArrayCopy (cB, a [bestIND].c, 0, 0, WHOLE_ARRAY);
              }
            }
            //————————————————————————————————————————————————————————————————————
            


            Test results

            The CoSO algorithm works quite well, with decent results. You can, of course, experiment with the parameters.
            CoSO|Community of Scientist Optimization|10.0|150.0|3.0|10.0|0.7|
            =============================
            5 Hilly's; Func runs: 10000; result: 0.8047081198587067
            25 Hilly's; Func runs: 10000; result: 0.5429326559833119
            500 Hilly's; Func runs: 10000; result: 0.30916988715342353
            =============================
            5 Forest's; Func runs: 10000; result: 0.7383405771205314
            25 Forest's; Func runs: 10000; result: 0.38224371519203115
            500 Forest's; Func runs: 10000; result: 0.20600693936217676
            =============================
            5 Megacity's; Func runs: 10000; result: 0.553846153846154
            25 Megacity's; Func runs: 10000; result: 0.2550769230769231
            500 Megacity's; Func runs: 10000; result: 0.11129230769230862
            =============================
            All score: 3.90362 (43.37%)

            This is the first time I have encountered such an unusual visualization of an algorithm, which is due to the multi-layered implementation logic.

            Hilly

            CoSO on the Hilly test function

            Forest

            CoSO on the Forest test function

            Megacity

            CoSO on the Megacity test function

            Based on the results, the CoSO algorithm is presented for informational purposes in the ranking table. I would like to point out that our table is becoming more compact, and results below 45% are already going beyond its limits.

            # AO Description Hilly Hilly
            Final
            Forest Forest
            Final
            Megacity (discrete) Megacity
            Final
            Final
            Result
            % of
            MAX
            10 p (5 F) 50 p (25 F) 1000 p (500 F) 10 p (5 F) 50 p (25 F) 1000 p (500 F) 10 p (5 F) 50 p (25 F) 1000 p (500 F)
            1 ANS across neighbourhood search 0.94948 0.84776 0.43857 2.23581 1.00000 0.92334 0.39988 2.32323 0.70923 0.63477 0.23091 1.57491 6.134 68.15
            2 CLA code lock algorithm (joo) 0.95345 0.87107 0.37590 2.20042 0.98942 0.91709 0.31642 2.22294 0.79692 0.69385 0.19303 1.68380 6.107 67.86
            3 AMOm animal migration ptimization M 0.90358 0.84317 0.46284 2.20959 0.99001 0.92436 0.46598 2.38034 0.56769 0.59132 0.23773 1.39675 5.987 66.52
            4 (P+O)ES (P+O) evolution strategies 0.92256 0.88101 0.40021 2.20379 0.97750 0.87490 0.31945 2.17185 0.67385 0.62985 0.18634 1.49003 5.866 65.17
            5 CTA comet tail algorithm (joo) 0.95346 0.86319 0.27770 2.09435 0.99794 0.85740 0.33949 2.19484 0.88769 0.56431 0.10512 1.55712 5.846 64.96
            6 TETA time evolution travel algorithm (joo) 0.91362 0.82349 0.31990 2.05701 0.97096 0.89532 0.29324 2.15952 0.73462 0.68569 0.16021 1.58052 5.797 64.41
            7 SDSm stochastic diffusion search M 0.93066 0.85445 0.39476 2.17988 0.99983 0.89244 0.19619 2.08846 0.72333 0.61100 0.10670 1.44103 5.709 63.44
            8 BOAm billiards optimization algorithm M 0.95757 0.82599 0.25235 2.03590 1.00000 0.90036 0.30502 2.20538 0.73538 0.52523 0.09563 1.35625 5.598 62.19
            9 AAm archery algorithm M 0.91744 0.70876 0.42160 2.04780 0.92527 0.75802 0.35328 2.03657 0.67385 0.55200 0.23738 1.46323 5.548 61.64
            10 ESG evolution of social groups (joo) 0.99906 0.79654 0.35056 2.14616 1.00000 0.82863 0.13102 1.95965 0.82333 0.55300 0.04725 1.42358 5.529 61.44
            11 SIA simulated isotropic annealing (joo) 0.95784 0.84264 0.41465 2.21513 0.98239 0.79586 0.20507 1.98332 0.68667 0.49300 0.09053 1.27020 5.469 60.76
            12 EOm extremal_optimization_M 0.76166 0.77242 0.31747 1.85155 0.99999 0.76751 0.23527 2.00277 0.74769 0.53969 0.14249 1.42987 5.284 58.71
            13 BBO biogeography based optimization 0.94912 0.69456 0.35031 1.99399 0.93820 0.67365 0.25682 1.86867 0.74615 0.48277 0.17369 1.40261 5.265 58.50
            14 ACS artificial cooperative search 0.75547 0.74744 0.30407 1.80698 1.00000 0.88861 0.22413 2.11274 0.69077 0.48185 0.13322 1.30583 5.226 58.06
            15 DA dialectical algorithm 0.86183 0.70033 0.33724 1.89940 0.98163 0.72772 0.28718 1.99653 0.70308 0.45292 0.16367 1.31967 5.216 57.95
            16 BHAm black hole algorithm M 0.75236 0.76675 0.34583 1.86493 0.93593 0.80152 0.27177 2.00923 0.65077 0.51646 0.15472 1.32195 5.196 57.73
            17 ASO anarchy society optimization 0.84872 0.74646 0.31465 1.90983 0.96148 0.79150 0.23803 1.99101 0.57077 0.54062 0.16614 1.27752 5.178 57.54
            18 RFO royal flush optimization (joo) 0.83361 0.73742 0.34629 1.91733 0.89424 0.73824 0.24098 1.87346 0.63154 0.50292 0.16421 1.29867 5.089 56.55
            19 AOSm atomic orbital search M 0.80232 0.70449 0.31021 1.81702 0.85660 0.69451 0.21996 1.77107 0.74615 0.52862 0.14358 1.41835 5.006 55.63
            20 TSEA turtle shell evolution algorithm (joo) 0.96798 0.64480 0.29672 1.90949 0.99449 0.61981 0.22708 1.84139 0.69077 0.42646 0.13598 1.25322 5.004 55.60
            21 BSA backtracking_search_algorithm 0.97309 0.54534 0.29098 1.80941 0.99999 0.58543 0.21747 1.80289 0.84769 0.36953 0.12978 1.34700 4.959 55.10
            22 DE differential evolution 0.95044 0.61674 0.30308 1.87026 0.95317 0.78896 0.16652 1.90865 0.78667 0.36033 0.02953 1.17653 4.955 55.06
            23 SRA successful restaurateur algorithm (joo) 0.96883 0.63455 0.29217 1.89555 0.94637 0.55506 0.19124 1.69267 0.74923 0.44031 0.12526 1.31480 4.903 54.48
            24 CRO chemical reaction optimization 0.94629 0.66112 0.29853 1.90593 0.87906 0.58422 0.21146 1.67473 0.75846 0.42646 0.12686 1.31178 4.892 54.36
            25 BIO blood inheritance optimization (joo) 0.81568 0.65336 0.30877 1.77781 0.89937 0.65319 0.21760 1.77016 0.67846 0.47631 0.13902 1.29378 4.842 53.80
            26 BSA bird swarm algorithm 0.89306 0.64900 0.26250 1.80455 0.92420 0.71121 0.24939 1.88479 0.69385 0.32615 0.10012 1.12012 4.809 53.44
            27 DEA dolphin_echolocation_algorithm 0.75995 0.67572 0.34171 1.77738 0.89582 0.64223 0.23941 1.77746 0.61538 0.44031 0.15115 1.20684 4.762 52.91
            28 HS harmony search 0.86509 0.68782 0.32527 1.87818 0.99999 0.68002 0.09590 1.77592 0.62000 0.42267 0.05458 1.09725 4.751 52.79
            29 SSG saplings sowing and growing 0.77839 0.64925 0.39543 1.82308 0.85973 0.62467 0.17429 1.65869 0.64667 0.44133 0.10598 1.19398 4.676 51.95
            30 BCOm bacterial chemotaxis optimization M 0.75953 0.62268 0.31483 1.69704 0.89378 0.61339 0.22542 1.73259 0.65385 0.42092 0.14435 1.21912 4.649 51.65
            31 ABO african buffalo optimization 0.83337 0.62247 0.29964 1.75548 0.92170 0.58618 0.19723 1.70511 0.61000 0.43154 0.13225 1.17378 4.634 51.49
            32 (PO)ES (PO) evolution strategies 0.79025 0.62647 0.42935 1.84606 0.87616 0.60943 0.19591 1.68151 0.59000 0.37933 0.11322 1.08255 4.610 51.22
            33 FBA Fractal-Based Algorithm 0.79000 0.65134 0.28965 1.73099 0.87158 0.56823 0.18877 1.62858 0.61077 0.46062 0.12398 1.19537 4.555 50.61
            34 TSm tabu search M 0.87795 0.61431 0.29104 1.78330 0.92885 0.51844 0.19054 1.63783 0.61077 0.38215 0.12157 1.11449 4.536 50.40
            35 BSO brain storm optimization 0.93736 0.57616 0.29688 1.81041 0.93131 0.55866 0.23537 1.72534 0.55231 0.29077 0.11914 0.96222 4.498 49.98
            36 WOAm wale optimization algorithm M 0.84521 0.56298 0.26263 1.67081 0.93100 0.52278 0.16365 1.61743 0.66308 0.41138 0.11357 1.18803 4.476 49.74
            37 AEFA artificial electric field algorithm 0.87700 0.61753 0.25235 1.74688 0.92729 0.72698 0.18064 1.83490 0.66615 0.11631 0.09508 0.87754 4.459 49.55
            38 AEO artificial ecosystem-based optimization algorithm 0.91380 0.46713 0.26470 1.64563 0.90223 0.43705 0.21400 1.55327 0.66154 0.30800 0.28563 1.25517 4.454 49.49
            39 CAm camel algorithm M 0.78684 0.56042 0.35133 1.69859 0.82772 0.56041 0.24336 1.63149 0.64846 0.33092 0.13418 1.11356 4.444 49.37
            40 ACOm ant colony optimization M 0.88190 0.66127 0.30377 1.84693 0.85873 0.58680 0.15051 1.59604 0.59667 0.37333 0.02472 0.99472 4.438 49.31
            41 CMAES covariance_matrix_adaptation_evolution_strategy 0.76258 0.72089 0.00000 1.48347 0.82056 0.79616 0.00000 1.61672 0.75846 0.49077 0.00000 1.24923 4.349 48.33
            42 BFO-GA bacterial foraging optimization - ga 0.89150 0.55111 0.31529 1.75790 0.96982 0.39612 0.06305 1.42899 0.72667 0.27500 0.03525 1.03692 4.224 46.93
            43 SOA simple optimization algorithm 0.91520 0.46976 0.27089 1.65585 0.89675 0.37401 0.16984 1.44060 0.69538 0.28031 0.10852 1.08422 4.181 46.45
            44 ABHA artificial bee hive algorithm 0.84131 0.54227 0.26304 1.64663 0.87858 0.47779 0.17181 1.52818 0.50923 0.33877 0.10397 0.95197 4.127 45.85
            45 ACMO atmospheric cloud model optimization 0.90321 0.48546 0.30403 1.69270 0.80268 0.37857 0.19178 1.37303 0.62308 0.24400 0.10795 0.97503 4.041 44.90
            CoSO community_of_scientist_optimization 0.80471 0.54293 0.30917 1.65681 0.73834 0.38224 0.20600 1.32658 0.55384 0.25507 0.11129 0.92020 3.904 43.37
            RW random walk 0.48754 0.32159 0.25781 1.06694 0.37554 0.21944 0.15877 0.75375 0.27969 0.14917 0.09847 0.52734 2.348 26.09


            Summary

            The CoSO algorithm demonstrates average results on the test functions, achieving 43% of the maximum possible performance. Of course, I expected more impressive results. The test stand offers an expanded set of functions, including standard and well-known ones, where anyone can further experiment with both parameter selection and function selection to better exploit the algorithm capabilities.

            The main drawback of the current implementation is its high computational complexity. The multi-layer architecture with logs, dynamic resource allocation and adaptive population management creates a significant load. The algorithm is noticeably slower than other population methods, which limits its applicability to problems requiring fast solutions.

            However, the conceptual basis of CoSO is of considerable interest. The modeling of the scientific community introduces unique mechanisms that have been applied: a dynamic population automatically adapts computing resources to the complexity of the problem, a journal system ensures efficient information exchange without losing the best solutions, competitive funding creates natural selection without strict rules, and a recruitment mechanism attempts to solve the local/global search dilemma.

            The potential for improvement is clear. CoSO should not be viewed as a ready-made solution, but as a promising research platform. The algorithm opens a new direction in evolutionary optimization, where the social mechanisms of human communities become a source of computational strategies. With proper development, CoSO can find its niche in tasks where adaptability and resilience to change are important, and computation time is not critical.

            tab

            Figure 1. Color-coded comparison of algorithms across tests

            chart

            Figure 2. Histogram of algorithm testing results (scale from 0 to 100, the higher the better, where 100 is the maximum possible theoretical result, in the archive there is a script for calculating the rating table)

            CoSO pros and cons:

            Pros:

            1. a good foundation for developing new variants of the algorithm.

            Cons:

            1. a large number of external parameters;
            2. a tendency to stagnate on some problems;
            3. slow.

            An archive with the latest versions of the algorithm code is attached to the article. The author of the article is not responsible for the absolute accuracy in the description of canonical algorithms. Changes have been made to many of them to improve search capabilities. The conclusions and judgments presented in the articles are based on the results of the experiments.


            Programs used in the article

            # Name Type Description
            1 #C_AO.mqh
            Include
            Parent class of population optimization algorithms
            2 #C_AO_enum.mqh
            Include
            Enumeration of population optimization algorithms
            3 TestFunctions.mqh
            Include
            Library of test functions
            4
            TestStandFunctions.mqh
            Include
            Test stand function library
            5
            Utilities.mqh
            Include
            Library of auxiliary functions
            6
            CalculationTestResults.mqh
            Include
            Script for calculating results in the comparison table
            7
            Testing AOs.mq5
            Script The unified test stand for all population optimization algorithms
            8
            Simple use of population optimization algorithms.mq5
            Script
            A simple example of using population optimization algorithms without visualization
            9
            Test_AO_CoSO.mq5
            Script CoSO test stand

            Translated from Russian by MetaQuotes Ltd.
            Original article: https://www.mql5.com/ru/articles/18935

            Attached files |
            CoSO.zip (257.36 KB)
            From Basic to Intermediate: Object Events (II) From Basic to Intermediate: Object Events (II)
            In this article, we will look at how the last three types of events generated by an object work. Understanding this will be very interesting, because in the end we will do something that may seem crazy to many people, but it is entirely possible and produces a very surprising result.
            Market Microstructure in MQL5 (Part 6): Order Flow Market Microstructure in MQL5 (Part 6): Order Flow
            This article adds six order-flow functions and a new OrderFlowAnalysis struct to MicroStructureFoundation.mqh: VPINOHLC, signed flow imbalance, trade intensity versus a 20-session baseline, a late-minus-early smart-money index, flow momentum, and a wrapper that outputs a confidence weight. Flow confidence is gated by noise and jump intensity from Parts 5 and 4. Calibrated on 602 NQ M1 NY sessions, it provides ready-to-use intraday flow signals with documented thresholds.
            Market Simulation: Getting Started with SQL in MQL5 (V) Market Simulation: Getting Started with SQL in MQL5 (V)
            In the previous article, I showed how to proceed in order to add a query mechanism. This was needed so that, inside MQL5 code, you could fully use SQL and retrieve results when executing a SQL SELECT ... FROM query. But there is still one last function we need to implement. This is the DatabaseReadBind function. Since understanding it properly requires a slightly more detailed explanation, it was decided to cover it not in the previous article, but in today's article. So, since the topic will be fairly extensive, let us proceed directly to the next section.
            How to Detect and Normalize Chart Objects in MQL5 (Part 3): Alerting and Automated Trading from Manually Drawn Objects How to Detect and Normalize Chart Objects in MQL5 (Part 3): Alerting and Automated Trading from Manually Drawn Objects
            This article extends the chart‑object detector into a modular monitoring and execution layer. It defines objective interaction rules (touch, cross, breakout) for trendlines, Fibonacci levels, channels, rectangles, and pitchforks, then routes events through an interaction detector, alert manager, and optional trade executor. Orders use object geometry for stop‑loss and take‑profit. The result is a reproducible pipeline that converts static drawings into actionable alerts and, if enabled, trades.