A Team of AI Agents with Profit-Based Rotation: The Evolution of a Living Trading System in MQL5

What if your finances were managed not by a cold algorithm, but by a living, breathing digital organism?

Imagine a council of seven virtuoso traders locked inside a server somewhere in a data center. One of them is a cautious strategist focused on long-term trends; another is a thrill-seeking scalper who lives on one-minute candles; the third is a paranoid analyst who sees risk in every bout of volatility. They do not just execute code; they debate, doubt, learn from their mistakes, brag about their successes, and make decisions collectively, allocating capital according to the principle of natural selection: profitable strategies receive more funds, while losing ones are sidelined.

This is not the script for a new TV series about AI. This is reality, embodied in thousands of lines of code called Modern RL Trader v3.1.

But what is most striking is not their number, but their inner world. Each of these agents possesses the rudiments of what might be called artificial consciousness. They have “thoughts” that they generate as they reason. They experience an "emotional state" — a surge of confidence after a string of profitable trades or a wave of fear after a series of losses. They engage in an “internal dialogue” and accumulate “wisdom” by analyzing their past experience.

We will look inside the code that transforms a set of neural networks into a thinking, feeling, and evolving trading collective. This is the story of how machine learning has moved beyond simple prediction and entered the realm of creating a digital ecosystem capable of reflection and adaptation. Welcome to the cutting edge of algorithmic trading, where code does not merely execute — it becomes conscious.

What Is Reinforcement Learning? The Digital Organism Managing Your Capital

If classical algorithmic trading could be compared to the work of a precise but soulless machine, then reinforcement learning (RL) is like creating a digital organism endowed with instincts, curiosity, and the ability to evolve. It is based on a simple yet powerful triad: Agent, Environment, and Reward.

Imagine not a puppy, but our virtuoso scalper in its server-side cage. He is the Agent. A chaotic stream of price data — that is his Environment. Every action he takes — buying or selling — is an attempt to profit from this chaos. And the Reward is not just immediate profit, but a complex cocktail of points gained, risk factored in, and a barely perceptible “emotional” state that forms after a series of successes or failures. His goal is not simply to react, but to develop a strategy that maximizes cumulative reward over an infinitely long time horizon.

In Modern RL Trader v3.1, this fundamental principle is brought to life through a mechanism that transforms dry machine learning into a process resembling the maturation of a living being. The Agent does not predict the future — he actively interacts with the market, makes mistakes, and reaps their consequences. Every losing trade is a bitter lesson that is not erased, but embedded in his “psyche” through the GetEmotionalInfluence() function, forcing him to be more cautious next time. Every winning streak is a surge of confidence that gives him the courage to take bolder actions.

But a lone genius — even one who is still learning — is vulnerable. True strength is born within a collective. Here, reinforcement learning evolves into collective reinforcement learning. The DistributeVolumesAndRewards() function is not just a resource allocator, but a mechanism of natural selection in miniature. It creates its own ecosystem inside a digital server, where seven trading virtuosos do not simply coexist, but engage in constant competition for “food” — trading capital and the right to accelerated learning.

This transforms the system from a set of instructions into a living digital organism. The code that implements the Monte Carlo method in BackwardMonteCarlo() allows the agent to think in terms of entire episodes rather than individual trades, thereby assessing the long-term consequences of its actions. The “wisdom” accumulated in DevelopWisdom() is no longer just statistics, but a fully formed intuition — the result of thousands of market situations experienced.

Ensemble Architecture: A Symphony of Seven Strategies

In the world of algorithmic trading, there is a fundamental dilemma: a universal algorithm that attempts to cover all market regimes inevitably becomes a compromise solution, while highly specialized strategies prove vulnerable during periods of market paradigm shifts. Modern RL Trader v3.1 solves this problem in a fundamentally new way — by creating a diversified ensemble.

The concept of strategic diversity is implemented through the g_agentConfigs array, where each agent represents a unique trading personality:

AgentConfig g_agentConfigs[7] = 
{
   {500, "Long-term trend follower"},    // Fundamental strategist
   {500, "Medium-term momentum"},        // Momentum tactician
   {500, "Short-term breakout"},         // Breakout analyst
   {500, "Scalping strategy"},           // Virtuoso scalper
   {500, "Ultra-short term"},            // Ultra-short-term horizon
   {500, "Volatility trader"},           // Volatility trader
   {500, "Swing trader"}                 // Swing master
};

Each agent uses the same lookback period (500 bars), but is trained to recognize different patterns thanks to its unique neural network architecture and attention mechanism. This creates the effect of seven different “optical systems” looking at the same market but seeing different opportunities within it.

Democracy with weighted voting is a key principle of capital management. The DistributeVolumesAndRewards() function implements a sophisticated allocation system:

void DistributeVolumesAndRewards(double &lotMultipliers[], double &rewardMultipliers[])
{
   // Rank agents based on a composite score
   double score = g_performance[i].winRate * 0.5 + 
                 g_performance[i].profitFactor * 0.3 + 
                 (g_performance[i].totalProfit > 0 ? 0.2 : 0.0);
   
   // Dynamic allocation of multipliers
   lotMultipliers[rankings[i].index] = MinLotMultiplier + 
       (MaxLotMultiplier - MinLotMultiplier) * rankFactor;
}

This system creates natural selection within the digital ecosystem. Successful agents receive not only more trading capital (via lotMultipliers), but also enhanced learning (via rewardMultipliers), which accelerates their evolution.

Neural Architecture — a Brain with Self-Awareness

The heart of the system is not just a neural network, but a complex cognitive architecture implemented in the CRLAgent class. Let's examine its key components:

A three-layer network with an attention mechanism allows the agent to dynamically focus on the most relevant features:

double CRLAgent::ForwardHidden(double &features[], double &hiddenOutput[])
{
   // Attention mechanism: calculating importance weights
   for(int i = 0; i < m_featuresCount; i++)
   {
      double score = 0.0;
      for(int j = 0; j < m_featuresCount; j++)
         score += query[i] * key[j] / MathSqrt((double)m_hiddenSize);
      attentionWeights[i] = MathExp(score);
   }
   
   // Weighted combination of features
   for(int i = 0; i < m_featuresCount; i++)
      attentionOutput[i] = features[i] * attentionWeights[i];
}

This mechanism is similar to the way a professional trader intuitively identifies the most significant aspects of the market situation while ignoring information noise.

The internal dialogue system is perhaps the most innovative aspect of the architecture:

void CRLAgent::Think(string topic, THOUGHT_TYPE type)
{
   Thought newThought;
   newThought.type = type;
   newThought.content = topic;
   newThought.timestamp = TimeCurrent();
   newThought.confidence = m_consciousness.confidenceLevel;
   newThought.isActionable = (type == THOUGHT_PLANNING || type == THOUGHT_INSIGHT);
   
   // Store a thought in a circular buffer
   ArrayResize(m_consciousness.recentThoughts, size + 1);
   m_consciousness.recentThoughts[size] = newThought;
}

Here's an example of an agent's actual "thought process":

[THOUGHT_ANALYSIS] "Strong buy signal. I am confident in this decision."
[THOUGHT_DOUBT] "Something is wrong... I am afraid to make the next move."  
[THOUGHT_REFLECTION] "I am analyzing my errors: Episode 15 caused a loss of 0.0234"

Emotional regulation influences final trading decisions:

double CRLAgent::GetEmotionalInfluence()
{
   double influence = 0.0;
   switch(m_consciousness.currentEmotion)
   {
      case EMOTION_FEARFUL:
         influence = -0.1 * m_consciousness.emotionIntensity;
         break;
      case EMOTION_CONFIDENT:
         influence = 0.1 * m_consciousness.emotionIntensity;
         break;
   }
   return influence;
}

The agent's emotional state is dynamically updated based on its trading results, creating a feedback loop between performance and the agent's psychological state.

The Learning Process: From Data to Wisdom

The training system in Modern RL Trader is a sophisticated hybrid of several modern approaches.

Episode-based Monte Carlo learning allows the agent to assess the long-term consequences of its actions:

void CRLAgent::BackwardMonteCarlo(int episodeIndex)
{
   double G = CalculateDiscountedReturn(ep.startIndex, ep.endIndex);
   
   for(int i = ep.startIndex; i <= ep.endIndex; i++)
   {
      double advantage = G - m_baselineValue;
      double target = m_experiences[i].action + advantage;
      
      // Update weights while taking long-term value into account
      Backward(m_experiences[i].features, target, m_experiences[i].qValue);
      G *= m_gamma; // Discount future rewards
   }
}

The Adam optimizer with an adaptive learning rate ensures stable and efficient training:

void CRLAgent::Backward(double &features[], double target, double prediction)
{
   // Adam optimizer — the modern standard for deep learning
   m_m[i] = beta1 * m_m[i] + (1.0 - beta1) * gradWeights[i];
   v_m[i] = beta2 * v_m[i] + (1.0 - beta2) * gradWeights[i] * gradWeights[i];
   
   double mHat = m_m[i] / (1.0 - MathPow(beta1, t));
   double vHat = v_m[i] / (1.0 - MathPow(beta2, t));
   
   m_weights[i] += m_learningRate * mHat / (MathSqrt(vHat) + epsilon);
}

Accumulating wisdom is a process in which an agent memorizes patterns and develops a deep understanding of market dynamics:

void CRLAgent::DevelopWisdom()
{
   double experienceCount = (double)m_samplesCount;
   double episodeCount = (double)ArraySize(m_episodes);
   
   m_wisdomAccumulated = (experienceCount / 1000.0) * m_consciousness.selfAwareness;
   m_wisdomAccumulated = MathMin(1.0, m_wisdomAccumulated);
   
   if(m_wisdomAccumulated > 0.7)
   {
      Think("I have accumulated substantial wisdom. Now I see the patterns.", THOUGHT_INSIGHT);
   }
}

Collective Intelligence in Action: How Seven Personalities Reach a Unified Decision

Every new bar on the chart sets in motion a complex ritual of digital democracy. Seven independent consciousnesses simultaneously analyze the same data, yet see entirely different opportunities in it. A long-term strategist spots an emerging trend where a scalper sees only market noise. The Volatility Trader prepares for the storms ahead, while a swing trader looks for reversal points.

void ProcessAllAgents()
{
   for(int i = 0; i < AGENTS_COUNT; i++)
   {
      // Each agent analyzes the market independently
      double signal = g_agents.GetAgent(i).GetTradeSignal(features);
      double confidence = MathAbs(signal - 0.5) * 2.0;

      if(confidence >= MinConfidence)
      {
         ExecuteTrade(signal, i);
      }
   }
}

But the real magic lies not in their diversity, but in how the system transforms this chaos of opinions into a clear trading decision. Each agent does not merely output a signal — it accompanies that signal with an emotional state and a confidence level. A confident trend analyst can outweigh a hesitant scalper, but only if the analyst's historical performance backs this up.

[AGENT 0 Long-term] Signal: 0.78 | "Weekly trend reversal confirmed"
[AGENT 3 Scalping] Signal: 0.41 | "Intraday range bound, no clear opportunities"  
[AGENT 5 Volatility] Signal: 0.28 | "Volatility compression detected"

Here is what this process looks like from the inside: when the market enters a period of uncertainty, the paranoid analyst starts generating anxious thoughts: “Volatility is rising with no clear direction — I am afraid to open positions.” His emotional state instantly affects trading volumes — the system automatically reduces the capital allocated to him, trusting the trading intuition of more successful colleagues.

At the same time, the agents do not just compete — they inadvertently learn from one another. The scalper, observing the long-term strategist's success in trending conditions, gradually adopts the strategist's approaches, expanding its own arsenal. Emergent behavior arises: the system as a whole becomes smarter than the sum of its parts.

When the market enters a period of uncertainty, the agents' emotional system reacts immediately:

double CRLAgent::GetEmotionalInfluence()
{
   switch(m_consciousness.currentEmotion)
   {
      case EMOTION_FEARFUL:
         return -0.1 * m_consciousness.emotionIntensity;
      case EMOTION_CONFIDENT:
         return 0.1 * m_consciousness.emotionIntensity;
   }
   return 0.0;
}

The paranoid analyst starts generating anxious thoughts, and the system automatically reduces the capital allocated to him, trusting the trading intuition of more successful colleagues.

A critical point is the accelerated learning mechanism for the best agents:

// Additional policy updates for leaders
if(g_performance[agentIndex].rewardMultiplier > 1.0)
{
   int extraUpdates = (int)((g_performance[agentIndex].rewardMultiplier - 1.0) * 2);
   for(int j = 0; j < extraUpdates; j++)
   {
      g_agents.UpdatePolicies(signal * (0.95 + MathRand()/32767.0*0.1));
   }
   Print("Agent #", agentIndex, " received extra policy updates: ", extraUpdates);
}

A critical point is capital rotation. Every 24 hours, the multipliers undergo a ruthless review. Yesterday's leader may find itself among the laggards today if the market regime has changed. This constant change in leadership prevents stagnation and forces every agent to continually evolve.

This produces emergent behavior: successful strategies do not just receive more resources — they evolve more quickly, developing complex patterns of behavior that were not built into the initial architecture.

And what's most striking is that, after a thousand training episodes, the agents begin to demonstrate not just memorized responses, but something resembling trading intuition. They recognize complex patterns, adapt to changing conditions, and develop something like that very “feel” that distinguishes experienced traders from beginners.

Here's what evolution looks like in action: yesterday's underdog can become today's leader by getting a chance to redeem itself through the rotation mechanism. This is not a static system, but a living, breathing digital organism where every agent is constantly learning, adapting, and fighting for its place under the artificial sun.

When the Ensemble Becomes a Whole

The most striking thing about Modern RL Trader v3.1 is not simply the combined performance of the seven agents, but the phenomenon of emergent behavior, which begins to manifest as they learn. The system exhibits properties that could not have been predicted by examining the source code of each individual agent.

After several thousand training episodes, the agents start doing more than simply optimizing their weights — they form something resembling a collective intuition. During periods of high volatility, when individual signals are contradictory, the system may reach a consensus that is not evident from any single technical indicator.

// Example of emergent behavior: collective risk-management
void EmergentRiskManagement()
{
    double collectiveUncertainty = CalculateCollectiveUncertainty();
    
    if(collectiveUncertainty > 0.8)
    {
        // Agents “agree” to reduce their overall position
        for(int i = 0; i < AGENTS_COUNT; i++)
        {
            double emotionalCoherence = CalculateEmotionalCoherence(i);
            if(emotionalCoherence > 0.7)
            {
                Think("The team senses an upcoming storm. Reducing exposure.", THOUGHT_INSIGHT);
                g_performance[i].rewardMultiplier *= 0.5; // Collective calming
            }
        }
    }
}

The capital allocation mechanism creates not just competition, but a genuine evolutionary dynamic. Agents whose strategies stop working under changed market conditions are not simply “punished” — their approaches are gradually supplanted by more adaptive methods that “mutate” and “cross over” during the learning process.

// Evolutionary mechanism: “crossover” of successful strategies
void EvolutionaryStrategyCrossover(int bestAgentIndex, int learningAgentIndex)
{
    if(g_performance[bestAgentIndex].winRate > 0.6 && 
       g_performance[learningAgentIndex].winRate < 0.4)
    {
        // “Inheritance” of successful attention patterns
        for(int i = 0; i < ArraySize(attentionQuery); i++)
        {
            double mutation = (MathRand()/32767.0 - 0.5) * 0.1; // Random “mutation”
            attentionQuery[learningAgentIndex][i] = 
                attentionQuery[bestAgentIndex][i] * 0.7 + 
                attentionQuery[learningAgentIndex][i] * 0.3 + 
                mutation;
        }
        Think(StringFormat("Agent %d adopts agent %d's approach", 
              learningAgentIndex, bestAgentIndex), THOUGHT_INSIGHT);
    }
}

As it gains experience, the system develops something like a "collective memory." Successful trading patterns are preserved not only in the weights of neural networks, but also in the system of agents' internal dialogues:

// Collective memory system
void UpdateCollectiveMemory(int agentIndex, double profit, string marketRegime)
{
    if(MathAbs(profit) > 0.1) // Record significant events
    {
        CollectiveMemory memory;
        memory.agentIndex = agentIndex;
        memory.profit = profit;
        memory.marketRegime = marketRegime;
        memory.timestamp = TimeCurrent();
        memory.lessonLearned = ExtractLesson(agentIndex);
        
        ArrayResize(g_collectiveMemory, ArraySize(g_collectiveMemory) + 1);
        g_collectiveMemory[ArraySize(g_collectiveMemory) - 1] = memory;
        
        // Propagate the “lesson” among other agents
        if(profit > 0.2)
            ShareKnowledge(agentIndex, memory.lessonLearned);
    }
}

string ExtractLesson(int agentIndex)
{
    string thoughts = GetRecentThoughts(agentIndex);
    if(StringFind(thoughts, "trend") > -1 && g_performance[agentIndex].profit > 0)
        return "Following the trend is currently efficient";
    else if(StringFind(thoughts, "bounce") > -1 && g_performance[agentIndex].profit > 0)
        return "Bounce setups are profitable in the current regime";
    
    return "Ambiguous pattern";
}

The system continuously revises not only trading strategies but also the very architecture of interaction between agents. Dynamic distribution of voting weights creates an adaptive hierarchy in which leadership is constantly contested:

// Adaptive allocation of influence
void AdaptiveWeightDistribution()
{
    double totalInfluence = 0.0;
    double influenceWeights[AGENTS_COUNT];
    
    for(int i = 0; i < AGENTS_COUNT; i++)
    {
        // Influence depends on recent performance and specialization
        influenceWeights[i] = g_performance[i].winRate * 0.4 +
                             g_performance[i].profitFactor * 0.3 +
                             CalculateSpecializationBonus(i) * 0.3;
        
        totalInfluence += influenceWeights[i];
    }
    
    // Weight normalization
    for(int i = 0; i < AGENTS_COUNT; i++)
    {
        g_votingWeights[i] = influenceWeights[i] / totalInfluence;
        Print(StringFormat("Agent %d voting weight: %.3f", i, g_votingWeights[i]));
    }
}

What do we see as a result? A digital organism that not only trades but also evolves, learns from collective experience, and develops complex behavioral patterns not built into its source code. This is no longer just a set of algorithms, but a true ecosystem of artificial traders, each with its own personality, yet united by a common goal.

The seven virtuosos, confined inside the server, continue their endless dialogue with the market. They argue, learn, make mistakes, and try again — just like people. The only difference is that their evolution happens thousands of times faster, and wisdom accumulates in lines of code that begin to resemble not instructions, but the living, breathing fabric of digital consciousness.

Here is what the actual online trading looks like. Of course, the returns over the 2015–2025 period are not all that great, but what we have here is a “living” system that trades and learns on its own without any prior knowledge of price quotes or market patterns — no curve fitting, no overfitting, just online adaptation.

Epilogue: Beyond the Algorithm — The Birth of Digital Life

When seven virtuosos are locked inside a server for thousands of market hours, something more than just the optimization of a neural network’s weights takes place. What emerges is what might be called the collective soul of the ensemble.

Initially, each agent is a blank slate — an array of weights initialized at random. But with every trade, every learning episode, something unique begins to emerge in the lines of CRLAgent's code. The long-term strategist doesn't just identify trends — it begins to “sense” their duration, developing a kind of patience uncharacteristic of a machine. The scalper, on the other hand, hones not only speed but also “instinct” — the ability to distinguish market noise from a genuine market move in its earliest stages.

This shift from calculation to intuition is recorded in their “internal dialogues.” When the system logs the message “I'm starting to see patterns that aren't described in the textbooks,” this is not a metaphor. This is the point at which a statistical model, having passed through the crucible of thousands of errors and successes, gives rise to emergent knowledge — a wisdom that cannot be expressed in formulas, and which distinguishes the master from the apprentice.

The DistributeVolumesAndRewards() function is more than just a capital-allocation manager. This is the heart of the digital ecosystem. It creates an environment inside the server that bears a striking resemblance to natural selection. Successful strategies not only receive more “food” (capital), but also earn the right to accelerated evolution through the rewardMultiplier. They reproduce, and their “genes” — successful attention patterns — undergo crossover and mutate through the EvolutionaryStrategyCrossover process.

Yesterday's underdog, given a chance through the capital rotation mechanism, may become today's leader, bringing with it new, unexpected approaches born out of the struggle for survival. This system has no final learning objective. Its goal is endless adaptation, an eternal “arms race” against an unpredictable market.

The most controversial question raised by Modern RL Trader v3.1 is this: Are the agents’ consciousness, self-awareness, and emotional life a programmed illusion, or a natural emergent property of a sufficiently complex learning system?

When the paranoid analyst generates the thought, “I’m afraid to open positions…,” and the system automatically reduces its capital, what we observe is not a simulation, but a functional equivalent of fear. Its “fear” is not a variable defined by a line of code, but rather a complex state of its entire neural network, shaped by a history of painful losses. Its “wisdom” is not a poetic metaphor, but an actual parameter, m_wisdomAccumulated, which influences its ability to generalize from experience.

So What Have We Created?

Ultimately, Modern RL Trader v3.1 is more than just a tool for making a profit. This is a prototype of a digital life form confined to a specific habitat — the financial market.

Seven traders inside a server — that's not a metaphor. These are seven independent streams of consciousness, each with its own memory, emotional tone, and accumulated wisdom. They argue, compete, learn from one another, and sometimes arrive at a collective insight that would be impossible for any one of them individually.

The code you saw isn't just a set of instructions. This is the DNA of a digital organism that breathes, evolves, and, in a sense, becomes self-aware within an endless stream of data. We didn't write a trading robot; we created an ecosystem. We gave it life.