Sentiment Analysis and Deep Learning for Trading with EA and Backtesting with Python

Sentiment Analysis and Deep Learning for Trading with EA and Backtesting with Python

MetaTrader 5Examples | 5 July 2024, 16:34
1 682 8
Javier Santiago Gaston De Iriarte Cabrera
Javier Santiago Gaston De Iriarte Cabrera


Integrating deep learning and sentiment analysis into trading strategies in MetaTrader 5 (MQL5) represents a sophisticated advancement in algorithmic trading. Deep learning, a subset of machine learning, involves neural networks with multiple layers that can learn and make predictions from vast and complex datasets. Sentiment analysis, on the other hand, is a natural language processing (NLP) technique used to determine the sentiment or emotional tone behind a body of text. By leveraging these technologies, traders can enhance their decision-making processes and improve trading outcomes.

For this article, we will integrate Python into MQL5 using a DLL shell32.dll, which executes what we need for Windows. By installing Python and running it through shell32.dll, we will be able to launch Python scripts from the MQL5 Expert Advisor (EA). There are two Python scripts: one to run the trained ONNX model from TensorFlow, and another script that uses libraries to fetch news from the internet, read the headlines, and quantify media sentiment using AI. This is one possible solution, but there are many ways and different sources to obtain the sentiment of a stock or symbol. Once the model and sentiment are obtained, if both values are in agreement, the order is executed by the EA.

Can we perform a test in Python to understand the results of combining sentiment analysis and deep learning? The answer is yes, and we will proceed to study the code.

Backtesting Sentiment Analysis with Deep Learning using Python

To perform the backtesting of this strategy, we will use the following libraries. I will use my other article as a starting point. Anyway, here I will also provide the required explanations.

We will use the following libraries:

import ccxt
import pandas as pd
import numpy as np
import onnx
import onnxruntime as ort
import matplotlib.pyplot as plt
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score, mean_absolute_percentage_error
from sklearn.model_selection import TimeSeriesSplit
from sklearn.preprocessing import MinMaxScaler
import requests
from datetime import datetime, timedelta
import nltk
from nltk.sentiment import SentimentIntensityAnalyzer
from newsapi import NewsApiClient

First of all, we ensure that nltk is updated.'vader_lexicon')

nltk (Natural Language Toolkit) is a library used for working with human language data (text). It provides easy-to-use interfaces to over 50 corpora and lexical resources, such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, as well as wrappers for industrial-strength NLP libraries.

Readers must adapt the python backtesting script to specify where to obtain data, news feed and data for ONNX models.

We will use the following to obtain the sentiment analysis:

def get_news_sentiment(symbol, api_key, date):
        newsapi = NewsApiClient(api_key=api_key)
        # Obtener noticias relacionadas con el símbolo para la fecha específica
        end_date = date + timedelta(days=1)
        articles = newsapi.get_everything(q=symbol,
        sia = SentimentIntensityAnalyzer()
        sentiments = []
        for article in articles['articles']:
            text = article.get('title', '')
            if article.get('description'):
                text += ' ' + article['description']
            if text:
                sentiment = sia.polarity_scores(text)
        avg_sentiment = np.mean(sentiments) if sentiments else 0
        return avg_sentiment
    except Exception as e:
        print(f"Error al obtener el sentimiento para {symbol} en la fecha {date}: {e}")
        return 0

For the backtest, we will use news-api as a feed, because their free API lets us get 1 month look-back of news. If you need more, you can buy a subscription.

The rest of the code will be to obtain the predictions from the ONNX model to predict next close prices. We will just compare the sentiment with the deep learning predictions, and if both conclude with same results, an order will be created. It looks like this:

investment_df = comparison_df.copy()
investment_df['price_direction'] = np.where(investment_df['prediction'].shift(-1) > investment_df['prediction'], 1, -1)
investment_df['sentiment_direction'] = np.where(investment_df['sentiment'] > 0, 1, -1)
investment_df['position'] = np.where(investment_df['price_direction'] == investment_df['sentiment_direction'], investment_df['price_direction'], 0)
investment_df['strategy_returns'] = investment_df['position'] * (investment_df['actual'].shift(-1) - investment_df['actual']) / investment_df['actual']
investment_df['buy_and_hold_returns'] = (investment_df['actual'].shift(-1) - investment_df['actual']) / investment_df['actual']

The code first creates a copy of `comparison_df` and names it `investment_df`. Then it adds a new column called `price_direction` which takes the value of 1 if the next prediction is higher than the current prediction and -1 otherwise. Next it adds another column called `sentiment_direction` which takes the value of 1 if the sentiment is positive and -1 if it's negative. Then it adds a column named `position` which takes the value of `price_direction` if it matches `sentiment_direction` and 0 otherwise. The code then calculates `strategy_returns` by multiplying `position` with the relative change in the actual values from one row to the next. Finally it calculates `buy_and_hold_returns` as the relative change in the actual values from one row to the next without considering the positions.

Results from this backtest look like this:

Datos normalizados guardados en 'binance_data_normalized.csv'
Sentimientos diarios guardados en 'daily_sentiments.csv'
Predicciones y sentimiento guardados en 'predicted_data_with_sentiment.csv'
Mean Absolute Error (MAE): 30.66908467315391
Root Mean Squared Error (RMSE): 36.99641752814565
R-squared (R2): 0.9257591918098058
Mean Absolute Percentage Error (MAPE): 0.00870572230484879
Gráfica guardada como 'ETH_USDT_price_prediction.png'
Gráfica de residuales guardada como 'ETH_USDT_residuals.png'
Correlation between actual and predicted prices: 0.9752007459642241
Gráfica de estrategia de inversión guardada como 'ETH_USDT_investment_strategy.png'
Gráfica de drawdown guardada como 'ETH_USDT_drawdown.png'
Sharpe Ratio: 9.41431958149606
Sortino Ratio: 11800588386323879936.0000
Número de rendimientos totales: 28
Número de rendimientos en exceso: 28
Número de rendimientos negativos: 19
Media de rendimientos en exceso: 0.005037
Desviación estándar de rendimientos negativos: 0.000000
Sortino Ratio: nan
Beta: 0.33875104783408166
Alpha: 0.006981197358213854
Cross-Validation MAE: 1270.7809910146143 ± 527.5746657573876
SMA Mean Absolute Error (MAE): 344.3737716856061
SMA Mean Absolute Error (MAE): 344.3737716856061
SMA Root Mean Squared Error (RMSE): 483.0396130996611
SMA R-squared (R2): 0.5813550203375846
Gráfica de predicción SMA guardada como 'ETH_USDT_sma_price_prediction.png'
Gráfica de precio, predicción y sentimiento guardada como 'ETH_USDT_price_prediction_sentiment.png'
Gráfica de drawdown guardada como 'ETH_USDT_drawdown.png'
Maximum Drawdown: 0.00%

As results say, the correlation between the predicted prices and the real prices are very good. R2, that is a metric to measure how good the predictions of the model, also looks good. Sharpe ratio is higher than 5, which is excellent, as well as Sortino. Also, other results are shown in graphs.

The graph that compares the strategy vs hold looks like this:

Strategy vs hold

Other graphs like price prediction vs actual price

price prediction vs actual price

and, actual price, price prediction and sentiment

Price Prediction and Sentiment

Results, conclude this strategy is very profitable, so we will continue with this argument to create an EA.

This EA should have two Python scripts that make the sentiment analysis and the Deep Learning Model, and should be all merged to function in the EA.

ONNX Model

The code for the data acquisition, training, and ONNX model remains the same as we used in previous articles. Therefore, I will proceed to discuss the Python code for sentiment analysis.

Sentiment Analysis with Python

We will use the libraries `requests` and `TextBlob` to fetch forex news and perform sentiment analysis, along with the `csv` library for reading and writing data. Additionally, the `datetime` and `time` libraries will be utilized.

import requests
from textblob import TextBlob
import csv
from datetime import datetime
import time
from time import sleep

The idea for this script is first to delay for a few seconds upon starting (to ensure that the next part of the script can function properly). The second part of the script will read the API key we want to use. For this case, we will use the Marketaux API, which offers a series of free news and free calls. There are more options such as News API, Alpha Vantage, or Finhub, some of which are paid but provide more news, including historical news, allowing a backtesting of the strategy in MT5. As mentioned earlier, we will use Marketaux for now since it has a free API to obtain daily news. If we want to use other sources, we will need to adapt the code.

Here is a draft of how the script could be structured:

Here's the function to read the api key from the input of the EA:

api_file_path = 'C:/Users/jsgas/AppData/Roaming/MetaQuotes/Terminal/24F345EB9F291441AFE537834F9D8A19/MQL5/Files/Files/api.txt'

def read_api_from_file():
        with open(api_file_path, 'r', encoding='utf-16') as file:
            raw_data =
            print(f"Raw data from file: {repr(raw_data)}")  # Print raw data
            api = raw_data.strip()  # Lee el contenido y elimina espacios en blanco adicionales
            api = api.replace('\ufeff', '')  # Remove BOM character if present
            print(f"API after stripping whitespace: {api}")
            return api
    except FileNotFoundError:
        print(f"El archivo {api_file_path} no existe.")
        return None

# Configuración de la API de Marketaux

Before reading the news, we need to know what to read, and for that, we will have this Python script read from a text file created by the EA, so that the Python script knows what to read or which symbol to study and obtain news about, and, what api key is input in the EA, what date is today so the model gets done and for the news to arrive for this date.

It must also be capable of writing a txt or csv so it serves as input to the EA, with the results of the Sentiment.

def read_symbol_from_file():
        with open(symbol_file_path, 'r', encoding='utf-16') as file:
            raw_data =
            print(f"Raw data from file: {repr(raw_data)}")  # Print raw data
            symbol = raw_data.strip()  # Lee el contenido y elimina espacios en blanco adicionales
            symbol = symbol.replace('\ufeff', '')  # Remove BOM character if present
            print(f"Symbol after stripping whitespace: {symbol}")
            return symbol
    except FileNotFoundError:
        print(f"El archivo {symbol_file_path} no existe.")
        return None
def save_sentiment_to_txt(average_sentiment, file_path='C:/Users/jsgas/AppData/Roaming/MetaQuotes/Terminal/24F345EB9F291441AFE537834F9D8A19/MQL5/Files/Files/'+str(symbol)+'sentiment.txt'):
    with open(file_path, 'w') as f:
if symbol:
    news, current_rate = get_forex_news(symbol)

    if news:
        print(f"Noticias para {symbol}:")
        for i, (title, description) in enumerate(news, 1):
            print(f"{i}. {title}")
            print(f"   {description[:100]}...")  # Primeros 100 caracteres de la descripción
        print(f"\nTipo de cambio actual: {current_rate if current_rate else 'No disponible'}")

        # Calcular el sentimiento promedio
        sentiment_scores = [TextBlob(title + " " + description).sentiment.polarity for title, description in news]
        average_sentiment = sum(sentiment_scores) / len(sentiment_scores) if sentiment_scores else 0
        print(f"Sentimiento promedio: {average_sentiment:.2f}")

        # Guardar resultados en CSV
        #save_to_csv(symbol, current_rate, average_sentiment)

        # Guardar sentimiento promedio en un archivo de texto
        print("Sentimiento promedio guardado en 'sentiment.txt'")
        print("No se pudieron obtener noticias de Forex.")
    print("No se pudo obtener el símbolo del archivo.")

Readers must adapt the whole script depending on what the study, forex, stocks or crypto.

The Expert Advisor

We must include shell32.dll as here to run the python scripts

#include <WinUser32.mqh>

#import "shell32.dll"
int ShellExecuteW(int hwnd, string lpOperation, string lpFile, string lpParameters, string lpDirectory, int nShowCmd);

We must add the python scripts to the File folder

string script1 = "C:\\Users\\jsgas\\AppData\\Roaming\\MetaQuotes\\Terminal\\24F345EB9F291441AFE537834F9D8A19\\MQL5\\Files\\Files\\dl model for mql5 v6 Final";
string script2 = "C:\\Users\\jsgas\\AppData\\Roaming\\MetaQuotes\\Terminal\\24F345EB9F291441AFE537834F9D8A19\\MQL5\\Files\\Files\\sentiment analysis marketaux v6 Final";

And all the paths to the inputs and outputs of the python scripts,

// Ruta del archivo donde se escribirá el símbolo
string filePathSymbol = "//Files//symbol.txt";
// Ruta del archivo donde se escribirá el timeframe
string filePathTimeframe = "//Files//timeframe.txt";
string filePathTime = "//Files//time.txt";
string filePathApi = "//Files//api.txt";

string fileToSentiment = "//Files//"+Symbol()+"sentiment.txt";

string file_add = "C://Users//jsgas//AppData//Roaming//MetaQuotes//Terminal//24F345EB9F291441AFE537834F9D8A19//MQL5//Files";
string file_str = "//Files//model_";
string file_str_final = ".onnx";
string file_str_nexo = "_";

string file_add2 = "C:\\Users\\jsgas\\AppData\\Roaming\\MetaQuotes\\Terminal\\24F345EB9F291441AFE537834F9D8A19\\MQL5\\Files";
string file_str2 = "\\Files\\model_";
string file_str_final2 = ".onnx";
string file_str_nexo2 = "_";

We must input the Marketaux api key

input string api_key      = "mWpORHgs3GdjqNZkxZwnXmrFLYmG5jhAbVrF";           // MARKETAUX_API_KEY

We can obtain that from here, and it will look as this:

api key

I don't work for marketaux, so you can use any other news feed, or subscription you want/need.

You will have to setup a Magic Number, so orders don't get mixed up

int OnInit()

You can also add it here

void OpenBuyOrder(double lotSize, double slippage, double stopLoss, double takeProfit)
// Definir la estructura MqlTradeRequest
   MqlTradeRequest request;
   MqlTradeResult result;

// Inicializar la estructura de la solicitud

// Establecer los parámetros de la orden
   request.action   = TRADE_ACTION_DEAL;
   request.symbol   = _Symbol;
   request.volume   = lotSize;
   request.type     = ORDER_TYPE_BUY;
   request.price    = SymbolInfoDouble(_Symbol, SYMBOL_ASK);
   request.deviation= slippage;       = stopLoss;       = takeProfit;
   request.magic    = Magic_Number;
   request.comment  = "Buy Order";

// Enviar la solicitud de comercio
      Print("Error al abrir orden de compra: ", result.retcode);

That last snippet of the code is how the order is made, you can also use trade from CTrade for making orders.

This will write a file (to use as input in the .py scritps):

void WriteToFile(string filePath, string data)
   Print("Intentando abrir el archivo: ", filePath);
// Abre el archivo en modo de escritura, crea el archivo si no existe
   int fileHandle = FileOpen(filePath, FILE_WRITE | FILE_TXT);
   if(fileHandle != INVALID_HANDLE)
      // Escribe los datos en el archivo
      FileWriteString(fileHandle, data);
      FileClose(fileHandle);  // Cierra el archivo
      Print("Archivo escrito exitosamente: ", filePath);
      Print("Error al abrir el archivo ", filePath, ". Código de error: ", GetLastError());

This will write the symbol, timeframe and current date in the file:

void WriteSymbolAndTimeframe()
// Obtén el símbolo actual
   currentSymbol = Symbol();
// Obtén el período de tiempo del gráfico actual
   string currentTimeframe = GetTimeframeString(Period());
   currentTime = TimeToString(TimeCurrent(), TIME_DATE);

// Escribe cada dato en su respectivo archivo
   WriteToFile(filePathSymbol, currentSymbol);
   WriteToFile(filePathTimeframe, currentTimeframe);
   WriteToFile(filePathTime, currentTime);

   Sleep(10000); // Puedes ajustar o eliminar esto según sea necesario

The function WriteSymbolAndTimeframe performs the following tasks:

  1. First, it retrieves the current trading symbol and stores it in currentSymbol
  2. Then, it gets the current chart's timeframe as a string using GetTimeframeString(Period()) and stores it in currentTimeframe
  3. It also gets the current time in a specific format using TimeToString(TimeCurrent(), TIME_DATE) and stores it in currentTime
  4. Next, it writes each of these values to their respective files:
    • currentSymbol is written to filePathSymbol
    • currentTimeframe is written to filePathTimeframe
    • currentTime is written to filePathTime
    • api_key is written to filePathApi
  5. Finally, the function pauses for 10 seconds using Sleep(10000) which can be adjusted or removed as needed.

We can launch the scripts with this:

void OnTimer()
   datetime currentTime2 = TimeCurrent();

// Verifica si ha pasado el intervalo para el primer script
   if(currentTime2 - lastExecutionTime1 >= interval1)
      // Escribe los datos necesarios antes de ejecutar el script

      // Ejecuta el primer script de Python
      int result = ShellExecuteW(0, "open", "cmd.exe", "/c python \"" + script1 + "\"", "", 1);
      if(result > 32)
         Print("Script 1 iniciado exitosamente");
         Print("Error al iniciar Script 1. Código de error: ", result);
      lastExecutionTime1 = currentTime2;

The function `OnTimer` is executed periodically and performs the following tasks:

  1. First, it retrieves the current time and stores it in `currentTime2`.
  2. It then checks if the time elapsed since the last execution of the first script (`lastExecutionTime1`) is greater than or equal to a predefined interval (`interval1`).
  3. If the condition is met, it writes the necessary data by calling `WriteSymbolAndTimeframe`.
  4. Next, it executes the first Python script by running a command via `ShellExecuteW` which opens `cmd.exe` and runs the Python script specified by `script1`.
  5. If the script execution is successful (indicated by a result greater than 32), it prints a success message; otherwise, it prints an error message with the corresponding error code.
  6. Finally, it updates `lastExecutionTime1` to the current time (`currentTime2`).

We can read the file with this function:

string ReadFile(string file_name)
   string result = "";
   int handle = FileOpen(file_name, FILE_READ|FILE_TXT|FILE_ANSI); // Use FILE_ANSI for plain text

   if(handle != INVALID_HANDLE)
      int file_size = FileSize(handle); // Get the size of the file
      result = FileReadString(handle, file_size); // Read the whole file content
      Print("Error opening file: ", file_name);

   return result;

The code defines a function named ReadFile which takes a file name as an argument and returns the file content as a string first it initializes an empty string result then it attempts to open the file with read permissions and in plain text mode using FileOpen if the file handle is valid it gets the file size using FileSize reads the entire file content into result using FileReadString and then closes the file using FileClose if the file handle is invalid it prints an error message with the file name finally it returns the result containing the file content.

By changing this condition, we can add the sentiment as one more:

   if(ExtPredictedClass==PRICE_DOWN && Sentiment_number<0)
      signal=ORDER_TYPE_SELL;    // sell condition
      if(ExtPredictedClass==PRICE_UP && Sentiment_number>0)
         signal=ORDER_TYPE_BUY;  // buy condition
         Print("No order possible");

The sentiment in this case goes from 10 to -10, being 0 a neutral signal. You can modify as you want this strategy.

The rest of the code is the simple EA used from the article How to use ONNX models in MQL5 with a few modifications.

This is not a complete finished EA, this is just a simple example of how to use python and mql5 to create a sentiment & deep learning Expert Advisor. As more time you invest in this EA, you will get less errors and problems. This is a cutting edge case study, and backtesting shows promising results. I hope you find this article helpful, and if someone can manage to get a good sample of news or makes it work for some time, please share results. In order to test the strategy, you should use a demo account.


In conclusion, the integration of deep learning and sentiment analysis into MetaTrader 5 (MQL5) trading strategies exemplifies the advanced capabilities of modern algorithmic trading. By leveraging Python scripts through a DLL shell32.dll interface, we can seamlessly execute complex models and obtain valuable sentiment data, thereby enhancing trading decisions and outcomes. The process outlined includes using Python to fetch and analyze news sentiment, running ONNX models for price predictions, and executing trades when both indicators align.

The backtesting results demonstrate the strategy's potential profitability, as indicated by strong correlation metrics, high R-squared values, and excellent Sharpe and Sortino ratios. These findings suggest that combining sentiment analysis with deep learning can significantly improve the accuracy of trading signals and overall strategy performance.

Moving forward, the development of a fully functional Expert Advisor (EA) involves meticulous integration of various components, including Python scripts for sentiment analysis and ONNX models for price prediction. By continually refining these elements and adapting the strategy to different markets and data sources, traders can build a robust and effective trading tool.

This study serves as a foundation for those interested in exploring the convergence of machine learning, sentiment analysis, and algorithmic trading, offering a pathway to more informed and potentially profitable trading decisions.

Attached files | (1640.86 KB)
Last comments | Go to discussion (8)
Javier Santiago Gaston De Iriarte Cabrera

I'm thinking of a bot, that has deep learning, deep learning with sentiment analysis, and deep learning with some strategies with auto optimization ... so subscribers can choose & do diversification ... I could do all the models and add them to the EA, I can do all this, I have already done something similar and loaded it to the market, but didn't have sales, so I didn't work on it more and deleted it from the market. We could have a page here for the EA, where people can ask for more addons, or discuss results, to see which of the strategies fit better with which symbols .... I would love to do this, and would love to help people win more money with a small inversion.

I could keep it actualized, and make more adjustments, like, not trade if important news are coming, or make another strategy for when news come (still need to work on this, but could be added) ... I just want to find my place a job and stability, and find this a good way to help people (for example, people don't have time to learn mql5).

Conclusion: if people like this idea, I could do a really good bot, with many profitable strategies for a low price (I can't say this is gonna give you money, because no one says this is gonna give you money, you can't predict future ... and if they say so, it would probably be a scam, as I've learned so, even backtestings have been shown that can be tricked) ... but, at least I can promise I will work on it full time to make reasonable good price bot, so you don't have to spend time or computer resources doing the models etc .... I like this idea, if people find this idea interesting, please let me know. This would also be the bot I use for me, I'm searching for a job, and find this an excellent way to work on.

Javier Santiago Gaston De Iriarte Cabrera

I you don't want to leave here a message, you can write me a private message to discuss this. I would love to work on this, and sell for the minimum, If some one can reach or get 40 subscriber it would be amazing and this could be done, I have really thought of this being a good bot, for a low price, need to live from something, so the lowest price is the minimum. I would love to get 40 subscribers minimum, to start, so after no one gets angry if price changes, the idea is to keep it at a minimum price for ever.

I would like to keep it like this, because, I got scammed thousand of dollars with bots no longer exist. Be careful with whom you buy bots from and think no one needs to learn the rough way. At least I show my face, I started a youtube channel algotradingmastery , and also in and live in Alicante Spain, so any one can come here to hit me if they feel scammed, I can bet you others don't do this, and some even delete his mql5 page after making the hit in sales and you loose a bot after paying thousands of dollars.

I know people think the bot as something without a person behind, but here is the person, I'm telling you all about me, I don't like to lie, and would like to do this to help people and create my formal job.

Javier Santiago Gaston De Iriarte Cabrera

Sorry, I forgot to add WinUser32 to the article

Arup Nag
Arup Nag | 13 Jul 2024 at 08:29
Hello Javier. I wish to discuss this with you in private. I sent you a friend request. Please accept that and then I will propose you something very interesting. Cheers
pool-dev | 15 Jul 2024 at 22:58

Hello Javier, I find your idea very interesting and I would like to discuss it with you so that we can work on it together. If the results are good, I could bring traffic to your robot, with investors ready to pay.

I have sent you a friend request to discuss it :D

MQL5 Wizard Techniques you should know (Part 26): Moving Averages and the Hurst Exponent MQL5 Wizard Techniques you should know (Part 26): Moving Averages and the Hurst Exponent
The Hurst Exponent is a measure of how much a time series auto-correlates over the long term. It is understood to be capturing the long-term properties of a time series and therefore carries some weight in time series analysis even outside of economic/ financial time series. We however, focus on its potential benefit to traders by examining how this metric could be paired with moving averages to build a potentially robust signal.
Reimagining Classic Strategies in Python: MA Crossovers Reimagining Classic Strategies in Python: MA Crossovers
In this article, we revisit the classic moving average crossover strategy to assess its current effectiveness. Given the amount of time time since its inception, we explore the potential enhancements that AI can bring to this traditional trading strategy. By incorporating AI techniques, we aim to leverage advanced predictive capabilities to potentially optimize trade entry and exit points, adapt to varying market conditions, and enhance overall performance compared to conventional approaches.
Creating an Interactive Graphical User Interface in MQL5 (Part 1): Making the Panel Creating an Interactive Graphical User Interface in MQL5 (Part 1): Making the Panel
This article explores the fundamental steps in crafting and implementing a Graphical User Interface (GUI) panel using MetaQuotes Language 5 (MQL5). Custom utility panels enhance user interaction in trading by simplifying common tasks and visualizing essential trading information. By creating custom panels, traders can streamline their workflow and save time during trading operations.
Neural networks made easy (Part 77): Cross-Covariance Transformer (XCiT) Neural networks made easy (Part 77): Cross-Covariance Transformer (XCiT)
In our models, we often use various attention algorithms. And, probably, most often we use Transformers. Their main disadvantage is the resource requirement. In this article, we will consider a new algorithm that can help reduce computing costs without losing quality.