Machine learning in trading: theory, models, practice and algo-trading - page 900

New comment
 
Alexander Ivanov:

Is the grail ready yet? With the AI)

well that's it... you're already oligarchs ))

Of course, Maximko always achieves his goal.

 
Maxim Dmitrievsky:

of course, Maksimko always achieves his goal

)))

That's it... We abandon all our robots, get in line for the wonder-intelligent robot.

 
Alexander Ivanov:

)))

That's it... We're leaving all our robots behind and getting in line for the wonder-intelligent robot.

I'm already halfway to the Swiss bunker with paramilitary guards, following Alexander with his dusty sacks full of bills, don't mention it!

 
Maxim Dmitrievsky:

Alexander_K2 from another thread

I would like to connect mt5 with python to be able to visualize it and check it immediately... now I am considering the possibility to call python shell directly through winapi and send commands from bot to python, I don't know if it is possible. R and dll do not digest and do not know how and do not want to work with them, although python is not required (looking at articles and works of old-timers) less and less desire to get into the thick of it - 500000 packets and the output is the same as from the bot on 2 MA

Link to such a monster would be useful to many, but not many under its power.

If I wanted to use 10 MAs with 16 steps and make a predictor - for each one - the price opened above/below MA and one more - number of MAs when numbering all MAs from the top to the bottom, it would be a model that describes the market?

 
Maxim Dmitrievsky:

Did a little experiment with dropping whole models (10 models in total are given). Forward from 2018.04.01.

Without dropout:

The following numbers mark how many random models were left, the rest were dropped:

Well some kind of this is what it turns out to be. Probably not very revealing, since the model turned out pretty good without dropouts. And the models are too similar to each other, which is bad, you need to revise the learning algorithm (game theory to help). But still some flexibility in the choices appeared.

It looks good! Stop at 1 (1 model).

 
Aleksey Vyazmikin:

I have to be able to see how many people are interested in this kind of monster, but not many.

As for the MAs, just falling asleep I was thinking, what if we take 10 MAs with 16 steps and make a predictor - for each - the price opened above/below the MA and another - the number of MAs when numbering all MAs from the top of the chart to the bottom, will such a model describe the market?

That's no way to put the question. Experiments, experiments... ...because nothing is obvious in theory. I've tried about a hundred different variants of TS, and it's scary to think about it.

The branch history keeps a lot of information on how not to do it (and sometimes how to do it) is quite useful.

 
Aleksey Vyazmikin:

I have to be able to see how many people are interested in this kind of monster, but not many.

What about MAs, I was just thinking while falling asleep - what if I take 10 arrows with 16 steps and make a predictor - for each one - the price opened above/below MA and one more - number of MA when numbering all MAs from the top to the bottom, will such a model describe the market?

There was an article 10 years ago with such experiment with МА from 2 to 100.
 
elibrarius:
There was an article about 10 years ago with such an experiment with MA from 2 to 100.

And I think it's called a fan, if I'm not mistaken...

 

It seems to solve all the problems of mql4 communication with different languages. There is even a code for R. Here is the schematic.



The whole description is in three parts:

https://blog.darwinex.com/zeromq-interface-python-r-metatrader4/

https://blog.darwinex.com/zeromq-trade-execution-metatrader-zmq2/

https://blog.darwinex.com/zeromq-transaction-reporting-metatrader-zmq3/

Approved:

Why ZeroMQ?

Enables programmers to connect any code to any other code, in a number of ways.

2.Eliminates aMetaTrader user's dependency on just MetaTrader-supported technology (features, indicators, language constructs, libraries, etc.).

Traders can develop indicators and strategies in C/C#/C++, Python, R and Java (to name a few), and deploy to market via MetaTrader 4.

Leveragemachine learning toolskits in Python and R for complex data analysis and strategy development, while interfacing with MetaTrader 4 for trade execution and management.

ZeroMQ can be used as a high-performance transport layer in sophisticated, distributed trading systems otherwise difficult to implement in MQL.

Different strategy components can be built in different languages if required, and seamlessly talk to each other over TCP, in-process, inter-process or multicast protocols.

Multiple communication patterns and disconnected operation.

Here is the code

/+------------------------------------------------------------------+
//|                                       ZeroMQ_MT4_EA_Template.mq4 |
//|                                    Copyright 2017, Darwinex Labs |
//|                                        https://www.darwinex.com/ |
//+------------------------------------------------------------------+
#property copyright "Copyright 2017, Darwinex Labs."
#property link      "https://www.darwinex.com/"
#property version   "1.00"
#property strict

// Required: MQL-ZMQ from https://github.com/dingmaotu/mql-zmq
#include <Zmq/Zmq.mqh>

extern string PROJECT_NAME = "DWX_ZeroMQ_Example";
extern string ZEROMQ_PROTOCOL = "tcp";
extern string HOSTNAME = "*";
extern int REP_PORT = 5555;
extern int PUSH_PORT = 5556;
extern int MILLISECOND_TIMER = 1;  // 1 millisecond

extern string t0 = "--- Trading Parameters ---";
extern int MagicNumber = 123456;
extern int MaximumOrders = 1;
extern double MaximumLotSize = 0.01;

// CREATE ZeroMQ Context
Context context(PROJECT_NAME);

// CREATE ZMQ_REP SOCKET
Socket repSocket(context,ZMQ_REP);

// CREATE ZMQ_PUSH SOCKET
Socket pushSocket(context,ZMQ_PUSH);

// VARIABLES FOR LATER
uchar data[];
ZmqMsg request;

//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
//---

   EventSetMillisecondTimer(MILLISECOND_TIMER);     // Set Millisecond Timer to get client socket input
   
   Print("[REP] Binding MT4 Server to Socket on Port " + REP_PORT + "..");   
   Print("[PUSH] Binding MT4 Server to Socket on Port " + PUSH_PORT + "..");
   
   repSocket.bind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, REP_PORT));
   pushSocket.bind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, PUSH_PORT));
   
   /*
       Maximum amount of time in milliseconds that the thread will try to send messages 
       after its socket has been closed (the default value of -1 means to linger forever):
   */
   
   repSocket.setLinger(1000);  // 1000 milliseconds
   
   /* 
      If we initiate socket.send() without having a corresponding socket draining the queue, 
      we'll eat up memory as the socket just keeps enqueueing messages.
      
      So how many messages do we want ZeroMQ to buffer in RAM before blocking the socket?
   */
   
   repSocket.setSendHighWaterMark(5);     // 5 messages only.
   
//---
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
{
//---
   Print("[REP] Unbinding MT4 Server from Socket on Port " + REP_PORT + "..");
   repSocket.unbind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, REP_PORT));
   
   Print("[PUSH] Unbinding MT4 Server from Socket on Port " + PUSH_PORT + "..");
   pushSocket.unbind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, PUSH_PORT));
   
}
//+------------------------------------------------------------------+
//| Expert timer function                                            |
//+------------------------------------------------------------------+
void OnTimer()
{
//---

   /*
      For this example, we need:
      1) socket.recv(request,true)
      2) MessageHandler() to process the request
      3) socket.send(reply)
   */
   
   // Get client's response, but don't wait.
   repSocket.recv(request,true);
   
   // MessageHandler() should go here.   
   ZmqMsg reply = MessageHandler(request);
   
   // socket.send(reply) should go here.
   repSocket.send(reply);
}
//+------------------------------------------------------------------+

ZmqMsg MessageHandler(ZmqMsg &request) {
   
   // Output object
   ZmqMsg reply;
   
   // Message components for later.
   string components[];
   
   if(request.size() > 0) {
   
      // Get data from request   
      ArrayResize(data, request.size());
      request.getData(data);
      string dataStr = CharArrayToString(data);
      
      // Process data
      ParseZmqMessage(dataStr, components);
      
      // Interpret data
      InterpretZmqMessage(&pushSocket, components);
      
      // Construct response
      ZmqMsg ret(StringFormat("[SERVER] Processing: %s", dataStr));
      reply = ret;
      
   }
   else {
      // NO DATA RECEIVED
   }
   
   return(reply);
}

// Interpret Zmq Message and perform actions
void InterpretZmqMessage(Socket &pSocket, string& compArray[]) {

   Print("ZMQ: Interpreting Message..");
   
   // Message Structures:
   
   // 1) Trading
   // TRADE|ACTION|TYPE|SYMBOL|PRICE|SL|TP|COMMENT|TICKET
   // e.g. TRADE|OPEN|1|EURUSD|0|50|50|R-to-MetaTrader4|12345678
   
   // The 12345678 at the end is the ticket ID, for MODIFY and CLOSE.
   
   // 2) Data Requests
   
   // 2.1) RATES|SYMBOL   -> Returns Current Bid/Ask
   
   // 2.2) DATA|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME
   
   // NOTE: datetime has format: D'2015.01.01 00:00'
   
   /*
      compArray[0] = TRADE or RATES
      If RATES -> compArray[1] = Symbol
      
      If TRADE ->
         compArray[0] = TRADE
         compArray[1] = ACTION (e.g. OPEN, MODIFY, CLOSE)
         compArray[2] = TYPE (e.g. OP_BUY, OP_SELL, etc - only used when ACTION=OPEN)
         
         // ORDER TYPES: 
         // https://docs.mql4.com/constants/tradingconstants/orderproperties
         
         // OP_BUY = 0
         // OP_SELL = 1
         // OP_BUYLIMIT = 2
         // OP_SELLLIMIT = 3
         // OP_BUYSTOP = 4
         // OP_SELLSTOP = 5
         
         compArray[3] = Symbol (e.g. EURUSD, etc.)
         compArray[4] = Open/Close Price (ignored if ACTION = MODIFY)
         compArray[5] = SL
         compArray[6] = TP
         compArray[7] = Trade Comment
   */
   
   int switch_action = 0;
   
   if(compArray[0] == "TRADE" && compArray[1] == "OPEN")
      switch_action = 1;
   if(compArray[0] == "RATES")
      switch_action = 2;
   if(compArray[0] == "TRADE" && compArray[1] == "CLOSE")
      switch_action = 3;
   if(compArray[0] == "DATA")
      switch_action = 4;
   
   string ret = "";
   int ticket = -1;
   bool ans = FALSE;
   double price_array[];
   ArraySetAsSeries(price_array, true);
   
   int price_count = 0;
   
   switch(switch_action) 
   {
      case 1: 
         InformPullClient(pSocket, "OPEN TRADE Instruction Received");
         // IMPLEMENT OPEN TRADE LOGIC HERE
         break;
      case 2: 
         ret = "N/A"; 
         if(ArraySize(compArray) > 1) 
            ret = GetBidAsk(compArray[1]); 
            
         InformPullClient(pSocket, ret); 
         break;
      case 3:
         InformPullClient(pSocket, "CLOSE TRADE Instruction Received");
         
         // IMPLEMENT CLOSE TRADE LOGIC HERE
         
         ret = StringFormat("Trade Closed (Ticket: %d)", ticket);
         InformPullClient(pSocket, ret);
         
         break;
      
      case 4:
         InformPullClient(pSocket, "HISTORICAL DATA Instruction Received");
         
         // Format: DATA|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME
         price_count = CopyClose(compArray[1], StrToInteger(compArray[2]), 
                        StrToTime(compArray[3]), StrToTime(compArray[4]), 
                        price_array);
         
         if (price_count > 0) {
            
            ret = "";
            
            // Construct string of price|price|price|.. etc and send to PULL client.
            for(int i = 0; i < price_count; i++ ) {
               
               if(i == 0)
                  ret = compArray[1] + "|" + DoubleToStr(price_array[i], 5);
               else if(i > 0) {
                  ret = ret + "|" + DoubleToStr(price_array[i], 5);
               }   
            }
            
            Print("Sending: " + ret);
            
            // Send data to PULL client.
            InformPullClient(pSocket, StringFormat("%s", ret));
            // ret = "";
         }
            
         break;
         
      default: 
         break;
   }
}

// Parse Zmq Message
void ParseZmqMessage(string& message, string& retArray[]) {
   
   Print("Parsing: " + message);
   
   string sep = "|";
   ushort u_sep = StringGetCharacter(sep,0);
   
   int splits = StringSplit(message, u_sep, retArray);
   
   for(int i = 0; i < splits; i++) {
      Print(i + ") " + retArray[i]);
   }
}

//+------------------------------------------------------------------+
// Generate string for Bid/Ask by symbol
string GetBidAsk(string symbol) {
   
   double bid = MarketInfo(symbol, MODE_BID);
   double ask = MarketInfo(symbol, MODE_ASK);
   
   return(StringFormat("%f|%f", bid, ask));
}

// Inform Client
void InformPullClient(Socket& pushSocket, string message) {

   ZmqMsg pushReply(StringFormat("%s", message));
   // pushSocket.send(pushReply,true,false);
   
   pushSocket.send(pushReply,true); // NON-BLOCKING
   // pushSocket.send(pushReply,false); // BLOCKING
   
}
 

And here is the code for r

#+------------------------------------------------------------------+
#|                                          ZeroMQ_MT4_R_Template.R |
#|                                    Copyright 2017, Darwinex Labs |
#|                                        https://www.darwinex.com/ |
#+------------------------------------------------------------------+

# Load "rzmq" library. If not installed, run install.packages("rzmq")
library(rzmq)

# Random placeholder for PULL later.
pull.msg <- "N/A"

# Function to send commands to ZeroMQ MT4 EA
remote.send <- function(rSocket,data) {
  send.raw.string(rSocket, data)
  msg <- receive.string(rSocket)
  
  print(msg)
}

# Function to PULL data from ZeroMQ MT4 EA PUSH socket.
remote.pull <- function(pSocket) {

  msg <- receive.socket(pSocket, unserialize = FALSE, dont.wait = TRUE)
  
  if(is.null(msg)) {
    msg <- "No data PUSHED yet.."
    print(msg)
  } else {
    msg <- rawToChar(msg)
    print(msg)  
  }

  return(msg)
}

# CREATE ZeroMQ Context
context = init.context()

# Initialize ZeroMQ REQ Socket
reqSocket = init.socket(context,"ZMQ_REQ")

# Initialize ZeroMQ PULL Socket
pullSocket = init.socket(context, "ZMQ_PULL")

# Connect to REQ Socket on port 5555
connect.socket(reqSocket,"tcp://localhost:5555")

# Connect to PULL Socket on port 5556
connect.socket(pullSocket,"tcp://localhost:5556")

# Run Tests
while(TRUE) {
  
  # REMEMBER: If the data you're pulling isn't "downloaded" in MT4's History Centre,
  #           it's very likely your PULL will produce no data.
  
  #           So if you're going to be pulling data for a currency pair from MT4,
  #           make sure its data is downloaded, and chart open just in case.
  
  # Pull from server
  remote.pull(pullSocket)
  
  f <- file("stdin")
  open(f)
  
  print("Enter Command for MetaTrader 4 ZeroMQ Server, 'q' to quit")
  # e.g. RATES|EURUSD -> Retrieves Current Bid/Ask for EURUSD from MT4.
  mt4.command <- readLines(f, n=1)
  
  if(tolower(mt4.command) == "q") {
    break
  }
  
  # Send to ZeroMQ MetaTrader 4 Server
  if(!grepl("PULL", mt4.command))
    remote.send(reqSocket, mt4.command)
  
  # Pull from ZeroMQ MetaTrader 4 Server
  pull.msg <- remote.pull(pullSocket)
}
1...893894895896897898899900901902903904905906907...3516
New comment
Reason: