CCXT Application Server Showcase: access crypto exchanges via MQL5 library from MetaTrader 5

CCXT is a JavaScript library for working with all top-100 crypto exchanges. For further details on CCXT, please, visit common CCXT documentation and more advanced CCXT PRO.

CcxtAppServerLib is a MQL5 library for working with the crypto exchanges via Node.js and CCXT Application Server built on top of CCXT.

Latest beta-version of CcxtAppServer library includes optimized caching of exchange "markets" (instrument specifications and other trading conditions), less memory footprint during parsing of huge json-files, better error handling, and new examples.

This blogpost will present an introductory showcase with most important functions from public APIs - the script CcxtAppSrvShowcase.mq5.

First, include the headers.

#include "ccxtjsmtlib.mqh" // import ccxtappsrvlib.ex5
#include "ccxtutil.mqh"

In the inputs, the Node server setup should be done (by default, it's localhost and port 8124).

input group "Connection settings"
input string NodeServer = "http://127.0.0.1:8124";
input string NodeAuth = ""; // NodeAuth (for non-localhost connection)

Next, provide a specific exchange you want to work with. Leave the input empty to view a printout with a list of all supported exchanges.

input string Exchange = ""; // Exchange (empty - list exchanges)

Also provide a ticker you're interested in. If you don't know the name, run the script first time and look at MQL5/Files/CCXT/ folder where all received data is dumped by default, so you can find json-files with full markets info.

input string Ticker = "BCH/USDT";

For watching tests (subscriptions via websockets) specify their duration:

input uint WatchingDuration = 10; // WatchingDuration (seconds, 0 - infinite)

The script demonstrates how to setup credentials for private APIs, but will not use any private function.

input group "Exchange settings (Private API)"
input string ApiKey = "";
input string Secret = "";
input string Uid = "";
input string Login = "";
input string Password = "";

Additional settings allow you to control the logging level, dumping of all received data, and timeouts.

input group "Auxiliary settings"
input ushort Logging = 1; // Logging (0 - no, 1 - minimal, 2 - headers, 3 - full)
input bool Dumping = true;
input uint Timeout = 5; // Timeout (seconds)

Then the main event handler OnStart comes into play. The imported functions, classes and methods from the library are highlighted in yellow. Some parts are omitted for brevity (for details look at the full source code, distributed with the library).

Inline comments are self-explanatory, I think.

If the script is running very first time, it will ask to unpack (manually) CCXT Application Server (extracted as ccxtappsrvbundle.jsc from a built-in resource), and run Node.js with it.

void OnStart()
{
   // I. Initialization
   // performed once after the product download
   
   PrintFormat("CCXT AppSrvLibrary version: %.2f", AppSrvLibraryVersion());
   const static string status[] = {"Can't deploy",
      "App server ZIP is deployed, but not extracted",
      "App server files are deployed"};
   const int d = DeployCcxtAppServer();
   Print(status[d + 1]);
   if(d <= 0)
   {
      return; // CCXT Application Server is not yet unpacked (manually)
   }

   // II. Library and application server setup
   
   // prepare all global/static settings
   SetNodeServer(NodeServer, NodeAuth);

   CcxtLink::Settings settings = {Logging, Dumping, Testing, Timeout, 0};
   CcxtLink *link = GetLink();  // NB: the link is used by all further calls
   link.applySettings(settings);
   
   // get list of exchanges and component versions
   if(!StringLen(Exchange))
   {
      Print("Complete list of exchanges:");
      Print(ListExchanges().stringify());
      Print("Pro list of exchanges with websockets support:");
      Print(ListExchanges(true).stringify());

      Print("App Server Version: ", AppSrvVersion().stringify());
      Print("CCXT lib version: ", CcxtVersion()["version"].stringify());
      return;
   }

   // III. Specific exchange setup
   
   // settings per exchange
   CCXT::Credentials credentials = {ApiKey, Secret, Uid, Login, Password};
   
   // initialization (variants), will request and create an exchange object on Node and cache its properties locally in MQL5 object
   CcxtJsExchangeProIntf *ccxt = CreateExchangePro(Exchange, credentials, false/* true by default - load markets on Node */);
   AutoPtr<CcxtJsExchangeProIntf> auto(ccxt);
   // or
   // CcxtJsExchangeProIntf *ccxt = CreateExchangePro(Exchange, false);
   // AutoPtr<JsValue> c = credentials.toJSON();
   // ccxt.set(c[]);
   
   if(link.getLastHttpCode() != 200) // it's recommended to check for errors after every call and exit
   {
      return;
   }

   // show main properties right from the local object (no requests performed)
   const bool isPro = !!*ccxt["pro"];
   if(ShowExchangeProperties)
   {
      Print("Is pro: ", isPro);
      Print("Required Credentials:");
      ccxt["requiredCredentials"].print();
      Print("Supported features:");         // as declared in the exchange description
      ccxt["has"].print();
      AutoPtr<JsValue> f = ccxt.get("facilities"); // as implemented in JavaScript (may contain function stubs)
      Print("Facilities: ", f[].stringify());
   }

   // IV. Load markets

   // get whole (built-in) settings of the exchange from Node, can check if the provided above credentials are applied
   // NB! if Node was already running and this exchange loaded markets before, you'll still get object without markets,
   // to read markets locally, call .loadMarkets(true/false, *true*)
   AutoPtr<JsValue> check = ccxt.get();
   if(Dumping) DumpJsonToFile("CCXT/check-" + Exchange, check[]);
   
   // lets load markets from the exchange, if they are not loaded on Node yet
   // (this is true in this test during very first run, because the exchange is constructed with false for loading markets)
   // any call will fail if markets not loaded, normally 'now' returns a number of milliseconds elapsed since 1970
   // NB: failure will throw Alert!
   if(ccxt.call("now").t != JS_PRIMITIVE)
   {
      /* empty object */ccxt.loadMarkets(false /* don't reload on Node */, false /*don't parse and cache whole object in MQL5*/);
      JsValue *info = ccxt.get("markets"); // request only "markets" branch/dictionary to use less memory
      if(Dumping) DumpJsonToFile("CCXT/onlymarkets-" + Exchange, info);

      // or (to get a big object with markets and etc)
      // CcxtJsExchange ccxt(Exchange, credentials /*, true by default - load on Node */);
      // JsValue *info = ccxt.loadMarkets(false, true);
      // DumpJsonToFile(...)
      
      // NB: 'info' will be cleaned up below along with other orphan objects by calling ccxt.fetchAnything(NULL)
   }
   else
   {
      Print("Markets are already loaded on Node");
   }

   // V. Fetching data through WebRequests

   // Fetches make this thread busy for some time,
   // so if you'd started websockets before (and they keep working on Node in parallel),
   // new incoming messages go into a queue for handing over to MQL5 after fetches

   JsValue *orderbook = ccxt.fetchOrderBook(Ticker, 10);
   if(Dumping) DumpJsonToFile("CCXT/orderbook-" + Exchange + "-" + Escape(Ticker), orderbook);
   
   JsValue *ticker = ccxt.fetchTicker(Ticker);
   if(Dumping) DumpJsonToFile("CCXT/ticker-" + Exchange + "-" + Escape(Ticker), ticker);
   
   JsValue *ohlcv = ccxt.fetchOHLCV(Ticker, "1m", t ? t - 1000 * 60 * 10 : 0, 10);
   if(Dumping) DumpJsonToFile("CCXT/ohlcv-" + Exchange + "-" + Escape(Ticker), ohlcv);

   JsValue *trades = ccxt.fetchTrades(Ticker, t ? t - 10000 : 0, 10);
   if(Dumping) DumpJsonToFile("CCXT/trades-" + Exchange + "-" + Escape(Ticker), trades);
   
   if(!!*ccxt["has"]["fetchBidsAsks"]) // dereference object by * for operator! to check if something is supported
   {
      string array[] = {Ticker};
      JsValue *bidsasks = ccxt.fetchBidsAsks(array);
      if(Dumping) DumpJsonToFile("CCXT/bidsasks-" + Exchange + "-" + Escape(Ticker), bidsasks);
   }

   // periodically clean-up cached json objects after fetches when they're not used anymore, such as 'ticker' above,
   // the program will clean them up automatically on exit, but they consume memory,
   // markets and currencies are not stored in this cache
   ccxt.fetchAnything(NULL);
   ...

The requested URLs and corresponding names of saved files are shown in the log.

And now goes the CCXT PRO part based on websockets and live notification subscriptions.

   // VI. Subscribing for online data changes notifications via websockets  
  
   // start websockets connection (if the exchange supports it)
   if(isPro && ccxt.upgrade())
   {
      // subscribe for watching something
      ccxt.watchOrderBook(Ticker);
      ccxt.watchTrades(Ticker);
      string tickers[] = {Ticker};
      ccxt.watchBidsAsks(tickers);
      ccxt.watchTrades(Ticker); // increment counter to 2, no new subscription
      const uint start = GetTickCount();
      while(!IsStopped() && ccxt.isConnected() && (!WatchingDuration || GetTickCount() - start < WatchingDuration * 1000))
      {
         AutoPtr<JsValue> j = ccxt.readMessage(); // blocking until new message or timeout
         if(j[])
         {
            Comment(j[].stringify()); // pretty print on chart
         }
         else
         {
            // timeout
         }
      }
   }
   else
   {
      if(isPro && ccxt.isConnected())
      {
         Print("Can't upgrade to websockets");
         string headers[][2];
         if(ccxt.ws().getHeaders(headers))
         {
            // TODO: analyze status and solve it
         }
         ccxt.ws().close(); // close to prevent further attempts to read/write from non-upgraded socket
      }
   }

   // if still connected check for new watches until unsubscription takes effect
   if(ccxt.isConnected())
   {
      Print("Unsubscribing...");
      // try to unsubscribe and wait a bit
      ccxt.un().watchOrderBook(Ticker);
      ccxt.un().watchTrades(Ticker);
      ccxt.un().watchBidsAsks(); // this will not be unwatched because we do not pass exact parameters
      ccxt.un().watchTrades(Ticker); // try again to decrement counter to 0 and release

      const uint start = GetTickCount(); // because unsubscription may fail, setup a small timeout
      while(!IsStopped() && ccxt.isConnected() && (GetTickCount() - start < 5 * 1000))
      {
         Print("reading...");
         AutoPtr<JsValue> j = ccxt.readMessage();
         if(j[])
         {
            Comment(j[].stringify());
         }
         else
         {
            break;
         }
      }
      
      Print("Closing...");
      // trying to close gracefully
      ccxt.close();
      
      while(!IsStopped()) // probably something left in buffer
      {
         AutoPtr<JsValue> j = ccxt.readMessage();
         if(j[])
         {
            Comment(j[].stringify());
         }
         else
         {
            break;
         }
      }
   }
}

When the script is running, all incoming websocket data is shown as comments on the chart (though the info can update very quickly).

The CCXT Application Server Library is currently in beta-stage and available for testing upon request.