ATcl - Tcl interpreter for MT4 - page 2

[Alexey Oreshkin]

Maxim Kuznetsov:
Who will control the pips, implement the protocol and analyse the results? it's long, slow and only by polling.

Can I have a couple of links to read about this atcl, so you don't have to go through everything, but can be more focused on the topic.

[Maxim Kuznetsov]

Alexey Oreshkin:

Can a couple of links to read about this atcl not to dig through everything, and more purposefully get acquainted with the topic.

The main links to the library-integrator for MT4 (the very one I'm working hard on) are the current topic here, plus the one mentioned in the thread http://luxtrade.tk/atcl:start (historically my site, which is in a constant mess :-) )

the Tcl (root site) itself : http://www.tcl.tk

wiki: http://wiki.tcl.tk

"standard library" : https://core.tcl.tk/tcllib/doc/tcllib-1-18/embedded/www/toc.html (link to the docs, they are probably of primary interest)

Currently ActiveState's leading distributor: https://www.activestate.com/activetcl ,

They are "on the heels" of MagicSplat http://www.magicsplat.com/ and IronTcl https://irontcl.com/

As a language and scripting platform, Tcl is time-tested to say the least - first release in 1990, now it is present de facto in Linux/UNIX, now under Android as well. 98% you have it on your machine in some stripped-down form - python,ruby,R,maxima,octave and others use it for GUI (tkinter etc.)

PS. hope they won't kill my post and me along with a lot of links ? It's all free.

PPS. For local outsourcers - there's also a "tasty" bounty https://github.com/flightaware/Tcl-bounties. But there are only difficult tasks from 2500USD and there is a strict discipline.

[Maxim Kuznetsov]

With thoughts of the future:

with the help of some patches and such a mother, this is what we get:

although this is an experiment, and even in the current version, I will not yet include the patch to run Tk (firstly, it's too early, the other glitches are full, and secondly, it is made for beer, I'll review later)

But you can see what you want to get in the end and have confidence that technically it is possible. Yes, yes a fully scriptable GUI is also possible.

by the way, to local GUI builders, the window (the one with the alpha channel) is described with redundancy like this (it's tcl) :

set symbol "symbol"
set bid "bid"
set ask "ask"
set lots 0.1
set minLot 0.01
set maxLot 1
set lotStep 0.01
proc TradeFrame { w } {
        # всё в отдельном фрейме
        frame $w
        # виджеты
        label $w.symbol -textvariable symbol
        label $w.bid -textvariable bid
        label $w.ask -textvariable ask
        button $w.buy -text "BUY" -command {Trade buy}
        button $w.sell -text "SELL" -command {Trade sell}
        global minLot
        global maxLot
        global lotStep
        ttk::spinbox $w.lots -font "Courier 12" -from $minLot -to $maxLot -increment $lotStep -textvariable lots
        # размещение в таблице
        grid $w.symbol -row 0 -column 0 -columnspan 2 -sticky "ew"
        grid $w.bid    -row 1 -column 0 -sticky "e"
        grid $w.ask    -row 1 -column 1 -sticky "w"
        grid $w.buy    -row 2 -column 0 -sticky "nsew"
        grid $w.sell    -row 2 -column 1 -sticky "nsew"
        grid $w.lots   -row 3 -column 0 -columnspan 2 -sticky "ew"
        # настройка таблицы
        grid rowconfigure $w 2 -weight 1
        grid columnconfigure $w 0 -uniform same -weight 1
        grid columnconfigure $w 1 -uniform same -weight 1
        # привязки и прочие настройки
#       $w.lots set $minLot
        return $w
}
proc TradeWindow { w } {
        toplevel $w 
        wm title $w "TradePanel"
        wm attributes $w -alpha 0.75
        TradeFrame $w.panel 
        pack $w.panel -fill both -expand yes
        raise $w
}
proc Trade { op } {
        global symbol
        global lots
        tk_messageBox -message "Your request $op $lots $symbol lots"
}

[Maxim Kuznetsov]

I put the files in order - now in ZIP they are placed in folders Indicators, Scripts, Experts,Libraries,Include and in idea to install the library it is enough to simply
to unpack the archive into data directory and everything should fall into place and be ready to work (except that Tcl itself needs to be put :-) ).

The API is like this:

class ATcl {
protected:
   Tcl_Interp interp;
public:
   ATcl(bool safe=false);  // создание нового инерпретатора
   ATcl(Tcl_Interp);       // создание объекта из готового интерпретатора

   Tcl_Interp CreateSlave(string name,bool safe=false); // создать дочерный интерпретатор
   Tcl_Interp CreateSlave(Tcl_Obj name,bool safe=false);

   Tcl_Interp Interp(); // получить интерпретатор (например для вызова Tcl C API)
   
   ~ATcl();
   
   bool Ready();  // =true если можно исполнять команды
   int Update(); // запустить проход EventLoop
   
   // MQL specific handlers
   int OnInit();
   void OnDeinit(const int reason);
   void OnTimer();
   void OnChartEvent();
   void OnChartEvent(const int id,const long lparam,const double dparam,const string sparam);

   // Objects
   Tcl_Obj Obj();             // создать пустой временный объект
   Tcl_Obj Ref(Tcl_Obj obj);  // увеличить счётчик ссылок
   Tcl_Obj Unref(Tcl_Obj obj);// уменьшить счётчик ссылок, удалить объект при обнулении
   int Shared(Tcl_Obj obj);   // =true если объект зашарен и не может быть изменнён
   Tcl_Obj Clone(Tcl_Obj obj);// дублировать объект
   
   ///////////////////////////////  
   // Исполнение команд и скриптов
   ///////////////////////////////
   // методы возвращают код реультата: (стандартные коды: TCL_OK - успех, TCL_ERROR, TCL_BREAK,TCL_CONTINUE)
   // и сохраняют объект-"результат", который можно потом получить методоми Result(),StringResult(),...
   // Флаги: TCL_EVAL_GLOBAL - исполнять на глобальном уровне, TCL_EVAL_DIRECT - исполнять сразу, без компиляции
   int Eval(string script,int flags=0);
   int Eval(Tcl_Obj script,int flags=0);
   Tcl_Obj ObjEval(string script,int flags=0);  // simplified - Eval and return Result as Tcl_Obj
   Tcl_Obj ObjEval(Tcl_Obj script,int flags=0);
   
   int Call(Tcl_Obj &objv[],int flags=0);
   int Call(Tcl_Obj obj1,Tcl_Obj obj2=0,Tcl_Obj obj3=0,Tcl_Obj obj4=0,Tcl_Obj obj5=0,Tcl_Obj obj6=0,Tcl_Obj obj7=0,Tcl_Obj obj8=0,Tcl_Obj obj9=0,Tcl_Obj obj10=0,Tcl_Obj obj11=0);
   
   Tcl_Obj ObjCall(Tcl_Obj &objv[],int flags=0);// simplified - Call and return Result as Tcl_Obj
   Tcl_Obj ObjCall(Tcl_Obj obj1,Tcl_Obj obj2=0,Tcl_Obj obj3=0,Tcl_Obj obj4=0,Tcl_Obj obj5=0,Tcl_Obj obj6=0,Tcl_Obj obj7=0,Tcl_Obj obj8=0,Tcl_Obj obj9=0,Tcl_Obj obj10=0,Tcl_Obj obj11=0);
   
   Tcl_Obj Result(int ignoreThis=0);  // получить объект-результат
   int   ResetResult();             // сбросить(обнулить) результат

   ////////////////////////////////////////
   // Source - чтение и исполнение скриптов
   //////////////////////////////////////// 
   // use : Eval("source filename.tcl") оr Call("source","filename.tcl");
   // int Source(const string fileName,const string encoding="");
   // int Source(Tcl_Obj fileName,Tcl_Obj encoding=0);

   /////////////////
   // List - Списки
   /////////////////
   Tcl_Obj ListIndex(Tcl_Obj obj,int index=0);
   
   int ListLength(Tcl_Obj obj);  // =кол-во элементов списка
   int Count(Tcl_Obj obj);       // синоним
   
   int ListAppend(Tcl_Obj list,Tcl_Obj element);   // добавить эл-т к списку. список должен быть !Shared()
   int Append(Tcl_Obj list,Tcl_Obj element); // синоним
   
   /////////////////////////////////////////
   // Set: задать объект-значение переменной
   /////////////////////////////////////////
   // аналог в tcl: set var $value ; set hash($key) $value 
   Tcl_Obj Set(Tcl_Obj var,Tcl_Obj value);
   Tcl_Obj Set(Tcl_Obj hash,Tcl_Obj key,Tcl_Obj value);
   Tcl_Obj Set(string var,Tcl_Obj value);
   Tcl_Obj Set(string hash,string key,Tcl_Obj value);
   
   ///////////////////////////////////////////
   // Get: получить значение переменной в виде объекта
   //////////////////////////////////////////
   // аналог в tcl: set var ; set hash($key)
   Tcl_Obj Get(Tcl_Obj var);
   Tcl_Obj Get(Tcl_Obj hash,Tcl_Obj key);
   Tcl_Obj Get(string var);
   Tcl_Obj Get(string hash,string key);
   
   // IsSet: проверить существование переменной
   int IsSet(Tcl_Obj var);
   int IsSet(Tcl_Obj hash,Tcl_Obj key);
   int IsSet(string var);
   int IsSet(string hash,string key);
   
   // IsArray - проверить что hash это действительно хеш :-)
   int IsArray(Tcl_Obj hash);
   int IsArray(string hash);
   
   // Unset: удалить переменную или элемент хеша
   int Unset(Tcl_Obj var);
   int Unset(Tcl_Obj hash,Tcl_Obj key);
   int Unset(string var);
   int Unset(string hash,string key);
   
   // Link: связать скалярную переменную с элементом массива
   int Link(double &arr[],int pos,Tcl_Obj hash,Tcl_Obj key=0);
   int Link(double &arr[],int pos,string hash,string key="");
   // Unlink: разорвать свзяь
   int Unlink(Tcl_Obj hash,Tcl_Obj key=0);
   int Unlink(string hash,string key="");

   //////////////////////////
   // Double
   Tcl_Obj Obj(double);                // создать объект из double
   Tcl_Obj Obj(const double &arr[],int pos=0,int count=WHOLE_ARRAY); // создать объект из массива double
   double Double(Tcl_Obj);             // конвертировать объект в double
   double Double(Tcl_Obj,int index);   // получить эл-т списка как double   
   double DoubleResult(int ignoreThis=0);  // получить последний результат как double
   // упрощенный доступ к Call,Eval   
   // исполнить скрипт и срузу получить результат как double (0 при ошибке)
   double DoubleEval(string script,int flags=0); 
   double DoubleEval(Tcl_Obj script,int flags=0);
   // вызвать команду и сразу получить результат как double
   double DoubleCall(Tcl_Obj command,Tcl_Obj arg1=0,Tcl_Obj arg2=0,Tcl_Obj arg3=0,Tcl_Obj arg4=0,Tcl_Obj arg5=0,Tcl_Obj arg6=0,Tcl_Obj arg7=0,Tcl_Obj arg8=0,Tcl_Obj arg9=0,Tcl_Obj arg10=0);
   double DoubleCall(Tcl_Obj &objv[],int objc=WHOLE_ARRAY);
   
   ///////////////////////////
   // String
   Tcl_Obj Obj(string);
   Tcl_Obj Obj(const string &arr[],int pos=0,int count=WHOLE_ARRAY);
   string String(Tcl_Obj);
   string String(Tcl_Obj,int index);
   string StringResult(int ignoreThis=0);
   
   string StringEval(string script,int flags=0); 
   string StringEval(Tcl_Obj script,int flags=0);
   string StringCall(Tcl_Obj command,Tcl_Obj arg1=0,Tcl_Obj arg2=0,Tcl_Obj arg3=0,Tcl_Obj arg4=0,Tcl_Obj arg5=0,Tcl_Obj arg6=0,Tcl_Obj arg7=0,Tcl_Obj arg8=0,Tcl_Obj arg9=0,Tcl_Obj arg10=0);
   string StringCall(Tcl_Obj &objv[],int objc=WHOLE_ARRAY);
   
   ///////////////////////////
   // Long
   Tcl_Obj Obj(long);
   Tcl_Obj Obj(const long &arr[],int pos=0,int count=WHOLE_ARRAY);
   long Long(Tcl_Obj);
   long Long(Tcl_Obj,int index);
   long LongResult(int ignoreThis=0);
   
   long LongEval(string script,int flags=0); 
   long LongEval(Tcl_Obj script,int flags=0);
   long LongCall(Tcl_Obj command,Tcl_Obj arg1=0,Tcl_Obj arg2=0,Tcl_Obj arg3=0,Tcl_Obj arg4=0,Tcl_Obj arg5=0,Tcl_Obj arg6=0,Tcl_Obj arg7=0,Tcl_Obj arg8=0,Tcl_Obj arg9=0,Tcl_Obj arg10=0);
   long LongCall(Tcl_Obj &objv[],int objc=WHOLE_ARRAY);
   
   ///////////////////////////
   // Datetime
   Tcl_Obj Obj(datetime);
   Tcl_Obj Obj(const datetime &arr[],int pos=0,int count=WHOLE_ARRAY);
   datetime Datetime(Tcl_Obj);
   datetime Datetime(Tcl_Obj,int index);
   datetime DatetimeResult(int ignoreThis=0);
   
   datetime DatetimeEval(string script,int flags=0); 
   datetime DatetimeEval(Tcl_Obj script,int flags=0);
   datetime DatetimeCall(Tcl_Obj command,Tcl_Obj arg1=0,Tcl_Obj arg2=0,Tcl_Obj arg3=0,Tcl_Obj arg4=0,Tcl_Obj arg5=0,Tcl_Obj arg6=0,Tcl_Obj arg7=0,Tcl_Obj arg8=0,Tcl_Obj arg9=0,Tcl_Obj arg10=0);
   datetime DatetimeCall(Tcl_Obj &objv[],int objc=WHOLE_ARRAY);
   
};

Seems to be pretty compact and usable.

The above zip-archive is attached, if someone needs it and has any problems with installation, post it here or in person

[Maxim Kuznetsov]

a roof-top example

"compiling on the fly and using C/C++ from ATcl inside an MQL script"

#property copyright "Maxim A.Kuznetsov"
#property link      "luxtrade.tk"
#property version   "1.00"
#property strict
#property description "ATcl demo"
#property description "------------"
#property description "This script demonstrate how to compile and use C/C++ on-the-fly"


#include <ATcl\\ATcl.mqh>

const string c=
"static double sum(double x, double y) {"
"    return x+y;"
"}";

ATcl *tcl;
void OnStart()
{
   ATcl_OnInit();
   tcl=new ATcl;
   if (tcl!=NULL && tcl.Ready()) {
      tcl.Eval("package require critcl");             // use critcl package (compiler environment)
      tcl.Call(tcl.Obj("critcl::ccode"),tcl.Obj(c));  // pass some c/c++ to compiler
      tcl.Eval("critcl::cproc mysum { double x double y } double { return sum(x,y);}");      // make callable proc
      double ret=tcl.DoubleCall(tcl.Obj("mysum"),tcl.Obj((double)5.0),tcl.Obj((double)7.1)); // call :-)
      PrintFormat("ret=%f",ret);
   }
   delete tcl;
   ATcl_OnDeinit();
}

[Maxim Kuznetsov]

Well, congratulations on the beta ! :-)

Briefly the features that through ATcl are now easily accessible from MT4 :

Tcl Features

• simple - simple scripting language
• Portable - running scripts and algorithms you can use in any system (Windows, Linux, Android and others)
• event driven - internal event model
• object orientation - all the modern OOP features.
• threading - you can use all the features of multithreading programming - system threads, mutexes, conditional variables, shared variables with the possibility of permanent storage.
• virtual file system - transparent access to zip,tar,ftp,webdav as a file system
• database - unified interface tdbc and sqlite are included in the tcl core. It actively supports PostresSQL,MySQL,MonetDB, Oracle and any ODBC database.
• Networking - client/server: tcp,http,mail,websocket,zmq. Virtually all protocols and services you might need.
• cryptography - checksums, electronic signatures, crypto hashes. Symmetric and asymmetric encryption. Public key infrastructure support, TLS, SASL.
• windows - windows environment tools - registry, DDE, COM client/server, ldap client
• tcllib - Tcl's standard library provides additional features (programming tools, text processing, maths and statistics, and more)
• c/c++ - The TinyC compiler is included in the libraries and the Critcl package allows the use of the system C/C++ compiler (vc,gcc,clang). Modules and functions can be compiled and used on the fly.
• Many libraries and systems provide a Tcl API, you can easily use them in your programs

more in the blog site: https://www.mql5.com/ru/blogs/post/715411 (which nobody reads :-))
and on the project page: http://luxtrade.tk/atcl:start

Happy to answer any questions on the subject and listen to ideas and comments.

[Andrey Khatimlianskii]

Congratulations!

It would be useful to do some practical examples, similar to what's in kodobase and articles.
For example, connecting to MySQL, Telegram or just connecting two terminals via TCP.

[Maxim Kuznetsov]

Andrey Khatimlianskii:

Congratulations!

It would be useful to do some practical examples, similar to what's in kodobase and articles.
For example, communication with MySQL, Telegram, or simply connecting two terminals via TCP.

in dems (in archive):

- A script saving quotes into a SQLite database. For MySQL it's the same, but not everyone has it :-)

- tcp server as indicator and advisor (two variants) - after start it accepts connection on port 8000 (can be set in settings at start) and translates there incoming ticks. You can telnet and see

- http client indicator, which periodically goes to the site, takes the jobs page, parses it and displays a summary of the information in the chart as Comment

demos will certainly continue to grow, you can propose ideas.

As we go along, I'll probably make demos illustrating

- "web server" to show that you can use a browser, and how it's done.

- use of TSV (thread shared variables), an extended analogue of GlobalVariable - use of shared data in the applications

I have to come up with more scripts :-) to make it clear, concise and at least a little useful.

[Maxim Kuznetsov]

There was a rather fascinating Skype conversation, some of which I think I need to voice because I find it useful :-)

1. Why some Tcl and not the much more popular Python, Ruby?

Of course, it will not be the first one because I know it well and I am using it not only for trading. And the technological point is very important - Python and Ruby are not compatible with metatrader threading-model. Theoretically they can be run, but I have to either dig out very low API or break the language. The same is true for R and many others.
To make them work, one must meet condition "several isolated instances can be created independently within one process and one thread".
Lua is the only other scripting language I know, but it is "a language without standards".
By the way, this bold type should be observed by all DLL developers in their products

2. How to properly and quickly analyze data

Correct and fast - is to perform calculations in metatrader. This is what metatrader is designed for - to receive data, perform calculations and issue orders. It would be foolish to duplicate its features in your DLL. After all, MQL itself is rather similar to C++ and it only takes a programmer a day or two to start writing in it. And in the DLL we have to do what the platform doesn't cover and physically can't cover - communication with databases, services, integration and so on. And plus the things that can and should be used outside MetaTrader.

3. Where and when is version 5 available?

As soon as possible :-) Technically it's not hard to drag and drop DLL from 4 to 5, scripts don't care. Tcl is both 32 and 64 bit. But it's much more convenient to develop on 4, it's not touched :-) it's stable and no surprises are expected. But with the 5 we will see, the platform developers promise "service tasks" (along with indicators, scripts and Expert Advisors) and surely something in relations of DLL and threads will change several builds in a row. Until this is settled, there will be no version for 5.

[Maxim Kuznetsov]

During the runtime and based on user feedback, a few problems were found related to msvcrt DLL dependencies.
and a tiny bug (mine) in finding the same DLL. Relative paths in the ATcl.mqh handler were prescribed :-)

all fixed, you can get the corrected version (attached), installation instructions are on the traditional place http://luxtrade.tk/atcl:install

For DLL developers, this is a useful piece of information:

- DLL compiled with define -D_CRTIMP_ALTERNATIVE which forces compiler to inline almost everything from CRT :-)

- The DLL is now built with the /MT switch, which forces the linker to use static linking and not cause unnecessary dependencies. The Redistributable (VC support package) is like not having to lug around everywhere

The DLL has grown in size, but there are fewer dependencies and it should be easier to lug the library around

PS/ small update - changed the attached archive. The library is built with full multithread support