Keeping the internal workings of an EA on an external server

 

I'm working on creating a free service for traders in which I can write proven manual systems into an automatic EA for people.


I have all the experience I need to write an EA just about as complicated as you could ask, however I am missing the knowledge to actually finish this project. I have a Linux box online which I can host whatever I need basically but I need some reading material or know how on how I can distribute free EA's to people that don't include any of the math to decide entries/exits from the market.


Basically what I want to do is have my server communicate with the client that they should enter or exit a trade based on price/indicator data sent from an EA I will provide to the client. The data sent from the client to my server will then need to be interpreted with math I could write to determine the entries and exits. So the goal is to protect how the EA works from the client so it cannot be de-compiled or copies (like ex4 and dll's can be).


Now knowing how to write a great EA is pretty useless if I don't know how to communicate from the clients ex4 to a server and in what sort of way would the server do these calculations, would it just be another ex4 file on the server or what?


Any reading or step-by-step how to's or anything would be really helpful.

 
bool SendFTP( string filename, string ftp_path=NULL) 
 

And how would that be used in the clients ex4 file? Just create a text file with all the data in it and send it to the FTP? And furthermore, what would I do on the server? And if I gave the client access to the FTP server then what would stop them from just using the FTP to store files or fill up the server with uploads in the case that I disallow downloading. The one line answers with no explanations are not very appreciated. I thought this is supposed to be a place of learning not a place full of arrogance?


What it seems like I need to do is write an application on my server which listens for updates to a file and then based on the name of the EA and the data passed to the server I can do the math needed to send back what action the clients EA should execute. If this is not possible then please tell me now. And if it is possible and anyone can give possible starting points for how to do this (both the ex4 communication and the server side integration) that would be largely appreciated.

 
decidence:

The one line answers with no explanations are not very appreciated. I thought this is supposed to be a place of learning not a place full of arrogance?

Whoa there. On the one hand, you're right that a curt mention of SendFTP() isn't very helpful. One the other hand, you're talking about a prospective service where a very large part of its value would lie in how, technologically, it was delivered, and you're expecting people to hand that to you for free. That's a whole different ballgame to the tips which people share on this site about writing EAs. You say that you're "missing the knowledge to actually finish this project". Personally, I'd say that you're missing the knowledge to start it. Accusing other people of arrogance isn't a good way to get help in this kind of situation.

 

Sorry if that sounded rude.


The part of the project I'm looking to learn may very well be the most important and could stop me entirely from pursuing delivering this service. But chronologically it is actually the final thing blocking me from making this project a reality which is why I say it is something I need to know how to do to "finish". However like I said, this service is something I'm looking to offer to traders of a community I frequent for free and is much smaller scale then you seem to be imagining thus I'm not looking for someone to teach me or mentor me at a cost, I am looking for freely available information like most of the internet is built around. Preferably something that can be duplicated quite easily for each small automated trading system I deliver to someone.


If the information I am looking for can be taught from a book on amazon or something then that may be something I would look into since I am a programmer by profession and learning this stuff is equally as interesting as any of my day to day responsibilities.


My main fear is that if I do not protect my work this way (since I can't think of any other way) is that someone will come to me with a great manual system they trade (or just something that could be marketable), I will automate it for them and then they will turn around and sell my work to hundreds of people without me ever knowing.

 
decidence:

Sorry if that sounded rude.


The part of the project I'm looking to learn may very well be the most important and could stop me entirely from pursuing delivering this service. But chronologically it is actually the final thing blocking me from making this project a reality which is why I say it is something I need to know how to do to "finish". However like I said, this service is something I'm looking to offer to traders of a community I frequent for free and is much smaller scale then you seem to be imagining thus I'm not looking for someone to teach me or mentor me at a cost, I am looking for freely available information like most of the internet is built around. Preferably something that can be duplicated quite easily for each small automated trading system I deliver to someone.


If the information I am looking for can be taught from a book on amazon or something then that may be something I would look into since I am a programmer by profession and learning this stuff is equally as interesting as any of my day to day responsibilities.


My main fear is that if I do not protect my work this way (since I can't think of any other way) is that someone will come to me with a great manual system they trade (or just something that could be marketable), I will automate it for them and then they will turn around and sell my work to hundreds of people without me ever knowing.

This 'final part' can probably be implemented only via a DLL or interfacing some external program, hence u r not likely to find your solution in this forum.

If you want to write it in C++ then go to a windows/networking C++ forum, etc. If u have questions/problems interfacing MQL4 to a DLL (in whatever language), then this is the place.


jjc is right though, u r talking about a technology-centric solution. Nobody is going to hand that out to you and it's probably a lot more complicated to program than your EA. Hence I agree with jjc - you're missing the knowledge to start it.

 

Your question seems to be how to implement a pub/sub architecture model, where MT4 client is the publisher and a web-server is the subscriber.


Assuming that you run the web-server, then you could try the following in order to avoid an additional DLL hop:


On MetaTrader:

1. Maintain your signal in the experts/files structure using MQL file handling operations. There are a couple of functions I wrote for someone else in this thread, but they should help. https://www.mql5.com/en/forum/123129

2. Use https://docs.mql4.com/common/SendFTP within your EA to deliver the file to your web server on each occasion you change your signal.


On your Web-server

1. Configure your FTP-server to have the ftp root somewhere that is accessible to your web-server; perhaps somewhere within htdocs if you're running Apache for example.

2. Use something like the php function file_get_contents to extract your info for inclusion into the appropriate html. I think this would be easier (and more architecturally correct) than maintaining a piece of php script and html within the experts/files folder.


CB

 
decidence:

[...] What it seems like I need to do is write an application on my server which listens for updates to a file and then based on the name of the EA and the data passed to the server I can do the math needed to send back what action the clients EA should execute. If this is not possible then please tell me now. And if it is possible and anyone can give possible starting points for how to do this (both the ex4 communication and the server side integration) that would be largely appreciated.

Switching back into helpfulness mode, albeit saying a few things which you may not want to hear. The technological bit comes towards the end..


As I understand it, you're talking about taking a manual system from somebody, automating it, and handing back the black-box in a form where they can't distribute it to other people without cutting you in. There are three skills and/or pieces of intellectual property here: (A) the manual trading system; (B) working out what the manual trader is doing and turning that into an unambiguous set of rules which can be run by a computer; and (C) coding it up in the specific form of a MetaTrader EA. Incidentally, B often isn't possible. Crudely speaking, that's why the programmers on Wall Street and in the City of London are very well paid, but don't earn as much as the traders.


(C) is a commodity. The likes of cloudbreaker and meikel have quite correctly reminded people many times that there's a world of difference between "an EA" versus a properly resilient, trustworthy EA. Nevertheless, step C by itself is the sort of thing where you get paid a flat rate, either for the job or by the hour. If your rate is too high, it's easy to find someone else to do the same job for less (e.g. $15). C is much less valuable than B, which in turn is much less valuable than A - if the system really works, which most don't. The trouble is that the line between B and C is blurred and, if you try to impose too many restrictions on people and what they can do with the EA you give them, they will simply go elsewhere in the belief that they're dealing with something entirely commoditised. I can't see your service working without establishing a really solid reputation for B rather than C. The obfuscation of the code is the next most important bit. The ability to write an EA is a very small part of what you're planning.


In technological terms, you have two choices: protect the code on the user's computer, or protect it by hiding part of it on a server. I've done both of these before personally, and neither is anything like simple. The server route is particularly nasty because it gives you latency issues and multiple points of failure. Personally, I wouldn't touch FTP because - as I understand it - the EA not only needs to send data to the server but also needs a response as well. With FTP you'd have to upload some data, have a mechanism on the server which spots the arrival of the new data, and then have the EA asynchronously check for a response. Ouch. Instead, I'd consider a route such as the following:


  • The client-side EA watches for potential signals, and assembles the bare minimum of data which can be sent to the server for signal-validation without giving the game away about the nature of the strategy and the work you've done in B.
  • The client-side EA then uses one of the many example blocks of HTTP code on this forum to send the data to the server on a synchronous basis - i.e. the EA waits for a response from the server.
  • A web page on the server (PHP, ASP.NET, whatever) looks at the data and responds with a fairly simple go/no-go message.


But, assuming that you currently have experience in neither DLLs nor web development, it's debatable whether this would be a flatter learning curve than putting the crucial segment of the code into a DLL. That route may be more secure than you think it is. It's certainly not unbreakable, but it doesn't have to be. The cost of getting something reverse-engineered out of a DLL simply has to be more than you're charging for your expertise in the form of B.

 

Interesting, I'm very thankful for all your responses they have been quite helpful.


JJC, you say that a DLL might actually be secure enough to deter most people from trying to reverse engineer something. This was my original thought over a week ago but I was told by another source that a DLL might be a poor choice as it is only "marginally" harder to decompile then an ex4.

 
decidence:

I was told by another source that a DLL might be a poor choice as it is only "marginally" harder to decompile then an ex4.

Your source is either very skilled in assembly language, or wrong. The output from decompiling a DLL is fundamentally different to the output from decompiling an ex4 file.

 

Haha ok maybe he is, he does seem very smart with that stuff. So I guess the conclusion I have come to then would be to compile most logic into a dll which I'm sure if I get stuck I could find most that information online for how a dll and ex4 talk to each other?


The last thing then is still in which manner the dll will authenticate the user. Should I make a MySQL database which stores keys and when a user turns on their EA the key is checked against the database and logs the IP address to make sure there isn't more then one person using the EA at the same time or does someone have a better/simpler authentication method that is less likely to result in false user lockouts? To me the IP logging doesn't seem very accurate.

Reason: