web scrapping system

To add comments, please log in or register
pmemari
102
pmemari  

Hi,

How can MQL palce an order based on a LIVE information published on a specific website (like fxstreet.com, etc...)

Is there any library tool for that ?


Thanks in advance,

PS: Please do not answer like "use Python". The question is a "how" not "what".

pmemari
102
pmemari  
I would appreciate if you were more specific
Flavio Jarabeck
10230
Flavio Jarabeck  
pmemari:

Hi,

How can MQL palce an order based on a LIVE information published on a specific website (like fxstreet.com, etc...)

Is there any library tool for that ?


Thanks in advance,

PS: Please do not answer like "use Python". The question is a "how" not "what".

Ok...

You want, not only a Web Scraping method, but also an AI or NLP algorithm to put a BUY/or/SELL order based on market sentiment in MQL5, and, for FREE...    Is that it?

pmemari
102
pmemari  
Flavio Jarabeck:

Ok...

You want, not only a Web Scraping method, but also an AI or NLP algorithm to put a BUY/or/SELL order based on market sentiment in MQL5, and, for FREE...    Is that it?

If it is for FREE, why not ! LOL

I would appreciate even if you introduce some off the shelf paid tool, library, code etc...

I do not see really where NLP or AI do in my LIVE data capture system. I want to give the MQL the exact target website and the specific data to be read in LIVE! so should not be as sophisticated as it seems

Flavio Jarabeck
10230
Flavio Jarabeck  
pmemari:
I do not see really where NLP or AI do in my LIVE data capture system. I want to give the MQL the exact target website and the specific data to be read in LIVE! so should not be as sophisticated as it seems

OK, good luck with that...

;)

Zee Zhou Ma
1882
Zee Zhou Ma  

You have to code your DLL to screen scrap a website then do your include file and let your expert adviser use the library via the include file.

pmemari
102
pmemari  
Zee Zhou Ma:

You have to code your DLL to screen scrap a website then do your include file and let your expert adviser use the library via the include file.

Thanks Zee, your comment was very helpfull.

any hint on how to write such DLL or do you know a person who sells these scrapping  DLL files ?
nicholi shen
2393
nicholi shen  
pmemari:

Thanks Zee, your comment was very helpfull.

any hint on how to write such DLL or do you know a person who sells these scrapping  DLL files ?

High level languages like python are made for tasks like web-[SCRAPE]-ing (not scrapping), and in fact it's so trivial I even posted the code here. If you want to create a dll from C libraries to do the same, then so be it... good luck. Most of the sources you want are dynamically rendered which will make it impossible for you to scrape with a basic http-url library. This means that you'll need to use something external to render to javascript OR you'll need to get clever and figure out to to target the right AJAX apis with your requests. If you're getting std JSON responses then this of course could be done in MQL, but then you'd still need to write all the parsing stuff from scratch.. What you want can be done, but at a great cost of developer time and you will pay a high price for a working solution.


If it's performance you're concerned about then just know that I/O latency is going to be your bottleneck, not the language implementation... And if you think you will get HFT news trading results from [anything] + MQL then you will be sorely disappointed. 

pmemari
102
pmemari  
nicholi shen:

High level languages like python are made for tasks like web-[SCRAPE]-ing (not scrapping), and in fact it's so trivial I even posted the code here. If you want to create a dll from C libraries to do the same, then so be it... good luck. Most of the sources you want are dynamically rendered which will make it impossible for you to scrape with a basic http-url library. This means that you'll need to use something external to render to javascript OR you'll need to get clever and figure out to to target the right AJAX apis with your requests. If you're getting std JSON responses then this of course could be done in MQL, but then you'd still need to write all the parsing stuff from scratch.. What you want can be done, but at a great cost of developer time and you will pay a high price for a working solution.


If it's performance you're concerned about then just know that I/O latency is going to be your bottleneck, not the language implementation... And if you think you will get HFT news trading results from [anything] + MQL then you will be sorely disappointed. 

I just saw your answer to my other post.

Thank you so much! The world needs more guys like you

To add comments, please log in or register