web scrapping system

[PEYMAN MEMARINAMIN]

Hi,

How can MQL palce an order based on a LIVE information published on a specific website (like fxstreet.com, etc...)

Is there any library tool for that ?

Thanks in advance,

PS: Please do not answer like "use Python". The question is a "how" not "what".

[PEYMAN MEMARINAMIN]

I would appreciate if you were more specific

[Flavio Javier Jarabeck]

pmemari:

Hi,

How can MQL palce an order based on a LIVE information published on a specific website (like fxstreet.com, etc...)

Is there any library tool for that ?

Thanks in advance,

PS: Please do not answer like "use Python". The question is a "how" not "what".

Ok...

You want, not only a Web Scraping method, but also an AI or NLP algorithm to put a BUY/or/SELL order based on market sentiment in MQL5, and, for FREE...    Is that it?

[PEYMAN MEMARINAMIN]

Flavio Jarabeck:

Ok...

You want, not only a Web Scraping method, but also an AI or NLP algorithm to put a BUY/or/SELL order based on market sentiment in MQL5, and, for FREE...    Is that it?

If it is for FREE, why not ! LOL

I would appreciate even if you introduce some off the shelf paid tool, library, code etc...

I do not see really where NLP or AI do in my LIVE data capture system. I want to give the MQL the exact target website and the specific data to be read in LIVE! so should not be as sophisticated as it seems

[Flavio Javier Jarabeck]

pmemari:
I do not see really where NLP or AI do in my LIVE data capture system. I want to give the MQL the exact target website and the specific data to be read in LIVE! so should not be as sophisticated as it seems

OK, good luck with that...

;)

[[Deleted]]

You have to code your DLL to screen scrap a website then do your include file and let your expert adviser use the library via the include file.

[PEYMAN MEMARINAMIN]

Zee Zhou Ma:

You have to code your DLL to screen scrap a website then do your include file and let your expert adviser use the library via the include file.

Thanks Zee, your comment was very helpfull.

any hint on how to write such DLL or do you know a person who sells these scrapping  DLL files ?

[nicholish en]

pmemari:

Thanks Zee, your comment was very helpfull.

any hint on how to write such DLL or do you know a person who sells these scrapping  DLL files ?

High level languages like python are made for tasks like web-[SCRAPE]-ing (not scrapping), and in fact it's so trivial I even posted the code here. If you want to create a dll from C libraries to do the same, then so be it... good luck. Most of the sources you want are dynamically rendered which will make it impossible for you to scrape with a basic http-url library. This means that you'll need to use something external to render to javascript OR you'll need to get clever and figure out to to target the right AJAX apis with your requests. If you're getting std JSON responses then this of course could be done in MQL, but then you'd still need to write all the parsing stuff from scratch.. What you want can be done, but at a great cost of developer time and you will pay a high price for a working solution.

If it's performance you're concerned about then just know that I/O latency is going to be your bottleneck, not the language implementation... And if you think you will get HFT news trading results from [anything] + MQL then you will be sorely disappointed. 

[PEYMAN MEMARINAMIN]

nicholi shen:

High level languages like python are made for tasks like web-[SCRAPE]-ing (not scrapping), and in fact it's so trivial I even posted the code here. If you want to create a dll from C libraries to do the same, then so be it... good luck. Most of the sources you want are dynamically rendered which will make it impossible for you to scrape with a basic http-url library. This means that you'll need to use something external to render to javascript OR you'll need to get clever and figure out to to target the right AJAX apis with your requests. If you're getting std JSON responses then this of course could be done in MQL, but then you'd still need to write all the parsing stuff from scratch.. What you want can be done, but at a great cost of developer time and you will pay a high price for a working solution.

If it's performance you're concerned about then just know that I/O latency is going to be your bottleneck, not the language implementation... And if you think you will get HFT news trading results from [anything] + MQL then you will be sorely disappointed. 

I just saw your answer to my other post.

Thank you so much! The world needs more guys like you

[OSCAR GUTIERREZ JIMENEZ]

PEYMAN MEMARINAMIN:

Hi,

How can MQL palce an order based on a LIVE information published on a specific website (like fxstreet.com, etc...)

Is there any library tool for that ?

Thanks in advance,

PS: Please do not answer like "use Python". The question is a "how" not "what".

https://github.com/thayton/fxstreet/blob/master/scraper.py