Discussion of article "Scraping bond yield data from the web"

To add comments, please log in or register
MetaQuotes Software Corp.
Moderator
205576
MetaQuotes Software Corp.  

New article Scraping bond yield data from the web has been published:

Automate the collection of interest rate data to improve the performance of an Expert Advisor.

Automated trading is based almost entirely on technical indicators that use past price action to forecast future price action. However, the trader who ignores fundamental forces that move markets is at a disadvantage to traders who factor fundamental data into their trading decisions. An indicator based on fundamental data gathered automatically can improve the performance of an Expert Advisor. The fundamental data that have the most effect on exchange rates are interest rates, which affect the perceived value of currencies. While central bank rates are not volatile, the yields on government bonds, such as the U.S. 10-year treasury note, fluctuate on all time frames in global bond markets. Those yields reflect the expectation the market has as to where future central bank rates will go. Bond yields are often a leading indicator of interest rates and of exchange rates. In the forex market, the metric that applies to a currency pair is the interest rate differential, especially the delta, or change in the interest rate differential, on various time frames. Figure 1 shows a case where movement of the interest rate differential, expressed in basis points, in the positive direction was a leading indicator of movement of the EUR/USD currency pair in the same direction. This article shows how to gather bond yield data from the web and to derive from that data expressions of interest rate differential and delta.

Interest rate differential leading indicator

Figure 1. Interest rate differential indicator on hourly EUR/USD chart.

Author: Steven Brown

jimjack
15
jimjack  

this is what I get when I attach script05

and nothing is printed on the chart or indicator area...

what am I missing?


2019.04.14 17:11:18.623 Scripts script bonds (EURUSD,Daily) loaded successfully

2019.04.14 17:11:18.984 Scripts script bonds (EURUSD,Daily) removed



Steven Brown
658
Steven Brown  
jimjack:

this is what I get when I attach script05

and nothing is printed on the chart or indicator area...

what am I missing?


2019.04.14 17:11:18.623 Scripts script bonds (EURUSD,Daily) loaded successfully

2019.04.14 17:11:18.984 Scripts script bonds (EURUSD,Daily) removed



ScraperBot06.mq4 works ok in MT5, and it does the same thing as ScraperBot05.mq5. The difference is that ScraperBot05 uses WebRequest(), while ScraperBot06 uses the functions in WinINet.dll, which seems to be more reliable than WebRequest(). I'll look into the problem to see if ScraperBot05.mq5 using WebRequest() can be made to work. It worked ok when I submitted the article in October 2018.

Edit: The call to WebRequest() in ScraperBot05.mql and other scripts will work only if the server named in the web request is allowed in Metatrader options. This is explained in the article. 

Steven Brown
658
Steven Brown  
ScraperBot06.mq5 attached to this message is the same as ScraperBot05.mq5 but uses the functions in WinINet.dll instead of WebRequest(). I uploaded ScraperBot06.mq5 because WebRequest() in ScraperBot05.mq5 now returns error code -1, after which GetLastError() returns code 4014, 

ERR_FUNCTION_NOT_ALLOWED

4014

Function is not allowed for call

 

Metatrader 5 may have changed the implementation of WebRequest() since I submitted the article in October 2018, but the documentation for WebRequest() has not changed.

Edit: the problem is solved on my computer after allowing requests to the web server "https://api.wsj.net" in Metatrader options. Apparently, the permission was removed by a Metatrader update. 

Files:
jimjack
15
jimjack  
Steven Brown:
ScraperBot06.mq5 attached to this message is the same as ScraperBot05.mq5 but uses the functions in WinINet.dll instead of WebRequest(). I uploaded ScraperBot06.mq5 because WebRequest() in ScraperBot05.mq5 now returns error code -1, after which GetLastError() returns code 4014, 

ERR_FUNCTION_NOT_ALLOWED

4014

Function is not allowed for call

 

Metatrader 5 may have changed the implementation of WebRequest() since I submitted the article in October 2018, but the documentation for WebRequest() has not changed.




thank you for your reply.

I compiled it again and there's bunch of warning saying  "possible loss of data due to type conversion"

still no luck on seeing anything in the terminal. dll is allowed.

do I need any other packages from microsoft, .net etc...?


thanks



Steven Brown
658
Steven Brown  
IMPORTANT! The call to WebRequest() in ScraperBot04.mq5 and ScraperBot05.mq5 will work only if the server named in the web request is allowed in Metatrader options. To enable a web server, click on Tools\Options, and in the dialog box that appears, click on the tab Expert Advisors. Check "Allow web request for listed URL," click "Add new URL," and enter the base URL of the web server. In the case of ScraperBot05, that URL is "https://api.wsj.net." If you do not do that, the call to WebRequest() will fail, reporting error codes -1 and 4014. This need to specifically allow a web server is mentioned in the article under the heading "ScraperBot04." I wrote "In order for WebRequest to work, the base URL for the server, in this case 'https://api.wsj.net', needs to be included in the list of allowed servers under "Tools\Options\Expert Advisors" in the MetaTrader platform."

I had the web server "https://api.wsj.net" allowed in my copy of Metatrader 5, but it was removed from the list by an update to a new version of Metatrader 5. That should not happen, and it appears to be a bug that should be fixed by Metaquotes. Once a web server is allowed, it should stay allowed through updates.  
Steven Brown
658
Steven Brown  
jimjack:




thank you for your reply.

I compiled it again and there's bunch of warning saying  "possible loss of data due to type conversion"

still no luck on seeing anything in the terminal. dll is allowed.

do I need any other packages from microsoft, .net etc...?


thanks



The compiler warnings about loss of data can be ignored, because they are the result of using type float instead of type double. I chose to use type float because the precision of type double is not required in this application. After the warnings, compilation succeeds with zero errors. Did you download, compile, and try the script I attached to my reply, the one named ScraperBot06.mq5? It uses WinINet.dll instead of WebRequest(). As for the call to WebRequest() failing in ScraperBot05, do you have the web server enabled in Metatrader options? See my previous post, the one that begins "IMPORTANT!" I wrote in the article that WebRequest() will fail unless the web server is specifically allowed in the Options.
jimjack
15
jimjack  

hmmm,

let me re-install  metatrader,  because the url was allowed from the beginning. yea I allowed wininet.dll and checked my win (10) directories... its there.

To add comments, please log in or register