Discussion of article "Scraping bond yield data from the web"

New comment
 

New article Scraping bond yield data from the web has been published:

Automate the collection of interest rate data to improve the performance of an Expert Advisor.

Automated trading is based almost entirely on technical indicators that use past price action to forecast future price action. However, the trader who ignores fundamental forces that move markets is at a disadvantage to traders who factor fundamental data into their trading decisions. An indicator based on fundamental data gathered automatically can improve the performance of an Expert Advisor. The fundamental data that have the most effect on exchange rates are interest rates, which affect the perceived value of currencies. While central bank rates are not volatile, the yields on government bonds, such as the U.S. 10-year treasury note, fluctuate on all time frames in global bond markets. Those yields reflect the expectation the market has as to where future central bank rates will go. Bond yields are often a leading indicator of interest rates and of exchange rates. In the forex market, the metric that applies to a currency pair is the interest rate differential, especially the delta, or change in the interest rate differential, on various time frames. Figure 1 shows a case where movement of the interest rate differential, expressed in basis points, in the positive direction was a leading indicator of movement of the EUR/USD currency pair in the same direction. This article shows how to gather bond yield data from the web and to derive from that data expressions of interest rate differential and delta.

Interest rate differential leading indicator

Figure 1. Interest rate differential indicator on hourly EUR/USD chart.

Author: Steven Brown

 

this is what I get when I attach script05

and nothing is printed on the chart or indicator area...

what am I missing?


2019.04.14 17:11:18.623 Scripts script bonds (EURUSD,Daily) loaded successfully

2019.04.14 17:11:18.984 Scripts script bonds (EURUSD,Daily) removed



 
jimjack:

this is what I get when I attach script05

and nothing is printed on the chart or indicator area...

what am I missing?


2019.04.14 17:11:18.623 Scripts script bonds (EURUSD,Daily) loaded successfully

2019.04.14 17:11:18.984 Scripts script bonds (EURUSD,Daily) removed



ScraperBot06.mq4 works ok in MT5, and it does the same thing as ScraperBot05.mq5. The difference is that ScraperBot05 uses WebRequest(), while ScraperBot06 uses the functions in WinINet.dll, which seems to be more reliable than WebRequest(). I'll look into the problem to see if ScraperBot05.mq5 using WebRequest() can be made to work. It worked ok when I submitted the article in October 2018.

Edit: The call to WebRequest() in ScraperBot05.mql and other scripts will work only if the server named in the web request is allowed in Metatrader options. This is explained in the article. 

 
ScraperBot06.mq5 attached to this message is the same as ScraperBot05.mq5 but uses the functions in WinINet.dll instead of WebRequest(). I uploaded ScraperBot06.mq5 because WebRequest() in ScraperBot05.mq5 now returns error code -1, after which GetLastError() returns code 4014, 

ERR_FUNCTION_NOT_ALLOWED

4014

Function is not allowed for call

 

Metatrader 5 may have changed the implementation of WebRequest() since I submitted the article in October 2018, but the documentation for WebRequest() has not changed.

Edit: the problem is solved on my computer after allowing requests to the web server "https://api.wsj.net" in Metatrader options. Apparently, the permission was removed by a Metatrader update. 

Files:
ScraperBot06.mq5  7 kb
 
Steven Brown:
ScraperBot06.mq5 attached to this message is the same as ScraperBot05.mq5 but uses the functions in WinINet.dll instead of WebRequest(). I uploaded ScraperBot06.mq5 because WebRequest() in ScraperBot05.mq5 now returns error code -1, after which GetLastError() returns code 4014, 

ERR_FUNCTION_NOT_ALLOWED

4014

Function is not allowed for call

 

Metatrader 5 may have changed the implementation of WebRequest() since I submitted the article in October 2018, but the documentation for WebRequest() has not changed.




thank you for your reply.

I compiled it again and there's bunch of warning saying  "possible loss of data due to type conversion"

still no luck on seeing anything in the terminal. dll is allowed.

do I need any other packages from microsoft, .net etc...?


thanks



 
IMPORTANT! The call to WebRequest() in ScraperBot04.mq5 and ScraperBot05.mq5 will work only if the server named in the web request is allowed in Metatrader options. To enable a web server, click on Tools\Options, and in the dialog box that appears, click on the tab Expert Advisors. Check "Allow web request for listed URL," click "Add new URL," and enter the base URL of the web server. In the case of ScraperBot05, that URL is "https://api.wsj.net." If you do not do that, the call to WebRequest() will fail, reporting error codes -1 and 4014. This need to specifically allow a web server is mentioned in the article under the heading "ScraperBot04." I wrote "In order for WebRequest to work, the base URL for the server, in this case 'https://api.wsj.net', needs to be included in the list of allowed servers under "Tools\Options\Expert Advisors" in the MetaTrader platform."

I had the web server "https://api.wsj.net" allowed in my copy of Metatrader 5, but it was removed from the list by an update to a new version of Metatrader 5. That should not happen, and it appears to be a bug that should be fixed by Metaquotes. Once a web server is allowed, it should stay allowed through updates.  
 
jimjack:




thank you for your reply.

I compiled it again and there's bunch of warning saying  "possible loss of data due to type conversion"

still no luck on seeing anything in the terminal. dll is allowed.

do I need any other packages from microsoft, .net etc...?


thanks



The compiler warnings about loss of data can be ignored, because they are the result of using type float instead of type double. I chose to use type float because the precision of type double is not required in this application. After the warnings, compilation succeeds with zero errors. Did you download, compile, and try the script I attached to my reply, the one named ScraperBot06.mq5? It uses WinINet.dll instead of WebRequest(). As for the call to WebRequest() failing in ScraperBot05, do you have the web server enabled in Metatrader options? See my previous post, the one that begins "IMPORTANT!" I wrote in the article that WebRequest() will fail unless the web server is specifically allowed in the Options.
 

hmmm,

let me re-install  metatrader,  because the url was allowed from the beginning. yea I allowed wininet.dll and checked my win (10) directories... its there.

 
Thanks for the article, very interesting! Please report the results of using this system in real trading.
 
Thank you for the article. Do you, or anyone, know if there is already an indicator, not an EA, that can draw the interest rate differential or bond yield spread between two currencies?
 
@Steve Brown , Do you have the indicator codebase that would really be helpful. The scripts ain't that helpful even though they are a start, thank your for the great work and  for sharing we do appreciate it. if you post the indicator the community can work on it and expand on your idea. I am new to MT5 MQL5 programming though i am an expert in C++ programming , python and A.I , so my assumption is that no major learning for me with MT5 .
12
New comment
Reason: