(Issue) Webrequest returning invalid 404/403 - page 2

 
Last night I received a Webrequest error 403. Server had no issues. It seems the problem still exists. Problem does not appear to be persistent (it worked just now). Can anyone confirm what exactly was fixed in the latest update with regards to Webrequest?
 
Rob Josephus Maria Janssen #:
Last night I received a Webrequest error 403. Server had no issues. It seems the problem still exists. Problem does not appear to be persistent (it worked just now). Can anyone confirm what exactly was fixed in the latest update with regards to Webrequest?

Bug fixed in 5430.
If server drops the connection, client tries to reconnect with wrong url.

If you think there is a bug on MT5 side, you need to post source code and all details to reproduce it.

 

Alain, thank you!

This bug would exactly create the problem I am facing, because the URL I am requesting has an API key as a parameter. If this parameter becomes corrupted, a 403 would be returned.

Previously I was also receiving error 404s (page not found), so this would also corroborate sending the wrong URL.

As I have not yet had a 404, I therefore believe the problem could be partially fixed. As stated earlier, the problem is impossible to reproduce.

Sorry for stabbing in the dark here, I know this is not how things get resolved. Because of the API key I also cannot provide my source code: it is simply a URL with an API key as a parameter (using ?) using GET as the request. Context code is the same as the example provided on WebRequest

I do know that prior to version 3540 I never (not once in 4 years) have experienced this issue. Server reports no problems

Documentation on MQL5: WebRequest / Network Functions
Documentation on MQL5: WebRequest / Network Functions
  • www.mql5.com
The function sends an HTTP request to a specified server. The function has two versions: 1. Sending simple requests of type "key=value...
 
Rob Josephus Maria Janssen #:

Alain, thank you!

This bug would exactly create the problem I am facing, because the URL I am requesting has an API key as a parameter. If this parameter becomes corrupted, a 403 would be returned.

Previously I was also receiving error 404s (page not found), so this would also corroborate sending the wrong URL.

As I have not yet had a 404, I therefore believe the problem could be partially fixed. As stated earlier, the problem is impossible to reproduce.

Sorry for stabbing in the dark here, I know this is not how things get resolved. Because of the API key I also cannot provide my source code: it is simply a URL with an API key as a parameter (using ?) using GET as the request. Context code is the same as the example provided on WebRequest

I do know that prior to version 3540 I never (not once in 4 years) have experienced this issue. Server reports no problems

3540 ?

The previous build where WebRequest had not the bug fixed in 5430 was 5326.

If you can't provide minimal source code to reproduce the issue I don't see how I could help. I have to say that I hardly see how this error 403 is linked to the fixed bug. I am using WebRequest intensively, including with API key and I never experimented a corrupted parameter sent to the server.

 

Did you try install a proxy like Fiddler  and route MT5 through it to capture the raw HTTPS traffic? This way you can see exacly what MT5 is sending to your remote server when the WebRequest fires.

If you compare a “good” request (200 OK) and a “bad” one (403 / 404), you’ll see right away:

  • if the full URL is correct or if something gets chopped (missing ? , broken params, etc)

  • if the API key param is somehow corrupted or partially cut

  • if there are weird diffs in headers (Host, User-Agent, encoding stuff)

  • if MT5 is sometimes sending extra chars or wrong encoding

Since the bug is imposible to reproduce, capturing the trafic is probaly the only solid way to understand what’s really going on and check if the issue comes from MT5 itself, your URL build, or your backend

 
Laurent Xavier Richer #:

Did you try install a proxy like Fiddler  and route MT5 through it to capture the raw HTTPS traffic? This way you can see exacly what MT5 is sending to your remote server when the WebRequest fires.

If you compare a “good” request (200 OK) and a “bad” one (403 / 404), you’ll see right away:

  • if the full URL is correct or if something gets chopped (missing ? , broken params, etc)

  • if the API key param is somehow corrupted or partially cut

  • if there are weird diffs in headers (Host, User-Agent, encoding stuff)

  • if MT5 is sometimes sending extra chars or wrong encoding

Since the bug is imposible to reproduce, capturing the trafic is probaly the only solid way to understand what’s really going on and check if the issue comes from MT5 itself, your URL build, or your backend

Laurent, I am so sorry I just now see your reply (me and Alain have been talking about this issue on DM and still no resolve).

What I have discovered is that when I receive the (intermittent) 401, 403 or 404: as soon as I turn on Fiddler I receive a 200. When I turn Fiddler off again, I again receive the 401/403/404. 

This sounds like absolute bonkers, but I have tested this for a long time, going back and forth. It's almost like Fiddler creates a tunnel that prevents the error from occurring

PS: what I've also noticed is that when MT5 is run with Fiddler on, 'SendNotification' fails because of an 'invalid certificate'. Not directly related to the above problem but it does make me aware that Fiddler 'changes' the traffic going from terminal to server.

 
I had similar issue and this online free tool helps me to troubleshot. Be aware not to send sensitive data in your webrequest to the tool.



You can see what/how MT5 webrequest corrupts your request data, as the tool will send back your request data.


Good luck.