WebRequest(): "Accept: application/json" header not working?

Anatoli Kazharski 2022.01.20 15:24

Hi all!

I encountered an error when working with WebRequest(). First I will demonstrate the results from the command line.

We make a request without any headers and get this result. In this case everything is correct and should be so. The answer is returned in text/html format.

C:\Users\tol64>curl -I localhost:8000/api/user
HTTP/1.1 500 Internal Server Error
Host: localhost:8000
Date: Thu, 20 Jan 2022 12:16:21 GMT
Connection: close
X-Powered-By: PHP/7.4.6
Cache-Control: no-cache, private
date: Thu, 20 Jan 2022 12:16:21 GMT
Content-Type: text/html; charset=UTF-8
Vary: Origin

If you change the requested data type to application/json, you get the following result:

C:\Users\tol64>curl -H "Accept: application/json" -I localhost:8000/api/user
HTTP/1.1 401 Unauthorized
Host: localhost:8000
Date: Thu, 20 Jan 2022 12:17:02 GMT
Connection: close
X-Powered-By: PHP/7.4.6
Cache-Control: no-cache, private
Date: Thu, 20 Jan 2022 12:17:02 GMT
Content-Type: application/json
Vary: Origin

C:\Users\tol64>curl -H "Accept: application/json" localhost:8000/api/user
{"message":"Unauthenticated."}

//---

We always get the response when sending request from MetaTrader 5 with Accept: application/json header:

Content-Type: text/html; charset=UTF-8

//---

How can I get the expected result?

Everything works as expected on the command line, in Postman and in the browser application.

Mohammad Hossein Sadeghi 2022.01.20 16:13 #1

Please show your MQL code.

Anatoli Kazharski 2022.01.20 16:34 #2

Mohammad Hossein Sadeghi #:
Please show your MQL code.

The MQL code is very simple.

The only question is why the response comes with Content-Type: text/html; charset=UTF-8, if Accept: application/json is set in the request.

   string url = "http://localhost:8000/api/user";
   string request_headers = "Accept: application/json";
   uchar  request_data[];
   uchar  response_body[];
   string response_headers;
   int    response_code = -1;

   ResetLastError();
   response_code = WebRequest("GET", url, 
         request_headers, NULL, request_data, 
         response_body, response_headers);
   
   int error = GetLastError();
   Print("\nresponse_code: ", response_code, ", error: ", error);
   
   Print("[response_headers]");
   Print(response_headers);

//---

response_code: 500, error: 0
[response_headers]
Host: localhost
Date: Thu, 20 Jan 2022 16:25:15 GMT
Connection: close
X-Powered-By: PHP/7.4.6
Cache-Control: no-cache, private
date: Thu, 20 Jan 2022 16:25:15 GMT
Content-Type: text/html; charset=UTF-8
Vary: Origin

Laszlo Tormasi 2022.01.20 18:10 #3

Anatoli Kazharski #:

The MQL code is very simple.

The only question is why the response comes with Content-Type: text/html; charset=UTF-8, if Accept: application/json is set in the request.

//---

You are missing the \r\n at the end of the header.

Anatoli Kazharski 2022.01.20 19:04 #4

Laszlo Tormasi #:
You are missing the \r\n at the end of the header.

I tried this one (\r\n). This also does not work as expected.

Alain Verleyen 2022.01.20 19:17 #5

Anatoli Kazharski:

Hi all!

I encountered an error when working with WebRequest(). First I will demonstrate the results from the command line.

We make a request without any headers and get this result. In this case everything is correct and should be so. The answer is returned in text/html format.

If you change the requested data type to application/json, you get the following result:

//---

We always get the response when sending request from MetaTrader 5 with Accept: application/json header:

//---

How can I get the expected result?

Everything works as expected on the command line, in Postman and in the browser application.

Hi Anatoli,

I reported this issue years ago, I never got a reply from Metaquotes.

Anatoli Kazharski 2022.01.20 19:52 #6

Alain Verleyen #:

Hi Anatoli,

I reported this issue years ago, I never got a reply from Metaquotes.

Thank you for the information.

Форум по трейдингу, автоматическим торговым системам и тестированию торговых стратегий

Новая версия платформы MetaTrader 5 build 2007: Экономический календарь, MQL5-программы в виде сервисов

Alain Verleyen, 2019.04.16 03:50

Build 2007/2025.

Кажется, невозможно изменить заголовок «Accept» в WebRequest (с mql4 это работает).

Прикрепленный код должен возвращать JSON-ответ, например:

{ "responseParameters": { "doui_setResponseParameters": [ "8AA8D0CD6A05153A016A16735AF94E0E"]}}

Но ответ - HTML-файл, потому что серверу нужен заголовок «Accept: application / json». Я знаю это, потому что это работает с Почтальоном, например.

При проверке HTTP, отправляемого MT5 с Wireshark, заголовок «Принять» всегда:

Принять: * / *

Это ошибка или я что-то упустил?

MT5/mql5 reported and confirmed Metatester64.exe is broken in Time Issue in Version

Dominik Egert 2022.01.20 21:08 #7

I would like to point out, the end of a header is defined by <cr><lf><cr><lf>

https://en.wikipedia.org/wiki/List_of_HTTP_header_fields

This would mean, the server is not recognizing your header. Maybe you can check the logs.

Anyways, it's not compliant the way you do it, although I need to admit, I did not test it.

So maybe you could try \r\n\r\n at the end of the header definition.

Trade Request "Position Close", and again dll and CryptDecode with modifier CRYPT_ARCH_ZIP

Anatoli Kazharski 2022.01.20 21:16 #8

Dominik Christian Egert #:

I would like to point out, the end of a header is defined by <cr><lf><cr><lf>

https://en.wikipedia.org/wiki/List_of_HTTP_header_fields

This would mean, the server is not recognizing your header. Maybe you can check the logs.

Anyways, it's not compliant the way you do it, although I need to admit, I did not test it.

So maybe you could try \r\n\r\n at the end of the header definition.

All other headers in my tests are accepted by the server normally with \r\n.

string CProgram::GetHeaders(void) {
   string eol = "\r\n";
   string headers = "";
   headers += (m_xsrf_token != ""? "X-XSRF-TOKEN: " + m_xsrf_token + eol : "");
   headers += (m_cookie != ""? m_cookie + eol : "");
   headers += "Accept: application/json" + eol;
   headers += "Referer: localhost:3000";
   return headers;
}

The bug only appears with Accept: application/json. MetaTrader 5 replaces it with Accept: */*.

You can verify this with Wireshark:

Dominik Egert 2022.01.20 21:24 #9

Oh, great...

Thank you for replying.

Then you will need to modify the Apache cfg to have an appropriate mime definition.

Make a custom version and use something like application/mqljson

In the hope, mql does not manipulate this as well.

XP Moving Average! Indicators: NonLag Where did the "STI"

Alain Verleyen 2022.01.20 21:25 #10

Anatoli Kazharski #:

All other headers in my tests are accepted by the server normally with \r\n.

The bug only appears with Accept: application/json. MetaTrader 5 replaces it with Accept: */*.

You can verify this with Wireshark:

I worked around it using a Socket(), in my case it was suitable. But if Metaquotes can at least clarify why they are doing that it would be interesting.

1 2

New comment