Send headers via WebRequest()

Eric Sebestyen Ferreira 2019.04.15 13:47

Hi,

I am trying to use WebRequest to send a header and get a response. Basically I have the follow parameters:

url: "http://www.b3.com.br/main.jsp?lumPageId=8A488AE9543E47C601543FA7E9807C6F&lumA=1&lumII=8A488AE9543E47C601543FA9249D027A&doui_processActionId=commit&doui_fromForm=Form_8A488AE9543E47C601543FA9249D027A"

header="Accept:application/json, text/javascript, */*; q=0.0\r\nContent-Type:application/x-www-form-urlencoded"

data:"dataDownload:12/04/2019&tipo:PE"

If I put this parameters on POSTMAN, exactly as it is I get the response:

{"responseParameters":{"doui_setResponseParameters":["8AA8D0CD6A05153A016A16735AF94E0E"]}}

PostMan

But Whe I try a single script with WebRequest() I only receive the web site html code:

I really,really,really apreciate if someone can help me.

void OnStart() 
  { 
   string cookie=NULL,referer=NULL,headers="",header; 
   char   post[],result[];
//url 
   string url="http://www.b3.com.br/main.jsp?lumPageId=8A488AE9543E47C601543FA7E9807C6F&lumA=1&lumII=8A488AE9543E47C601543FA9249D027A&doui_processActionId=commit&doui_fromForm=Form_8A488AE9543E47C601543FA9249D027A";
   char data[];

//header  
   header="Accept:application/json, text/javascript, */*; q=0.0\r\nContent-Type:application/x-www-form-urlencoded";

//  data
  StringToCharArray("dataDownload:12/04/2019&tipo:PE",data);

  int res=WebRequest("POST",url,header,10000,data,result,headers);

  string mais= CharArrayToString(result);
   

   if(res==-1) 
     { 
      Print("Erro no WebRequest. Código de erro =",GetLastError()); 
      //--- é possível que a URL não esteja na lista, exibimos uma mensagem sobre a necessidade de adicioná-la 
      MessageBox("É necessário adicionar um endereço '"+url+"' à lista de URL permitidas na guia 'Experts'","Erro",MB_ICONINFORMATION); 
     } 
   else 
     { 
      if(res==200) 
        { 
         //--- download bem-sucedido 
         PrintFormat("O arquivo foi baixado com sucesso, tamanho %d bytes.",ArraySize(result)); 
         //PrintFormat("Cabeçalhos do servidor: %s",headers); 
         //--- salvamos os dados em um arquivo 
         int filehandle=FileOpen("url.txt",FILE_WRITE|FILE_BIN); 
         if(filehandle!=INVALID_HANDLE) 
           { 
            //--- armazenamos o conteúdo do array result[] no arquivo 
            FileWriteArray(filehandle,result,0,ArraySize(result)); 
            //--- fechamos o arquivo 
            FileClose(filehandle); 
           } 
         else 
            Print("Erro em FileOpen. Código de erro =",GetLastError()); 
        } 
      else 
         PrintFormat("Erro de download '%s', código %d",url,res); 
     }

Alain Verleyen 2019.04.15 14:29 #1

You need to format your headers correctly. It should finished by "\r\n".

Eric Sebestyen Ferreira 2019.04.15 14:54 #2

I apreciate your comment! I put "\r\n" but I still got the html page as result, I am expeting receive:

{"responseParameters":{"doui_setResponseParameters":["8AA8D0CD6A05153A016A16735AF94E0E"]}}

If you have any other suggestion, I spent the weekend tring to receive this message :(

void OnStart() 
  { 
   string cookie=NULL,referer=NULL,headers="",header; 
   char   post[],result[]; 
//url
   string url="http://www.b3.com.br/main.jsp?lumPageId=8A488AE9543E47C601543FA7E9807C6F&lumA=1&lumII=8A488AE9543E47C601543FA9249D027A&doui_processActionId=commit&doui_fromForm=Form_8A488AE9543E47C601543FA9249D027A";
   char data[];
//header
  
   header="Accept:application/json, text/javascript, */*; q=0.0\r\nContent-Type:application/x-www-form-urlencoded\r\n";

//data
 // StringToCharArray("dataDownload=12%2F04%2F2019&tipo=PE",data);
    StringToCharArray("dataDownload:12/04/2019&tipo:PE",data);

  int res=WebRequest("POST",url,header,10000,data,result,headers);

  string mais= CharArrayToString(result);
   

   if(res==-1) 
     { 
      Print("Erro no WebRequest. Código de erro =",GetLastError()); 
      //--- é possível que a URL não esteja na lista, exibimos uma mensagem sobre a necessidade de adicioná-la 
      MessageBox("É necessário adicionar um endereço '"+url+"' à lista de URL permitidas na guia 'Experts'","Erro",MB_ICONINFORMATION); 
     } 
   else 
     { 
      if(res==200) 
        { 
         //--- download bem-sucedido 
         PrintFormat("O arquivo foi baixado com sucesso, tamanho %d bytes.",ArraySize(result)); 
         //PrintFormat("Cabeçalhos do servidor: %s",headers); 
         //--- salvamos os dados em um arquivo 
         int filehandle=FileOpen("url.txt",FILE_WRITE|FILE_BIN); 
         if(filehandle!=INVALID_HANDLE) 
           { 
            //--- armazenamos o conteúdo do array result[] no arquivo 
            FileWriteArray(filehandle,result,0,ArraySize(result)); 
            //--- fechamos o arquivo 
            FileClose(filehandle); 
           } 
         else 
            Print("Erro em FileOpen. Código de erro =",GetLastError()); 
        } 
      else 
         PrintFormat("Erro de download '%s', código %d",url,res); 
     } 
  }

'Close all'/'Open' tools Down Arrow on current MT5/mql5 reported and confirmed

Eric Sebestyen Ferreira 2019.04.15 16:34 #3

Just to you see I did a very single macro on excel with the same parameters, if you run it it goes ok, this shows that the parameters are ok

Sub macroPOST()

    Set objHTTP = CreateObject("MSXML2.ServerXMLHTTP")

    Url = "http://www.b3.com.br/main.jsp?lumPageId=8A488AE9543E47C601543FA7E9807C6F&lumA=1&lumII=8A488AE9543E47C601543FA9249D027A&doui_processActionId=commit&doui_fromForm=Form_8A488AE9543E47C601543FA9249D027A"

    objHTTP.Open "POST", Url, False

    objHTTP.setRequestHeader "Accept", "application/json"

    objHTTP.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"

    objHTTP.Send ("dataDownload=12/04/2019&tipo=PE")

    replyTXT = objHTTP.responseText
  

    If objHTTP.Status = "200" Then 'success

        MsgBox replyTXT

    Else

        'Do something

    End If

End Sub

Alain Verleyen 2019.04.15 17:06 #4

ericsebestyen:

I apreciate your comment! I put "\r\n" but I still got the html page as result, I am expeting receive:

{"responseParameters":{"doui_setResponseParameters":["8AA8D0CD6A05153A016A16735AF94E0E"]}}

If you have any other suggestion, I spent the weekend tring to receive this message :(

You would need to check the HTTP request send by mql and Postman with a tool like Wireshark. HTTP protocol is very sensitive and there is probably a small detail somewhere that differs.

Eric Sebestyen Ferreira 2019.04.15 19:15 #5

I have checked the strings on WS it is the same, it is very strange, I really lost my head trying solve it, in VBA excel and POSTMAN it works, but in mql5 not, I found some related posts like this

https://www.mql5.com/en/code/12296 but got no solution.

What do you recommend in this case? it looks to be a bug, I tried every possible combination, no results.

Automatic Posting with WebRequest()

www.mql5.com

The WebRequest function has two call options, the first option is used to send simple queries, the example of it is available in the script. The second function option allows you to create queries of any type (GET, POST, HEAD, etc.) set of headers. This options allows implementing flexible interaction with various Web services. This script is...

Questions: to start with Teach me how to DAILY BREAKOUT EA ,

Alain Verleyen 2019.04.15 21:09 #6

ericsebestyen:

I have checked the strings on WS it is the same, it is very strange, I really lost my head trying solve it, in VBA excel and POSTMAN it works, but in mql5 not, I found some related posts like this

https://www.mql5.com/en/code/12296 but got no solution.

What do you recommend in this case? it looks to be a bug, I tried every possible combination, no results.

I don't think it's a bug. What "strings" have you checked ? The requests must be different somehow, check each header, each separator etc...

E5DCDF5E ada 2019.04.15 21:50 #7

ericsebestyen:

I apreciate your comment! I put "\r\n" but I still got the html page as result, I am expeting receive:

{"responseParameters":{"doui_setResponseParameters":["8AA8D0CD6A05153A016A16735AF94E0E"]}}

If you have any other suggestion, I spent the weekend tring to receive this message :(

Eric Sebestyen Ferreira 2019.04.15 22:05 #8

Well I can post later the results from wire shark but I created a order, only the file I'd will be enough.

https://www.mql5.com/en/job/97010

Freelance service at MQL5.com: Download Automaticaly file

www.mql5.com

Hi, I have an EA that is feed with a file wich need to be extracted from internet every day from a web site. To do this mannualy I need to click on a button in this site and the file is downloaded (picture bellow), but I would like to do this automatically, Using WebRequest() The file must be downloaded on actual date of the form to "File" folder and then unziped there overwirting the older file. http://www.b3.com...

Nikolai Karetnikov 2020.06.01 04:38 #9

Eric Sebestyen Ferreira:

Well I can post later the results from wire shark but I created a order, only the file I'd will be enough.

https://www.mql5.com/en/job/97010

Hi Eric Sebestyen Ferreira!

Have you figured out what was the problem?

I encountered a similar issue

postman works as expected, WebRequest does not

New comment