Webrequest on page that initializes with javascript not working.

New comment
 
Hello community, i have tried pulling prices from etoro using webrequest but the problem
is that it returns javasript that the page initializes with before displaying main webpage. I treid increasing timeout but still don't get it.

link: 

I used FILE_BIN to save it to xml file and cookie (can be NULL) from my browser (chrome).
Below is the code snip and screenshot from the webpage.

If anyone has idea how to approach it, i would humbly appreciate.

The main page that i need i.e  to get GBPUSD price  and other things.

page

page initialization

page2


Code snip:

//+------------------------------------------------------------------+
//|                                              community_etoro.mq4 |
//|                                                              NAF |
//|                                             https://www.____.com |
//+------------------------------------------------------------------+
#property copyright "NAF"
#property link      "https://www.____.com"
#property version   "1.00"
#property strict

string Readfile()
  {
   Print("  1- ---start_web_reading");
   string cookie=NULL;
   string headers;
   char post[],result[],data[];     string TXT="";
   int res; string name="etoro.xml";

 string mr_url2="";

 int timeout=10000;  //timeout
   res=WebRequest("GET",mr_url2,cookie,NULL,timeout,post,0,result,headers);
   if(res==-1)
     {
      Print("  1  A- ---Web_Failed");
      Print("WebRequest error, err.code  =",GetLastError());
     }
   else
     {
      Print("  2- ---Web_reading_ -Sucessful");
      int filehandle=FileOpen(name,FILE_WRITE|FILE_BIN);
      if(filehandle!=INVALID_HANDLE)
        {
         FileWriteArray(filehandle,result,0,ArraySize(result));
         FileClose(filehandle);

         int filehandle2=FileOpen(name,FILE_READ|FILE_BIN);
         TXT=StringTrimLeft(StringTrimRight(FileReadString(filehandle2,ArraySize(result))));
         
         FileClose(filehandle2);
           }else{
         Print("Error in FileOpen. Error code =",GetLastError());
        }
     }
   return(TXT);
  }
 
//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
//---
 string TEXT=Readfile();
  }
//+------------------------------------------------------------------+



Sorry about links:  just use etoro/watchlist, i.e the web trader

 

It's not HTML.

 
Marco vd Heijden:

It's not HTML.


But why when I inspected using ctrl+shift+i on chrome, i am getting what i wanted?

 

Maybe it inspects the generated document object, so after it has been generated.

 
Marco vd Heijden:

Maybe it inspects the generated document object, so after it has been generated.


Probably. Let me try otherwise.

New comment