WebRequest to download ForexFactory News
Goutagny Philippe:
Hi
My code to download FF news don't work anymore since this WE ! Anyone using webrequest to download news has this problem ??
result = 0 bytes downloaded. No modification in the code since last week when it was always running well since months.
both webpages https://cdn-nfs.forexfactory.net/ff_calendar_thisweek.xml and http://www.forexfactory.com/ffcal_week_this.xml always working if paste in Internet Explorer.
string cookie=NULL,headers;
char post[],result[];
int res=0;
//--- to enable access to the server, you should add URL
//--- in the list of allowed URLs (Main Menu->Tools->Options, tab "Expert Advisors"):
string lURL="https://cdn-nfs.forexfactory.net/ff_calendar_thisweek.xml"; //avant 3/3/17 -> "http://www.forexfactory.com/ffcal_week_this.xml";
//--- Reset the last error code
ResetLastError();
//--- Loading a html page from Google Finance
int timeout=5000; //--- Timeout below 1000 (1 sec.) is not enough for slow Internet connection
if(gExt_ForceLectureFichierManuel==false)
{
res=WebRequest("GET",lURL,cookie,NULL,timeout,post,0,result,headers);
}
I have the same problem, result = 9 bytes downloaded with this content ("forbidden")
Sergio Putzu:
I have the same problem, result = 9 bytes downloaded with this content ("forbidden")
The forexfactory.net URL works OK for me. It looks like they have blocked some user-agents on the .com URL. I guess they want to move the RSS traffic over to the .net URL. I got the .com URL to work by changing the user-agent.I have the same problem, result = 9 bytes downloaded with this content ("forbidden")
void OnStart()
{
string cookie=NULL, headers;
string reqheaders="User-Agent: Mozilla/4.0\r\n";
char post[],result[];
int res;
string url="http://www.forexfactory.com/ffcal_week_this.xml";
ResetLastError();
int timeout=5000;
res=WebRequest("GET",url,reqheaders,timeout,post,result,headers);
if(res==-1)
{
Print("Error in WebRequest. Error code =",GetLastError());
//--- Perhaps the URL is not listed, display a message about the necessity to add the address
MessageBox("Add the address '"+url+"' in the list of allowed URLs on tab 'Expert Advisors'","Error",MB_ICONINFORMATION);
}
else
{
//--- Load successfully
PrintFormat("The file has been successfully loaded, File size =%d bytes.",ArraySize(result));
//--- Save the data to a file
int filehandle=FileOpen("test.xml",FILE_WRITE|FILE_BIN);
//--- Checking errors
if(filehandle!=INVALID_HANDLE)
{
//--- Save the contents of the result[] array to a file
FileWriteArray(filehandle,result,0,ArraySize(result));
//--- Close the file
FileClose(filehandle);
}
else Print("Error in FileOpen. Error code=",GetLastError());
}
}
{
string cookie=NULL, headers;
string reqheaders="User-Agent: Mozilla/4.0\r\n";
char post[],result[];
int res;
string url="http://www.forexfactory.com/ffcal_week_this.xml";
ResetLastError();
int timeout=5000;
res=WebRequest("GET",url,reqheaders,timeout,post,result,headers);
if(res==-1)
{
Print("Error in WebRequest. Error code =",GetLastError());
//--- Perhaps the URL is not listed, display a message about the necessity to add the address
MessageBox("Add the address '"+url+"' in the list of allowed URLs on tab 'Expert Advisors'","Error",MB_ICONINFORMATION);
}
else
{
//--- Load successfully
PrintFormat("The file has been successfully loaded, File size =%d bytes.",ArraySize(result));
//--- Save the data to a file
int filehandle=FileOpen("test.xml",FILE_WRITE|FILE_BIN);
//--- Checking errors
if(filehandle!=INVALID_HANDLE)
{
//--- Save the contents of the result[] array to a file
FileWriteArray(filehandle,result,0,ArraySize(result));
//--- Close the file
FileClose(filehandle);
}
else Print("Error in FileOpen. Error code=",GetLastError());
}
}
Tim Morris:
The forexfactory.net URL works OK for me. It looks like they have blocked some user-agents on the .com URL. I guess they want to move the RSS traffic over to the .net URL. I got the .com URL to work by changing the user-agent.
The forexfactory.net URL works OK for me. It looks like they have blocked some user-agents on the .com URL. I guess they want to move the RSS traffic over to the .net URL. I got the .com URL to work by changing the user-agent.
void OnStart()
{
string cookie=NULL, headers;
string reqheaders="User-Agent: Mozilla/4.0\r\n";
char post[],result[];
int res;
string url="http://www.forexfactory.com/ffcal_week_this.xml";
ResetLastError();
int timeout=5000;
res=WebRequest("GET",url,reqheaders,timeout,post,result,headers);
if(res==-1)
{
Print("Error in WebRequest. Error code =",GetLastError());
//--- Perhaps the URL is not listed, display a message about the necessity to add the address
MessageBox("Add the address '"+url+"' in the list of allowed URLs on tab 'Expert Advisors'","Error",MB_ICONINFORMATION);
}
else
{
//--- Load successfully
PrintFormat("The file has been successfully loaded, File size =%d bytes.",ArraySize(result));
//--- Save the data to a file
int filehandle=FileOpen("test.xml",FILE_WRITE|FILE_BIN);
//--- Checking errors
if(filehandle!=INVALID_HANDLE)
{
//--- Save the contents of the result[] array to a file
FileWriteArray(filehandle,result,0,ArraySize(result));
//--- Close the file
FileClose(filehandle);
}
else Print("Error in FileOpen. Error code=",GetLastError());
}
}
{
string cookie=NULL, headers;
string reqheaders="User-Agent: Mozilla/4.0\r\n";
char post[],result[];
int res;
string url="http://www.forexfactory.com/ffcal_week_this.xml";
ResetLastError();
int timeout=5000;
res=WebRequest("GET",url,reqheaders,timeout,post,result,headers);
if(res==-1)
{
Print("Error in WebRequest. Error code =",GetLastError());
//--- Perhaps the URL is not listed, display a message about the necessity to add the address
MessageBox("Add the address '"+url+"' in the list of allowed URLs on tab 'Expert Advisors'","Error",MB_ICONINFORMATION);
}
else
{
//--- Load successfully
PrintFormat("The file has been successfully loaded, File size =%d bytes.",ArraySize(result));
//--- Save the data to a file
int filehandle=FileOpen("test.xml",FILE_WRITE|FILE_BIN);
//--- Checking errors
if(filehandle!=INVALID_HANDLE)
{
//--- Save the contents of the result[] array to a file
FileWriteArray(filehandle,result,0,ArraySize(result));
//--- Close the file
FileClose(filehandle);
}
else Print("Error in FileOpen. Error code=",GetLastError());
}
}
Thanks a lot Tim !
Goutagny Philippe:
Thanks a lot Tim !
Tim Morris:
The forexfactory.net URL works OK for me. It looks like they have blocked some user-agents on the .com URL. I guess they want to move the RSS traffic over to the .net URL. I got the .com URL to work by changing the user-agent.
The forexfactory.net URL works OK for me. It looks like they have blocked some user-agents on the .com URL. I guess they want to move the RSS traffic over to the .net URL. I got the .com URL to work by changing the user-agent.
void OnStart()
{
string cookie=NULL, headers;
string reqheaders="User-Agent: Mozilla/4.0\r\n";
char post[],result[];
int res;
string url="http://www.forexfactory.com/ffcal_week_this.xml";
ResetLastError();
int timeout=5000;
res=WebRequest("GET",url,reqheaders,timeout,post,result,headers);
if(res==-1)
{
Print("Error in WebRequest. Error code =",GetLastError());
//--- Perhaps the URL is not listed, display a message about the necessity to add the address
MessageBox("Add the address '"+url+"' in the list of allowed URLs on tab 'Expert Advisors'","Error",MB_ICONINFORMATION);
}
else
{
//--- Load successfully
PrintFormat("The file has been successfully loaded, File size =%d bytes.",ArraySize(result));
//--- Save the data to a file
int filehandle=FileOpen("test.xml",FILE_WRITE|FILE_BIN);
//--- Checking errors
if(filehandle!=INVALID_HANDLE)
{
//--- Save the contents of the result[] array to a file
FileWriteArray(filehandle,result,0,ArraySize(result));
//--- Close the file
FileClose(filehandle);
}
else Print("Error in FileOpen. Error code=",GetLastError());
}
}
{
string cookie=NULL, headers;
string reqheaders="User-Agent: Mozilla/4.0\r\n";
char post[],result[];
int res;
string url="http://www.forexfactory.com/ffcal_week_this.xml";
ResetLastError();
int timeout=5000;
res=WebRequest("GET",url,reqheaders,timeout,post,result,headers);
if(res==-1)
{
Print("Error in WebRequest. Error code =",GetLastError());
//--- Perhaps the URL is not listed, display a message about the necessity to add the address
MessageBox("Add the address '"+url+"' in the list of allowed URLs on tab 'Expert Advisors'","Error",MB_ICONINFORMATION);
}
else
{
//--- Load successfully
PrintFormat("The file has been successfully loaded, File size =%d bytes.",ArraySize(result));
//--- Save the data to a file
int filehandle=FileOpen("test.xml",FILE_WRITE|FILE_BIN);
//--- Checking errors
if(filehandle!=INVALID_HANDLE)
{
//--- Save the contents of the result[] array to a file
FileWriteArray(filehandle,result,0,ArraySize(result));
//--- Close the file
FileClose(filehandle);
}
else Print("Error in FileOpen. Error code=",GetLastError());
}
}
string url="https://nfs.faireconomy.media/ff_calendar_thisweek.xml"; update
On the same topic. Once the file is downloaded, how do you work with it? Here is my file download:
void WebData() { string cookie=NULL, headers; string reqheaders="User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19042\r\n"; char post[],result[]; double r[]; int res; string url="https://data.bitcoinity.org/export_data.csv?c=e&data_type=volume&r=hour&t=b×pan=7d"; ResetLastError(); int timeout=5000; res=WebRequest("GET",url,reqheaders,timeout,post,result,headers); if(res==-1) { Print("Error in WebRequest. Error code =",GetLastError()); //--- Perhaps the URL is not listed, display a message about the necessity to add the address MessageBox("Add the address '"+url+"' in the list of allowed URLs on tab 'Expert Advisors'","Error",MB_ICONINFORMATION); } else { //--- Load successfully PrintFormat("The file has been successfully loaded, File size =%d bytes.",ArraySize(result)); //--- Save the data to a file int filehandle=FileOpen("test.csv",FILE_WRITE|FILE_BIN); //--- Checking errors if(filehandle!=INVALID_HANDLE) { //--- Save the contents of the result[] array to a file FileWriteArray(filehandle,result,0,ArraySize(result)); //--- Close the file FileClose(filehandle); } else Print("Error in FileOpen. Error code=",GetLastError()); } }
This works, I can open the file in notepad. Now how do I extract the data into a structure or even just an array, but I have no idea how to do this. I've searched the forum and read many examples.
int file = FileOpen("test.csv",FILE_READ|FILE_TXT|FILE_COMMON); if(file != INVALID_HANDLE) { while(!FileIsEnding(file) && !_StopFlag) { FileReadString(file); } FileClose(file); }
Code above doesn't work. Doesn't work if It's FILE_CSV. I tried FILE_BIN and FileReadArray...didn't work. I'm obviously not understanding how to work with files once they're downloaded. Any help would be appreciated.
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Hi
My code to download FF news don't work anymore since this WE ! Anyone using webrequest to download news has this problem ??
result = 0 bytes downloaded. No modification in the code since last week when it was always running well since months.
both webpages https://cdn-nfs.forexfactory.net/ff_calendar_thisweek.xml and http://www.forexfactory.com/ffcal_week_this.xml always working if paste in Internet Explorer.