How to find out if you can look into the future? - MQL4 and MetaTrader 4

Christo Tsvetanov 2007.08.06 12:16 #101

About the Three Stooges:

http://www.libex.ru/detail/book74148.html

Rashid Umarov 2007.08.06 13:47 #102

That's right, the cover is the same.

Yurixx 2007.08.12 20:03 #103

MetaQuotes:
Parade of pictures without sources and proofs...

2 Rosh

In your EA on page 5 you have checked for Bid and Close mismatches of all t/fs. You interpret the absence of such mismatches as a confirmation that looking into the future is not possible in the tester. I still found this connection strange at the time. From my point of view it is not the behaviour of Close that should be tested, but the behaviour of High and Low of the higher t/f. And after the unfair reproach from MQ mentioned above, I decided to devote some time to it. But it was too late.

Below is the code of the Expert Advisor that generates by itself the current High and Low of the hour or day on the one-minute chart. Then on every tick it compares it with the High and Low of the hour or day, obtained from the H1 or D1 and in case of a discrepancy it sends it to the log and file.

//+------------------------------------------------------------------+
//|                                           Simple Prospection.mq4 |
//+------------------------------------------------------------------+
#property copyright "Copyright c 2007, Yurixx"
#property link      ""
double curHi,curLo,HiH1,LoH1;
 int    mm,hh,dd,curM1,curH1,curD1,kk,nn,handle;
 string str,mHi,mLo,hHi,hLo;

//+------------------------------------------------------------------+
//| expert initialization function                                   |
//+------------------------------------------------------------------+
 int init()
  {
//----
  handle = FileOpen("FU.csv",FILE_CSV|FILE_WRITE," ");
  if(handle<1) { Print("File FU.csv not found, Error:", GetLastError());
                 return(false);   }
  if (Period()>PERIOD_M1)
  {  Print("Период тестирования не соответствует задаче");
     return(-1);
  }
  Print("Период тестирования ",Period()," минут");
  FileWrite(handle,"Date","Time","curHi","HiH1","curLo","LoH1");
  nn=D'2007.07.12 23:58:59';
  FileWrite(handle,TimeToStr(nn,TIME_DATE|TIME_SECONDS),
                   TimeSeconds(nn),TimeMinute(nn),TimeHour(nn),TimeDay(nn));
  nn=D'2007.07.13 00:58:59';
  FileWrite(handle,TimeToStr(nn,TIME_DATE|TIME_SECONDS),
                   TimeSeconds(nn),TimeMinute(nn),TimeHour(nn),TimeDay(nn));
  nn=D'2007.07.13 00:02:00';
  FileWrite(handle,TimeToStr(nn,TIME_DATE|TIME_SECONDS),
                   TimeSeconds(nn),TimeMinute(nn),TimeHour(nn),TimeDay(nn));
  curHi=0.0;
  curLo=1000.0;
  curD1=-1;
  curH1=-1;
  curM1=-1;
  nn=0;//----
   return(0);
  }
//+------------------------------------------------------------------+
//| expert start function                                            |
//+------------------------------------------------------------------+
 int start()
  {   
//----
  mm = TimeMinute(TimeCurrent());
  hh = TimeHour(TimeCurrent());
  dd = TimeDay(TimeCurrent());
  if (mm!=curM1)
  {  if (hh!=curH1)
     {  if (dd!=curD1)
        {  curHi=NormalizeDouble(Bid,Digits);
           curLo=NormalizeDouble(Bid,Digits);
           curD1=dd;
        }
        //curHi=NormalizeDouble(Bid,Digits);
        //curLo=NormalizeDouble(Bid,Digits);
        curH1=hh;
     }
     curM1=mm;
  }
  if (NormalizeDouble(Bid,Digits)>curHi) curHi=NormalizeDouble(Bid,Digits);
  if (NormalizeDouble(Bid,Digits)<curLo) curLo=NormalizeDouble(Bid,Digits);
  //HiH1 = iHigh(NULL,PERIOD_H1,0);
  //LoH1 =  iLow(NULL,PERIOD_H1,0);
  HiH1 = iHigh(NULL,PERIOD_D1,0);
  LoH1 =  iLow(NULL,PERIOD_D1,0);
  HiH1 = NormalizeDouble(HiH1,Digits);
  LoH1 = NormalizeDouble(LoH1,Digits);
  str = TimeToStr(TimeCurrent(),TIME_DATE|TIME_SECONDS);
  mHi = ", curHi=" + DoubleToStr(curHi,Digits);
  mLo = ", curLo=" + DoubleToStr(curLo,Digits);
  hHi = ", HiH1="  + DoubleToStr(HiH1, Digits);
  hLo = ", LoH1="  + DoubleToStr(LoH1, Digits);
  if (HiH1!=curHi||LoH1!=curLo) 
  {  Print(str,mHi,hHi,mLo,hLo);
     FileWrite(handle,TimeToStr(TimeCurrent(),TIME_DATE|TIME_SECONDS),curHi,HiH1,curLo,LoH1);
  }
//----
   return(0);
  }
//+------------------------------------------------------------------+
//| expert deinitialization function                                 |
//+------------------------------------------------------------------+
 int deinit()
{  Print("Работа закончена");
   FileClose(handle); 
//---- done
   return(0);
}

And this is a piece of the log, that was obtained by running this EA on EURUSD, M1 from 2007.07.10 to 2007.07.14. As you can see from the text of the EA, the comparison was with the daily data. However, when comparing to hourly data the situation is not better. I wanted to get a confirmation of the possibility to look into the future or to make sure that there is no such possibility. However, it turned out to be something completely different.

As you can see from the image, the time printed in the journal of the tester, and the time displayed by the Expert Advisor are periodically different from each other. In addition to that, there are some incomprehensible setbacks. The time 2007.07.13 00:58, 2007.07.12 00:58, 2007.07.13 00:02, 2007.07.13 00:04, 2007.07.13 00:06 and 2007.07.13 00:07. And every time the Expert Advisor outputs 2007.07.12 23:58:59.

Perhaps, the related High and Low data discrepancy at those moments has been caused exactly by those timing failures.

Additionally, I advise to pay attention to the test print to file, which is in the init() function. This print shows that the seconds are not working in the tester. Accordingly, TimeToStr() in seconds mode and TimeSeconds() function do not work. Maybe, it was intended that way, but then why do both the tester and the Expert Advisor print data with seconds?

I'm not even raising the question of discrepancy between High and Low data, because it's absolutely unclear where this data is coming from in this obscure time.

One more thing. When testing from 2007.07.09 to 2007.07.14 and not from 2007.07.10 to 2007.07.14 I encounter abnormal things like totally missing data from high and low balances, i.e. variables HiH1 and LoH1 always have zero values.

May be I made a mistake somewhere ?

Testing irregularities - Can Strategy Tester changing value Discrepancies between script and

Rashid Umarov 2007.08.13 06:23 #104

Hello Yurixx.
I ran your Expert Advisor without changing anything in the code. Here is all the data from the file it outputs:

Date Time curHi HiH1 curLo LoH1<br/ translate="no"> 2007.07.12 23:58:00 0 58 23 12
2007.07.13 00:58:00 0 58 0 13
2007.07.13 00:02:00 0 2 0 13

Here is the log:

2007.08.13 09:54:51 2007.07.13 22:59 Simple Prospection EURUSD,M1: Work Finished
2007.08.13 09:54:48 2007.07.10 00:00 Simple Prospection EURUSD,M1: Testing period 1 min
2007.08.13 09:54:48 Simple Prospection started for testing
2007.08.13 09:54:45 Simple Prospection: loaded successfully

Not a single error in the test from 2007.07.10 to 2007. 07.14. True, then I remembered that I have a test special build from last week. The usual 208 build from August 01 (now updated one term by LibeUpdate) has no error as well:

2007.08.13 10:13:04 2007.07.13 22:59 Simple Prospection EURUSD,M1: Work Finished
2007.08.13 10:13:04 2007.07.10 00:00 Simple Prospection EURUSD,M1: Testing period of 1 minute
2007.08.13 10:13:04 Simple Prospection started for testing

The matter is that before the test I have downloaded the missed data from History Center (I haven't started this terminal for a couple of months) and then recalculated all the timeframes - I pressed "Download" in History Center for the second time, in this case it suggests to recalculate all t/f automatically and period converter is not needed (in case people are not aware of this feature).

But before that there were error outputs to the log:

2007.08.13 10:08:19 1999.05.26 02:01 Simple Prospection GBPUSD,M1: Work completed
2007.08.13 10:08:18 1999.01.04 09:31 Simple Prospection GBPUSD,M1: 1999.01. 04 09:31:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:30 Simple Prospection GBPUSD, M1: 1999.01. 04 09:30:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:29 Simple Prospection GBPUSD, M1: 1999.01. 04 09:29:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:28 Simple Prospection GBPUSD, M1: 1999.01. 04 09:28:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:27 Simple Prospection GBPUSD, M1: 1999.01. 04 09:27:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:26 Simple Prospection GBPUSD, M1: 1999.01. 04 09:26:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:25 Simple Prospection GBPUSD, M1: 1999.01. 04 09:25:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:24 Simple Prospection GBPUSD, M1: 1999.01. 04 09:24:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:23 Simple Prospection GBPUSD, M1: 1999.01. 04 09:23:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:22 Simple Prospection GBPUSD, M1: 1999.01. 04 09:22:00, curHi=1.6718, HiH1=1.6718, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:21 Simple Prospection GBPUSD, M1: 1999.01. 04 09:21:00, curHi=1.6702, HiH1=1.6702, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:20 Simple Prospection GBPUSD, M1: 1999.01. 04 09:20:00, curHi=1.6702, HiH1=1.6702, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:19 Simple Prospection GBPUSD, M1: 1999.01. 04 09:19:00, curHi=1.6702, HiH1=1.6702, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:18 Simple Prospection GBPUSD, M1: 1999.01. 04 09:18:00, curHi=1.6702, HiH1=1.6702, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:17 Simple Prospection GBPUSD, M1: 1999.01. 04 09:17:00, curHi=1.6701, HiH1=1.6701, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:16 Simple Prospection GBPUSD, M1: 1999.01. 04 09:16:00, curHi=1.6701, HiH1=1.6701, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:15 Simple Prospection GBPUSD, M1: 1999.01. 04 09:15:00, curHi=1.6687, HiH1=1.6687, curLo=1.6682, LoH1=1. 6684
2007.08.13 10:08:18 1999.01.04 09:14 Simple Prospection GBPUSD, M1: 1999.01. 04 09:14:00, curHi=1.6687, HiH1=1.6687, curLo=1.6682, LoH1=1. 6686
2007.08.13 10:08:18 1999.01.04 09:13 Simple Prospection GBPUSD, M1: 1999.01. 04 09:13:00, curHi=1.6682, HiH1=1.6697, curLo=1.6682, LoH1=1. 6597
2007.08.13 10:08:15 1999.01.04 09:13 Simple Prospection GBPUSD,M1: Testing period 1 minute
2007.08.13 10:08:15 Simple Prospection started for testing

I downloaded that data for EURUSD, but started testing for GBPUSD, which wasn't pumped and recalculated. That was the reason for the inconsistencies.

Try to do the same - load data and synchronize automatically or use the period converter script.

We'll look into the seconds thing, thanks.

[ARCHIVE!] Any rookie question, Optimisation results differ from Features of the mql5

Yurixx 2007.08.13 08:06 #105

Hi Rosh !

Thanks for your reply. As far as I understand you, you think that the mismatch of High and Low data from different funtions is due to the quality of the quote flow. Thus, the candlesticks are shaped differently on different functions and therefore may be different by 1-2-3 pips. It is quite possible.

I've been testing using the data of the MQ demo server. And I'm not writing them in real time, but I download all the data once a week, on weekends. Actually I thought the candlesticks are drawn on all TFs, at least on the server, by the server software, synchronously for all TFs, based on a single quote flow and therefore such differences are impossible. If this is not the case, it's a pity, we will have to take it into account somehow ... Combining data, synchronisation etc to make everything look nice is the wrong way to go in my opinion. Both your server and brokers' servers deliver the data as they deliver it. And you have to trade on this very data and not on what it turns into after some time. For the testing process it is especially important. Everyone knows the problem of grails - a space in the tester and a loss in the real trading robot. Where does it come from ? MQ insists that looking into the future is impossible and that tick modelling is quite appropriate to the process. One must assume that this is the case. Hence, indeed, the problem is the data ? MQ cannot solve this problem, the data does not depend on it. Then you need to make the tester work correctly on any data, not just the data that is neatly combed.

But my post was not about discrepancies in data from different t/f's, but about confusion over time. And the fact that you posted doesn't remove the issue. On the contrary, in connection with your post I want to ask the following.

In my log the time data of both tester and EA contain seconds. In yours, data in the tester do not contain seconds at all, and data in your EA contain only zero seconds. This leads me to question: in what mode did you perform the test. I want to be clear - this EA is only intended for testing in "all ticks" mode. And the error I have encountered can only be reproduced from this mode. Therefore, if you have been testing in another mode, please repeat the test, it will only take a second.

In principle, it makes absolutely no difference on which pair and in which date range to test. And on TFs other than M1, the Expert Advisor will not work. However, in order to be able to compare our results, I ask you to do a test on EURUSD, in the range between 2007.07.10 and 2007.07.14, and, in a separate test, in the range between 2007.07.09 and 2007.07.14.

I thank you in advance.

Off-topic posts Can't get accurate results Tick data from different

Rashid Umarov 2007.08.13 08:25 #106

Yurixx:

Hi Rosh !

In my log, both the tester and EA time data contain seconds. In yours, the tester data contains no seconds at all and the EA data contains only zero seconds. Which begs the question: what mode did you use for testing? I want to be clear - this EA is only intended for testing in "all ticks" mode. And the error I have encountered can only be reproduced from this mode. Therefore, if you have been testing in another mode, please repeat the test, it will only take a second.

In principle, it does not matter at all on which pair and in which date range to test. And on TFs other than M1, the Expert Advisor will not work. However, in order to be able to compare our results, I ask you to do a test on EURUSD, in the range between 2007.07.10 and 2007.07.14, and, in a separate test, in the range between 2007.07.09 and 2007.07.14.

Thank you in advance.

Indeed, looked it up now and saw that I didn't have seconds. I think it has to do with the fact, that I didn't have data for GBPUSD automatically loaded for 1 hour and 1 day on the first test (I didn't have charts opened in the terminal at all) but I tried to check it now for the second time and there was no error - I loaded the necessary data on the first test run.

That is, the first time there was no data for H1 and D1 periods for GBPUSD and therefore there were errors in modelling.

Error Backtesting : no Input tab missing when Machine learning in trading:

Rashid Umarov 2007.08.13 08:27 #107

I did a test "on EURUSD in the range 2007.07.10 to 2007.07.14, and, in a separate test, in the range 2007.07.09 to 2007.07.14" as you asked, no difference.

Rashid Umarov 2007.08.13 11:39 #108

Rosh:

We'll look into the seconds thing, thank you.

The error with seconds has been fixed (in the compiler). Corrected build will be available soon.

Виктор 2007.08.13 16:56 #109

Rosh:
The bug with seconds has been fixed (in the compiler). Corrected build will be available soon.

Can we hope that in the corrected build the tester grail77 will not work anymore ?

Rashid Umarov 2007.08.13 17:34 #110

granit77:

Rosh:
The bug with seconds has been fixed (in the compiler). Corrected build will be available soon.

Can we hope that in corrected build tester grail77 will not work anymore ?

I will check it tomorrow.

28 !!! currency pairs, 1 expert. Another grail, but this one I think no one has ever shown. + DEMO ACCOUNT - page 11