Is there a "How MT4 really works" manual?
Is there a "How MT4 really works" manual? No.
It seems that "repainting" is a constant issue with MT4. Agreed, depending on how the Indicator was written.
Surely somebody has worked this out and documented a "how to write stable code for MT4" process? Talk to Rosh, JJC and SDC. They're indicator guys.
Is there some document that explains how MT4 really operates and how to code reliable indicators and EAs? No. At least not enough to satisfy everyone's opinion.
Here's links of interesting topics I've seen on this. Link1, Link2, Link3.
In Conclusion:
You didn't say if you're getting these in Back-tester or Live. Example, you start out talking about "back-test a system in Tradestation" followed by "real time" and "re-plots older bars seemingly at random". Are you referring to 0-Bar info in Back-tester or 0-Bar Info on Live charts? Or both? Is there any circumstances where it happens more often? Like hitting the Home-Key, or Refresh?
For something like this, you'll definitely need to provide Codes and a Picture.
Rosh works for Meta-Quotes. Contact him if you think its a Bug. He'll ask you for Code.
My Personal Opinions:
I don't like Indicators. Tho I have nothing against them. Their Job in my opinion is for Visual Aid. Or put another way make the Chat look pretty. Never written one and hope I never will have to unless I wanna make my chat Pretty. To a manual trader this is a necessity. For an EA trader spending more of his time in the back-tester they're almost Pointless in Mt4 IMO. If I'm in doubt about if a EA is doing what it's doing, I'll use objects. A trend based Indicator would give 90% the same info as another Trend based Indicator. Same for Counter-Trend etc. All one have to do is play around with the default Periods and they start to look alike. I don't see why people keep making new Indicators which are doing the same old things.
I'm sure the forex Police is going to hunt me for the above statements but thats just my Opinions.
Thanks, I will read through your linked articles.
Being a newb at MQL, I haven't yet written an EA. I started out with indicators because it seemed a simpler starting point. I've seen indicators repaint live -- e.g. a candle closes red, but later turns green, etc. What you see looking at historic bars is not what you would have seen in realtime, when you were using that indicator to make trading decisions.
I often find indicators very useful -- for debugging my systems / EAs.
Do I understand from your comments that "repainting"-like behavior is not a problem in EAs? MT4 doesn't re-evaluate bars multiple times in EAs?
I agree with your assessment of most indicators. Most of them are highly correlated to the rest of them. There are only so many ways to slice and dice Open, High, Low, and Close.Do I understand from your comments that "repainting"-like behavior is not a problem in EAs? MT4 doesn't re-evaluate bars multiple times in EAs?
Allow me to speak from the point of view of a Back-Tester because thats what I'm more familiar with. I Never-Ever use Bar-0 information (Except for Open lol...Because that does not change). When I was a Newbie looking at indicators on charts and pretending to place trades, I would see an indicator Cross.... Only to find on the next tick the Cross disappeared. If I would have placed that trade and the Cross Never materialize after the Bar-Closed. Someone else looking at that Order would wonder why I didn't stick to the system of only placing the trade when the Indicator Crossed.
This is how Indicators work in Real-Time. One would have to read about the Limitations of the Back-Tester in Detail (and also some understanding on the example above) in order to understand how to program a Back-Tester Friendly EA. Otherwise you'll be like countless people who run in here complaining about how mt4 is Soooooooooo buggy.
So Yes, what you see will not be what you get depending on how the EA is coded. Another thing about Bar-0 information is that the Back-Tester assumes the Bullish Bar went down-first.... and then went up.... and closed above Open-Price. And Vice-Versa for Bearish Bars. If in actually the price Bounced Up first and then down and then up again before it closed. Your scalping 9-Pips Stop-Loss on your Sell-Trade would have gotten hit before the 9-Pips profit. But the Back-Tester would report this as Profit getting hit first. <---- Just another example of Bar-0 issues.
If you need to test price sensitive strategies like scalping then you'll be better off using Tick-Data. There are lots of other examples. As far as your actual problem with Indicators, I think it has more to do with the Number of Bars on the Chart Changing (Outside of the normal New-Bar process). I'm not sure why it happens, the last link gives a lengthy discussion. One reason it happens is by Scrolling the History. Another suspect would be because broker updates data. If you're using IndicatorCounted() it'll try to correct itself and re-draw everything. At least thats what I've got from reading those topics.
Oh, naturally indicators on bar 0 can change, because bar 0 itself is changing.
I'm talking about bar 1 or older closed bars. They should NOT change, but indicators calculated on them sometimes do. The example I mentioned above is a trend indicator that paints candles red/green, and I've seen that indicator change multiple times on bar 1. I understand many of the MTF (multi time frame) indicators repaint farther back than that -- e.g if you're calculating an indicator on M5 - M15 - H1, the H1 indicator might repaint as many as 12 bars back because the "H1 virtual bars" contain 12 M5 bars. I believe the idea is that the H1 indicator would have the same value for the full 1 hour period = 12 M5 bars, and that "H1 bar 0" keeps changing until the hour is over. So if you had a trend indicator on the H1 indicator it might start out red, then repaint up to 12 bars green if the market turned up. I think.
Where would one read about Limitations of the Back-Tester in Detail?
MT4 may not be soooo buggy. But its design in this area seems less than ideal. In Tradestation you *can't* make the kinds of mistakes that are so common in MT4 indicators. TS just doesn't work that way. At least with the basic techniques, TS won't let you write systems/EAs that work differently in backtests than in realtime. And that's a very good thing.
TS just doesn't work that way. At least with the basic techniques, TS won't let you write systems/EAs that work differently in backtests than in realtime. And that's a very good thing.
I dis-agree strongly. Hate to be Meta-Traders greatest fan. However, we're talking more about what you're used to and how you think it should be instead of how it really is. In the newbie example I give above, I don't think even Trade-Station can avoid that. Because they're simply logical Errors. And these are what I'm talking about.
Meta-Trader won't let you write systems/EA that works differently in back-tests than real-time either. If someone's EA does not work in real-time as it did within the back-tester then they failed to consider something. It could be as simple as "slippage, fills" like you noted before, or re-prints because they're using current price as I noted.
Or it could be because they failed to acknowledge that IndicatorCounted() and #Bars on chart have a relationship in MT4. This may not be the case in Trade-Station. As both of these may have their advantages and dis-advantages. Someone going to the Trade-Station side might form the opinion that it's bad design because it doe's not re-calculate all History on "Dirty" bars.
When you have solid proof that Mt4 performs differently than Reality, I'd like to be one of the first people know.
[...] Talk to Rosh, JJC and SDC. They're indicator guys.
Er, no. I've had to write my fair share of indicators, but I'm not an expert. I've written tens, possibly hundreds, of times more lines of EA code than indicator code.
It's possible for indicators in MT4 to re-paint historically, but they don't have to. The same is true of every other trading platform I've seen (which does not include Tradestation). It's arguable that it's easier to do this by accident in MT4. The related thing which MT4 also makes dangerously easy is accidentally building an indicator which can see into the future on historic data.
A few specific issues:
* Ubzen's link3: if extra historic data is added to a chart for some reason (e.g. user scrolls back beyond the current start date), then MT4 tells the indicator to redraw its historic values, and does not itself bother to shift the existing indicator buffers/data. If the indicator does nothing, you then end up with the wrong values being shown for each historic bar.
* Multi-timeframe indicators are problematic for much the same reasons as when trying to compare multiple timeframes manually. (Every trader I know who's manually compared different timeframes has at least once accidentally given themselves the ability to see into the future.) In MT4 there is no built-in way of saying something like "give me the historic H4 moving average up to 135 minutes through the last H4 bar, i.e. the value as-at a historic time halfway through the bar". If, retrospectively, you do something like getting the H1 moving average from an M15 chart, then you see bar-end H1 values, and the data you plot for historic M15 bars ending at xx:14, xx:29, and xx:44 will be based on the H1 value in the future at xx:59. There's no such problem when working on a forward basis but, correspondingly, if the indicator has to re-paint for some reason, then the correct intra-bar values which were calculated live will be replaced by incorrect bar-end values calculated on a retrospective basis.
* The various discussions of IndicatorCounted() are more about performance than about functionality. If you're writing something normal like a moving average or RSI indicator, then it doesn't matter whether your indicator calculates/plots 1 bar or the last 1000 bars on each tick. The historic values will only "repaint", i.e. change, if the historic data somehow changes - or if there's a bug in your code. The discussions of IndicatorCounted() are mostly to do with trivial performance gains from ensuring that your indicator does not re-plot more bars than are actually necessary; they're not really dealing with any "repainting" issues.
EDIT... all this with the caveat that things like exponential moving averages are universally - not just in MT4 - dependent on the length of your data series. If you calculate an EMA once, and then add extra historic data at the beginning of the series, the EMA values for all subsequent bars change (albeit usually by trivial amounts). That's simply how things like EMAs work.
Phy is a walking encyclopaedia of MQL4 if you can reach him.
Er, no. I've had to write my fair share of indicators, but I'm not an expert. Fair enough, I'm not gonna put you out there like that next time ;).
Ok, now I'm confused about Multi-Timeframe Indicators. Last time I read about Multi-Timeframe the conventional way of using it went something like this: Lets use Macd above 0 as Up-trend and Macd below 0 as Down-Trend all other values for Macd as default. I have 3 Time-Frames which must say Up-or-Down before I place the trade. Example H1>0.00, M15>0.00 and M5>0.00 then Buy.
The current time is 12:24 pm (Mid-Day). Is someone actually looking at the H1 values for Macd beyond 12:00? Or the M15 value beyond 12:15? Or the M5 value beyond 12:20? To place that trade. If they are, then aren't they using Current Bar (Bar[0]) information for H1, M15, or M5.
If they're not, how can this information change?
Say at 12:00 pm (Mid-Day), my Indicator drew a Green Vertical line indicating Buy. If the indicator redrew everything for whatever reason at later time.
How is it possible that the Green Vertical Line will become Red indicating Sell?
Please forgave any mis-information I have about Multi-Timeframe or Indicators in general.
Ok, now I'm confused about Multi-Timeframe Indicators. Last time I read about Multi-Timeframe the conventional way of using it went something like this: Lets use Macd above 0 as Up-trend and Macd below 0 as Down-Trend all other values for Macd as default. I have 3 Time-Frames which must say Up-or-Down before I place the trade. Example H1>0.00, M15>0.00 and M5>0.00 then Buy.
There's no problem with that...
If they're not, how can this information change?
It's to do with calculating the historic values for display when an indicator loads (or is re-initialised). The same thing could potentially apply to an EA, but it's very unusual for an EA to need to do this kind of retrospective calculation.
I'll try to give you a clearer example, including some of the relevant MQL4 code. Let's say that you have an indicator on an M15 chart which wants to draw a buy signal if RSI is below 30, both on M15 and also on H1. It's currently 2:03pm, and the indicator wants to fill in the values for some historic bars as well as starting to draw itself on a forward, on-going basis.
Let's take the historic bar at 13:30-13:44, which will be bar #2 on the M15 chart (i.e. Time[2] etc). The indicator on this M15 chart can get the historic M15 RSI for this bar simply by doing iRSI(..., 2). No problem there.
The issue is around getting the H1 RSI part-way through a historic hour at the end of an M15 bar. The M15 bar starting at 13:30 is part of the H1 bar from 13:00-13:59, which will be bar #1 on the H1 timeframe. But if the indicator does iRSI(Symbol(), PERIOD_H1, ..., 1) then it gets the RSI value at the end of that historic H1 bar, i.e. at 13:59, not the interim value at 13:44. Therefore, against a historic M15 bar running from 13:30 to 13:44, the indicator is going to plot an H1 RSI value from 13:59. In effect, the indicator can see into the future. That's why lots of people like MTF indicators: they often appear spectacularly successful when applied to historic data...
There is no easy way round this in MT4. In order to calculate the true historic RSI on the higher timeframe, you have to (a) manually build your own H1 bar data with the correct interim value for the last of the bars, and then (b) do your own RSI calculation on this special bar series.
It's very uncommon - but not impossible - for this to apply to EAs simply because they typically do a comparison between timeframes based on current data only, and don't try to do such retrospective calculations at historic bar-ends.
There's a simple rule of thumb. If you have an indicator or EA which uses iWhatever(..., PERIOD_YY, ...., N) then you need to think carefully about what data you're actually looking at if YY is higher than the chart timeframe which the EA/indicator is running on, and if N > 0.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I've developed trading systems on other platforms for over 10 years. I had 12 years of experience working with Tradestation and I've used several other platforms.
If I backtest a system in Tradestation, I can be 99% sure that (other than slippage, fills, etc) it would have traded in realtime exactly the way it backtested. In other words if the backtest says the system performed like X, I can trust that that's the way it would have traded in realtime. I can look at a backtest report or a chart of indicators and be confident I would have seen the same thing in realtime.
I've been digging into MT4 for a few weeks. It seems that "repainting" is a constant issue with MT4, and that means you can't trust what you see on the historic chart. Having coded up a few indicators, I begin to see why. I avoided calling other indicators with iCustom because I didn't know for sure what they were doing, so I implemented it myself. Then I discovered that wasn't enough, because MT4 didn't work the way I thought. I thought it would scan through all the bars in history, and then deliver realtime ticks. I found it did this, but it also re-plots older bars seemingly at random -- sometimes just the last few bars, sometimes the whole chart. And unless you're very careful about initializing everything appropriately, the replot of the whole chart might end up re-using your values from realtime, resulting in a totally bogus result.
I've hacked together a few workarounds to try to deal with this behavior. For example I used the "history bars, then realtime ticks" assumption, but as soon as I got a realtime tick, I set a "realtime" flag and quit processing older bars. That sort of worked, except when it recalculated the whole chart and re-used my realtime values. So if it tries to replot more than, say, 10 bars, I'll say "ok, replot the whole thing from scratch." I think I've written my code so a "replot from scratch" operation will be stable and match realtime results, but I'm not 100% sure.
Surely somebody has worked this out and documented a "how to write stable code for MT4" process? Is there some document that explains how MT4 really operates and how to code reliable indicators and EAs? The MQL4 docs I've found so far, including at book.mql4.com, don't cover this, at least not in the parts I've gotten to so far.
Thanks for any pointers!
Gary