Heavy "Feature" with doubles produces unpredictable errors in EAs

To add comments, please log in or register
Doerk Hilger
1220
Doerk Hilger  

A simple math function like this brings a totally wrong result, and this served me a bug in an EA which I was searching for months!!!

MT5 Build 2085, but I am very sure this is the same in earlier versions, cause this is also the same with C# ... unbelievable but true. 


void SimpleMath()
   {
      double a=0.1;
      double b=0.5;
      
      for (int i=0;i<5;i++)
         {
         b-=a;
         Print(b);   
         }
   }


The final result should be 0. But it is really 2.775557561562891e-17  .... !!!

It does not seem with values>1 for a and b, but with all values <1.

Result with a=0.01 and b=0.05 in MT5:

In C# the same function has a final result of, so it´s not a bug of MT5.

 


This brings up the question: How to deal with microlots and minilots when you are trying to close several 0.1 Lot positions until your entire positioning is zero? You will not be able to calculate it when you try to close more than 4 positions and instead you will run into bad malfunction - fact!

Doerk Hilger
1220
Doerk Hilger  

Dealing with float instead of double seems to work for this ... is this the general solution? Avoid doubles at all?


void SimpleMath()
   {
      float a=0.01;
      float b=0.05;
      
      for (int i=0;i<5;i++)
         {
         b-=a;
         Print(b);   
         }
   }


Doerk Hilger
1220
Doerk Hilger  

Update: Could this be a Windows Kernel problem and that this is connected to the date and time?

It cannot be coincidence that my Frontend-Support received a message from a user just 15 minutes after I posted this, that - only since today - he has problems when he is trying to deal with lot sizes >= 0.4 and gets a margin call and all positions become closed. Of course, because 2.7 is 27x the size of 0.1. The version of the EA he is using was last updated in autumn 2018. 

Icham Aidibe
11580
Icham Aidibe  

https://dzone.com/articles/never-use-float-and-double-for-monetary-calculatio

LOL !

Did You Know?

Using Float, Double Instead of BigDecimal Could Be Fatal for Military

On February 25, 1991, a  loss of significance in a MIM-104 Patriot missile battery prevented it from intercepting an incoming Scud missile in Dhahran, Saudi Arabia, contributing to the death of 28 soldiers from the U.S. Army’s 14th Quartermaster Detachment.

Banker’s Rounding Mode

Since the introduction of IEEE 754, the default method (rounded to the nearest decimal, ties to even and is sometimes called Banker’s Rounding or   RoundingMode.HALF_EVEN) is more commonly used in the US. This method rounds the ideal (infinitely precise) result of an arithmetic operation to the nearest representable value and gives that representation as the result. In the case of a tie, the value that would make the significand end in an even digit is chosen.

Why You Should Never Use Float and Double for Monetary Calculations - DZone Java
Why You Should Never Use Float and Double for Monetary Calculations - DZone Java
  • 2018.08.21
  • Munish Chandel . See the original article here. Opinions expressed by DZone contributors are their own.
  • dzone.com
Float and double are bad for financial (even for military use) world, never use them for monetary calculations. If precision is one of your requirements, use instead. Let's explore this problem with the help of an example: All floating point values that can represent a currency amount (in dollars and cents) cannot be stored exactly as it is in...
Icham Aidibe
11580
Icham Aidibe  
Doerk Hilger:

A simple math function like this brings a totally wrong result, and this served me a bug in an EA which I was searching for months!!!

MT5 Build 2085, but I am very sure this is the same in earlier versions, cause this is also the same with C# ... unbelievable but true. 



The final result should be 0. But it is really 2.775557561562891e-17  .... !!!

It does not seem with values>1 for a and b, but with all values <1.

Result with a=0.01 and b=0.05 in MT5:

In C# the same function has a final result of, so it´s not a bug of MT5.

 


This brings up the question: How to deal with microlots and minilots when you are trying to close several 0.1 Lot positions until your entire positioning is zero? You will not be able to calculate it when you try to close more than 4 positions and instead you will run into bad malfunction - fact!

   printf("**** %s",DoubleToString(2.775557561562891e-17,8));
2019.07.16 10:06:09.950	xxx(EURUSD,M1)	**** 0.00000000

Doerk Hilger
1220
Doerk Hilger  
Icham Aidibe:

https://dzone.com/articles/never-use-float-and-double-for-monetary-calculatio

LOL !

Did You Know?

Using Float, Double Instead of BigDecimal Could Be Fatal for Military

On February 25, 1991, a  loss of significance in a MIM-104 Patriot missile battery prevented it from intercepting an incoming Scud missile in Dhahran, Saudi Arabia, contributing to the death of 28 soldiers from the U.S. Army’s 14th Quartermaster Detachment.

Banker’s Rounding Mode

Since the introduction of IEEE 754, the default method (rounded to the nearest decimal, ties to even and is sometimes called Banker’s Rounding or   RoundingMode.HALF_EVEN) is more commonly used in the US. This method rounds the ideal (infinitely precise) result of an arithmetic operation to the nearest representable value and gives that representation as the result. In the case of a tie, the value that would make the significand end in an even digit is chosen.

Clear so far, but the result should still be something round about zero, and not +2.77 or -3.46. Furthermore, lot size calculation is not financial stuff. Theoretically, a contract size could also be 0.001, decimal type - even if it´s not present in MQL - wouldn´t help here too.

Doerk Hilger
1220
Doerk Hilger  
Icham Aidibe:

Yes, of course. But that does not help when you calculate the way I´ve shown and when you compare the result to zero to get of a such a loop.  

Example - a normal == operation does not work, instead a comparison function must be used.

void SimpleMath()
   {
      double a=0.01;
      double b=0.05;
      
      for (int i=0;i<5;i++)
         {
         b-=a;
         Print("Value: ", b, "Comparison ==: ", b==0 ? 0: 1, " Comparison func: ", CompareValues(b,0,0.01)); 

         //--- Never true!!! 
         if (!MathIsValidNumber(b))
            Print("Invalid");
         }
   }
   
   
int CompareValues(double v1, double v2, double minimalvalue=1.0)
   {
      if (minimalvalue>0)
         {
         v1=MathRound(v1/minimalvalue);
         v2=MathRound(v2/minimalvalue);
         }
      if (v1==v2) return 0;
      return v1>v2 ? 1: -1;   
   }
As you see, I also added MathIsValidNumber(), which is always true. In other words, the calculation is definitely wrong. 
Thorsten Rohweder
1647
Thorsten Rohweder  
Doerk Hilger:

A simple math function like this brings a totally wrong result, and this served me a bug in an EA which I was searching for months!!!

MT5 Build 2085, but I am very sure this is the same in earlier versions, cause this is also the same with C# ... unbelievable but true. 



The final result should be 0. But it is really 2.775557561562891e-17  .... !!!

It does not seem with values>1 for a and b, but with all values <1.

Result with a=0.01 and b=0.05 in MT5:

In C# the same function has a final result of, so it´s not a bug of MT5.

 


This brings up the question: How to deal with microlots and minilots when you are trying to close several 0.1 Lot positions until your entire positioning is zero? You will not be able to calculate it when you try to close more than 4 positions and instead you will run into bad malfunction - fact!

Not a bug, use Normalize with needed decimals.

Doerk Hilger
1220
Doerk Hilger  
Thorsten Rohweder:

Not a bug, use Normalize with needed decimals.

It cannot be the general reason that you have to normalize or round any double variable after each simple math operation, cause this would make it impossible to use the precision of doubles for other exact calculations. Of course I know about normalizing and/or round doubles, but I never saw such a behavior before. 

William Roeder
19434
William Roeder  
NormalizeDouble, It's use is usually wrong.
  1. Floating point has infinite number of decimals, it's your not understanding floating point and that some numbers can't be represented exactly. (like 1/10.)
              Double-precision floating-point format - Wikipedia, the free encyclopedia

    See also The == operand. - MQL4 programming forum

  2. Print out your values to the precision you want with DoubleToString - Conversion Functions - MQL4 Reference.

  3. SL/TP (stops) need to be normalized to tick size (not Point.) (On 5Digit Broker Stops are only allowed to be placed on full pip values. How to find out in mql? - MQL4 programming forum) and abide by the limits Requirements and Limitations in Making Trades - Appendixes - MQL4 Tutorial and that requires understanding floating point equality Can price != price ? - MQL4 programming forum
  4. Open price for pending orders need to be adjusted. On Currencies, Point == TickSize, so you will get the same answer, but it won't work on Metals. So do it right: Trailing Bar Entry EA - MQL4 programming forum or Bid/Ask: (No Need) to use NormalizeDouble in OrderSend - MQL4 programming forum
  5. Lot size must also be adjusted to a multiple of LotStep and check against min and max. If that is not a power of 1/10 then NormalizeDouble is wrong. Do it right.
  6. MathRound() and NormalizeDouble() are rounding in a different way. Make it explicit.
              MT4:NormalizeDouble - General - MQL5 programming forum
              How to Normalize - Expert Advisors and Automated Trading - MQL5 programming forum
Doerk Hilger
1220
Doerk Hilger  
I will create class librarys for this stuff / datatypes, based on decimal calculation and without NormalizeDouble() as far as possible to avoid all this stuff. Im really tired of it. Once it is ready, i will publish it here. 
12
To add comments, please log in or register