MT5/mql5 reported and confirmed bugs. - page 6

New comment
 
amrali #:
Bug in MQL NormalizeDouble() function. Round-off errors in the function output.

Build 3194 and all the previous builds.

A minor bug is encountered in NormalizeDouble() function where small round-off errors (= 1 ULP; unit in the last place) is frequently introduced in the function output.

These small roundoff errors will be much more noticed in the future, after MetaQuotes has fixed the code to print floating-point numbers accurately.

This a test script to reproduce the bug:

I have some time to spare :-D

Example of a "fail" :

Original d1  = 1.0054400000000001

Temp string = "1.00544"   // DoubleToString(d1, digits)

New d1       = 1.0054399999999999 // StringToDouble(DoubleToString(d1, digits))

d2              = 1.0054400000000001 // NormalizeDouble(d1, digits)

The documentation says nothing about the rounding scheme used by NormalizeDouble. Why are you thinking the problem is NormalizeDouble (it could be but we have no idea about it) when StringToDouble give 1.0054399999999999 instead of 1.0054400000000001 ?

The problem could be in StringToDouble or both. There could also be no problem at all except some "incoherence", a function using one possible representation and one the other.

What the standard says about the binary representation to use for a number with 2 possible ones ?

 

OK, Alain let me explain this a bit.

First, you start with a rounded decimal string "1.00544" as if it is human typed. To parse this string to a machine number,  StringToDouble() converts it to the nearest binary approximation as a 64-bit floating point value.

To check if conversion string -> double is correct, we can examine it here https://baseconvert.com/ieee-754-floating-point

"1.00544" <string> -> 1.0054399999999998893684960421524010598659515380859375 <double>

So, StringToDouble() is Ok. A more automated way to the check correctness of StringToDouble() is thru round-tripping  string -> double -> string. as long as these functions  StringToDouble() and string() is ok, the round-trip will succeed in both directions. This is the case now after string() was fixed. StringToDouble() in mql is always correct in all versions.

Now, this is our rounded machine number in 64-bit hexadecimal notation:

"1.00544" <string> ->  3FF0164840E1719F <hexadecimal>

What happens, is that NormalizeDouble() instead of giving back the same number, it introduces sometimes a round-off error in the rightmost bit.

NormalizeDouble(3FF0164840E1719F)  = 3FF0164840E171A0, which is incorrect. As you see one ULP (bit) is added, and this is a round-off error. 

Therefore you get this un-rounded result 1.0054400000000001 from the rounded  input number.

Using the converter in the above link.

1.0054400000000001 <double> -> 3FF0164840E171A0 <hexadecimal>


A more formal methodology is that NormalizeDouble() to be cross-checked against a more robust function, like .Net round() that uses a 128-bit decimal type: 

https://docs.microsoft.com/en-us/dotnet/api/system.decimal.round?view=netframework-4.8

// RoundSharp.cs
// To compile: C:\Windows\Microsoft.NET\Framework\v4.0.30319\csc.exe /target:library RoundSharp.cs

using System;

public class DecimalPrecision
{
   public static double Round(double value, int digits)
   {
      // Rounds a decimal value to the specified precision using the specified rounding strategy.

      return (Double)Math.Round((decimal)value, digits, MidpointRounding.AwayFromZero);
   }
}

cross_check.mq5

#property script_show_inputs
input double num = 1.00544;
input int digits = 5; // Round numbers to specified digits

#import "RoundSharp.dll"

void OnStart()
  {
   double d1, d2;

   // Round a number correctly to specified digits.
   d1 = DecimalPrecision::Round(num, digits);

   // Bug in NormalizeDouble() function. Round-off errors in the function output.
   d2 = NormalizeDouble(num, digits);

   printf("DecimalPrecision::Round(%s,%d) = %s", string(num), digits, string(d1));
   printf("Build %d: NormalizeDouble(%s,%d) = %s", TerminalInfoInteger(TERMINAL_BUILD), string(num), digits, string(d2));
  }
//+------------------------------------------------------------------+

// Sample output:
// DecimalPrecision::Round(1.00544,5) = 1.00544
// Build 3210: NormalizeDouble(1.00544,5) = 1.0054400000000001

I hope I have explained the bug, clearly. It is a minor bug though!

Thanks


Edit:

Try other values in the above script like 1.00543 to check.

 
amrali #:

OK, Alain let me explain this a bit.

First, you start with a rounded decimal string "1.00544" as if it is human typed. To parse this string to a machine number,  StringToDouble() converts it to the nearest binary approximation of the 64-bit floating point value.

To check if conversion string -> double is correct, we can examine it here https://baseconvert.com/ieee-754-floating-point

A more automated way to the check correctness of StringToDouble() is thru round-tripping  string -> double -> string. as long as these functions  StringToDouble() and string() is ok, the round-trip will succeed in both directions. This is the case now after string() was fixed. StringToDouble() in mql is always correct in all versions.

This is our rounded machine number in 64-bit hexadecimal notation:

What happens, is that NormalizeDouble() instead of giving this same number, it introduces a round-off error in the rightmost bit.

NormalizeDouble(3FF0164840E1719F) -> 3FF0164840E171A0 as you see one ULP (bit) is added, and this is a round-off error. 

Therefore you get this un-rounded result 1.0054400000000001


A more formal methodology is that NormalizeDouble to be cross-checked against a more robust function, like .Net round() that uses a 128-bit decimal type: 

https://docs.microsoft.com/en-us/dotnet/api/system.decimal.round?view=netframework-4.8

cross_check.mq5


It's not obvious to me why 3FF0164840E1719F is the most accurate representation of "1.00544", why not 3FF0164840E171A0 ?

Is this defined in the standard ?

 
Alain Verleyen #:

It's not obvious to me why 3FF0164840E1719F is the most accurate representation of "1.00544", why not 3FF0164840E171A0 ?

Is this defined in the standard ?

Yes sure, it is the IEEE 754 Floating Point standard that all CPU manufacturers follow. Something like the ASCII and UTF standards for representation of strings on computers


#property script_show_inputs
input double num = 1.00544;

//+------------------------------------------------------------------+
//|                                                                  |
//+------------------------------------------------------------------+
string DoubleToHexadecimal(double value)
  {
   union _d {double value; long bits;} dbl;
   dbl.value = value;
   return StringFormat("0x%.16I64X", dbl.bits);
  }

void OnStart()
  {
   printf("DoubleToHexadecimal(%s) = %s", string(num), DoubleToHexadecimal(num));
  }
//+------------------------------------------------------------------+

// DoubleToHexadecimal(1.00544) = 0x3FF0164840E1719F

Edit:

Try also 1.0054400000000001 to see its hex.

 

FYI, this bug in NormalizeDouble() has zero impact on rounding the prices of orders as they are sent to trading sever, because Metaquotes servers set some higher tolerance much more > 1 bit (ULP), to consider the price received as not rounded to be rejected.

That is way no major issues have occurred, except that every now and then you see some posts on the forum reporting strange issues with rounding.

 
amrali #:

Yes sure, it is the IEEE 754 Floating Point standard that all CPU manufacturers follow. Something like the ASCII and UTF standards for representation of strings on computers


Edit:

Try also 1.0054400000000001 to see its hex.

I know the hex representation. I am just saying I don't see why one value is more accurate than the other. Apparently Metaquotes doesn't know either ;-)

 
I reported it as bug, so when you see some strange results, do not bother, it is minor and it will not affect your orders in any way. If this occurs in a scientific or mathematical program, then it will be a major flaw.
 
amrali #:
I reported it as bug, so when you see some strange results, do not bother, it is minor and it will not affect your orders in any way. If this occurs in a scientific or mathematical program, then it will be a major flaw.
Why would you want to use NormalizeDouble() in a scientific or mathematical program ?
 
Alain Verleyen #:
Why would you want to use NormalizeDouble() in a scientific or mathematical program ?
I mean rounding itself ;) 
 

Small details in floating-point calculations makes difference.

Let's say, you divide a floating-point number by 100, so you code it as number / 100.0 (floating-point division).

But also, you can write the above expression as number * 0.01 (using multiplication with the reciprocal of 100). This latter method is way faster but it gives a round-off error.

This is for demonstration only (not production code):

#property script_show_inputs
input double num = 1.00544;
input int digits = 5; // Round numbers to specified digits

double round_1(double n, int dig)
  {
   double pwr = MathPow(10, dig);
   return MathRound(n * pwr) / pwr;
  }

double round_2(double n, int dig)
  {
   double point = MathPow(10, -dig);
   return MathRound(n / point) * point;
  }

void OnStart()
  {
   double d1, d2;

   // Round a number correctly to specified digits.
   d1 = round_1(num, digits);

   // Round-off errors in the function output.
   d2 = round_2(num, digits);

   printf("round_1(%s,%d) = %s", string(num), digits, string(d1));
   printf("round_2(%s,%d) = %s", string(num), digits, string(d2));
  }
//+------------------------------------------------------------------+

// Sample output:
// round_1(1.00544,5) = 1.00544
// round_2(1.00544,5) = 1.0054400000000001
1234567891011
New comment
Reason: