Freigeben über

Small Basic - Mystery of Round Operation

  • Artikel

Computer calculation always has some errors.  I wrote following program to investigate how Math.Round operation works.  Guess what type of errors occur in following case.

TextWindow.WriteLine(Math.Round("2.50000000000000017763568394"))    ' 2

TextWindow.WriteLine(Math.Round( 2.5000000000000001776356839401 ))  ' 2

TextWindow.WriteLine(Math.Round("2.500000000000000200000"))         ' 2

TextWindow.WriteLine(Math.Round("2.5000000000000002000"))           ' 2

TextWindow.WriteLine(Math.Round("2.500000000000000200"))            ' 2

TextWindow.WriteLine(Math.Round("2.5000000000000002"))              ' 2

TextWindow.WriteLine(Math.Round( 2.5000000000000051 ))              ' 2

TextWindow.WriteLine(Math.Round( 2.5000000000000052 ))              ' 3

TextWindow.WriteLine(Math.Round("2.5000000000000001776356839401"))  ' 3

TextWindow.WriteLine(Math.Round("2.50000000000000020"))             ' 3

Actuary, I can't understand yet what happens in the program above...

Comments

  • Anonymous
    August 21, 2017
    Math.Round() Option Rounds A Decimal To The Nearest Even Number. For ExampleIf You Try TextWindow.WriteLine(Math.Round("11.5")) 'You Will Get The Output As 12TextWindow.WriteLine(Math.Round("12.5")) 'You Will Get The Output As 12This Is Because The Math.Round() Rounds The Decimal To The Nearest Even Number
    • Anonymous
      August 22, 2017
      RoshanPriyaKumar, thank you for your comment. Yes, that the most important point to understand Math.Round() operation.
  • Anonymous
    August 23, 2017
    it has to do also with some issues in .net frmwrk: https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/floating-point-numbershowever the numbers are not stored with same decimal precision, i.e. the larger the number is, the lower decimals are kept:example: GVZ923as seen, total number of significant digits is 16 in sb math