How does variable typing affect simple floating point arithmatic operations in Powershell, i.e. what is 1.1 – 1.04 ?
PowerShell is a great scripting language. You can get complex results by just a one liner, which would have taken a couple of lines in the other scripting languages to generate the same output, but don’t be surprised if you find a simple subtraction 1.1 - 1.04 won’t give you the correct answer of 0.06.
Try this for yourself on your PowerShell window and you can see the result:
PowerShell internally does a lots of work for you, so when you are specifying (1.1 – 1.04), it takes the data types of the operands as System. Double. We can check the data type by the following expression:
1.1.GetType().FullName
Float and Double are both imprecise types. Double is an implementation of IEEE floating point, and IEEE floating point is incapable of representing all numbers exactly. (IEEE Standard 754 Floating Point Numbers)
<ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4610935&isnumber=4610934>
So, if you need exact results, you need to use the decimal type. While this has greater precision than both float and double, it also has a smaller range.
Here is the PowerShell expression to get the correct result:
[decimal]1.1 - [decimal]1.04
The same behavior can be seen in any computer language which implements IEEE Standard 754. You can try this C# code to check it :
double var1 = 1.1;
double var2 = 1.04;
double result = var1 - var2;
Console.WriteLine(result);
I found this blog article very interesting and relevant to the topic:
blogs.msdn.com/powershell/archive/2007/01/09/two-plus-two-equals-monday.aspx
Here are few more web articles on MSDN site:
msdn.microsoft.com/en-us/library/system.decimal.aspx