Talmadge

After the fifth iteration, fC ends up with and additional 0.000037. This can be seen only in the debugger. After the seventh iteration, an extra .0001 shows up in the console. Is this a math error in the language I ran two versions of this code on two different computers, but they have the same processor.

using System;

using System.Collections.Generic;

using System.Text;

namespace ConsoleApplication1

{

class Program

{

static void Main(string[] args)

{

float fC = 1000.00F;

float fN = 23.23F;

float fInterferer = 45.20F;

for (int i = 0; i < 15; i++)

{

fC = fC - (fN + fInterferer);

System.Console.Out.WriteLine( fC.ToString() );

}

}

}

}



Re: Visual C# Language C# math error?

Nimrand

It looks like cumulative rounding errors to me. But, you're saying this only happens in the debug version of the program That is truely strange.





Re: Visual C# Language C# math error?

eradicator

Please read this: http://docs.sun.com/source/806-3568/ncg_goldberg.html

"What every computer scientist should know about floating-point arithmetic"

Basically the bottom line is trying to fit all floating point numbers into fixed 32-bit or 64-bit values in the range they support, is not possible. There are more numbers than 32/64 bits can truly represent. So you get an approximation at best. This is not a .NET / C# / or even a Java (since the doc is hosted at sun.com) specific problem -- all computer languages share it.

Since you're dealing with 2 places of decimals, I'd suggest you scale your numbers by 100 and just deal with simple integer arithmetic.






Re: Visual C# Language C# math error?

IsshouFuuraibou

I'd follow eradicator's suggestion, or if you can't go to integer math, you can switch to decimal which is more accurate than float but has a different range (does not appear to be a problem).

the document referenced has nothing to do with Java, sun is more than just java (though that's what they are primarilly known for anyways).

Specifically it is an IEEE 754 (floating point) specification problem, encoding a decimal (base 10) number system in a binary (base 2) system. A fixed point system may not have the same issue, but it has it's own issues (very limited range, and just inability to represent certain values and percisions)





Re: Visual C# Language C# math error?

Talmadge

Thanks IsshouFuuraibou and Eradicator. I read the document on the Sun website, and it helped me immensely. I also switch to using decimal in place of float. This solved my "rounding" error. I took this approach, because even though my example was to only 2 decimal places, the real data will have varying level. Thanks again.





Re: Visual C# Language C# math error?

IsshouFuuraibou

Just remember decimal doesn't solve all the problems of using decimal values in a binary world, but because it encodes the information about the value differently it will encounter different challenges, most likely none that you would face in your application.