logo

New Response

« Return to the blog entry

You are replying to:

  1. Yep, floating point is a pain, but nothing new. We've had 'irrational numbers' for centuries! 1/3 as a decimal = ? 0.33333... or root 2 = 1.414213..., natural log anybody? Binary just does a similar approximation in base 2.

    The main issue that people seem to fail to deal with is the difference between the internal binary representation and the 'displayed value' that often has an implied internal rounding process. Where this gets nasty is when you total up long lists of numbers and get a total which is slightly different to whats displayed. This is what 'Decimal/Money' types are for in other databases. They are 'scaled integers' and don't use floating point.

    Either get familiar with fix/int/mod/round or "\" and apply judicious rounding or scale your own integers. For example, %percents are often calculated as 0-100 and not 0.00-1.00.

    Its this sort of stuff that sorts the men from the boys..!

Your Comments

Name:
E-mail:
(optional)
Website:
(optional)
Comment: