Error Correction in Lotus Notes Maths
As a follow-up to a post I wrote recently about JavaScript maths errors here's another. This time it's about the same issue in Lotus Notes @Formula-based maths.
See Exhibit A below:
As mentioned previously it's nothing to do with the language used. It's all down to the hardware.
Always remember to @Round the result of any mathematics, like so:
I don't know if it's still the case but it used to be an issue with rounding that @Formula would round one way and LotusScript would round t'uther!
This came to light on an e-commerce site where the basket used @Formula on screen but the checkout processing was done by a LS lib behind the scenes and they always had differing amounts!
Reply
Scientific vs Accounting rounding. I'm not sure which one @formula uses by default but it's the reason the Currency datatype exists in LS.
Reply
Hide the rest of this thread
Julian Robichaux had blogged about this back in 2005, see http://www.nsftools.com/blog/blog-02-2005.htm.
Formula language uses wrong rounding (aka account rounding or banker's rounding), LS uses correct rounding (aka scientific rounding. Damien Katz approves the difference in his comment on this blog.
Seriously, there is no wrong rounding (or rather: no right rounding), it's all a matter of definition. I can see the rationale behind accounting rounding, but still think it's a really bad idea. Even worse to have different rounding algorithms implemented in different languages in the same software product.
Reply
Correction:
@Formulas USED to use wrong rounding. I just checked and the fix introduced in 6.5.1 is still present in 8.5.3.
Reply
Yep, floating point is a pain, but nothing new. We've had 'irrational numbers' for centuries! 1/3 as a decimal = ? 0.33333... or root 2 = 1.414213..., natural log anybody? Binary just does a similar approximation in base 2.
The main issue that people seem to fail to deal with is the difference between the internal binary representation and the 'displayed value' that often has an implied internal rounding process. Where this gets nasty is when you total up long lists of numbers and get a total which is slightly different to whats displayed. This is what 'Decimal/Money' types are for in other databases. They are 'scaled integers' and don't use floating point.
Either get familiar with fix/int/mod/round or "\" and apply judicious rounding or scale your own integers. For example, %percents are often calculated as 0-100 and not 0.00-1.00.
Its this sort of stuff that sorts the men from the boys..!
Reply
1/3 is not an irrational number. :-)
Reply
NoNoNo... You don't round, you tranfer that bit to your offshore account.
Reply
There's something wrong also in LSS:
Don't know if in 8.5.3 this was fixed...
Reply
Hey Jake, there's something wrong also on your blog parsing urls :)
Reply