logo

Comments

    • avatar
    • Andy Dempster`
    • Thu 1 Aug 2013 07:32 AM

    I don't know if it's still the case but it used to be an issue with rounding that @Formula would round one way and LotusScript would round t'uther!

    This came to light on an e-commerce site where the basket used @Formula on screen but the checkout processing was done by a LS lib behind the scenes and they always had differing amounts!

    1. Scientific vs Accounting rounding. I'm not sure which one @formula uses by default but it's the reason the Currency datatype exists in LS.

      Show the rest of this thread

  1. Yep, floating point is a pain, but nothing new. We've had 'irrational numbers' for centuries! 1/3 as a decimal = ? 0.33333... or root 2 = 1.414213..., natural log anybody? Binary just does a similar approximation in base 2.

    The main issue that people seem to fail to deal with is the difference between the internal binary representation and the 'displayed value' that often has an implied internal rounding process. Where this gets nasty is when you total up long lists of numbers and get a total which is slightly different to whats displayed. This is what 'Decimal/Money' types are for in other databases. They are 'scaled integers' and don't use floating point.

    Either get familiar with fix/int/mod/round or "\" and apply judicious rounding or scale your own integers. For example, %percents are often calculated as 0-100 and not 0.00-1.00.

    Its this sort of stuff that sorts the men from the boys..!

    1. 1/3 is not an irrational number. :-)

  2. NoNoNo... You don't round, you tranfer that bit to your offshore account.

  3. There's something wrong also in LSS:

    Don't know if in 8.5.3 this was fixed...

    1. Hey Jake, there's something wrong also on your blog parsing urls :)

Your Comments

Name:
E-mail:
(optional)
Website:
(optional)
Comment:


About This Page

Written by Jake Howlett on Thu 1 Aug 2013

Share This Page

# ( ) '

Comments

The most recent comments added:

Skip to the comments or add your own.

You can subscribe to an individual RSS feed of comments on this entry.

Let's Get Social


About This Website

CodeStore is all about web development. Concentrating on Lotus Domino, ASP.NET, Flex, SharePoint and all things internet.

Your host is Jake Howlett who runs his own web development company called Rockall Design and is always on the lookout for new and interesting work to do.

You can find me on Twitter and on Linked In.

Read more about this site »

More Content