Tap Forms – Organizer Database App for Mac, iPhone, and iPad › Forums › Script Talk › Numeric value returns 16 decimal places
November 27, 2019 at 12:51 AM #38315
I’ve a simple loop over all records.
The table has several same fields (Number, Currency Style, 2 Decimal Places).
Playing with that, every field has the value 1.99
In some cases getFieldValue returns “1.9899999999999998”
Is this a bug ?November 27, 2019 at 3:04 AM #38326
It’s a byproduct of the use of the Objective-C
I tried that already by using var.toFixed(2),
but this is running in a TypeError: toFixed is not a function.
This happens only when the script is running over such an affected value, for other values toFixed is working.November 27, 2019 at 6:15 PM #38350
My guess is that the number if actually a string, use
typeofto check what type it is. If it’s a string then it wouldn’t have
toFixedbecause that’s for numbers. If it is a string then perhaps try a
parseFloatas a workaround.November 28, 2019 at 12:44 AM #38359
But should this happen when the field is numeric ?
However, found a workaround to fix it.November 28, 2019 at 9:51 AM #38373
Probably shouldn’t happen but bugs happen :)
What was your workaround?November 28, 2019 at 11:26 AM #38375
Ugly and sort of tricky.
Detecting the amount of decimal places.
If > 2 split the string to x.xx and convert to number.
Still unclear for me why this happens only for some fields and some values only, even if they all have the same format and the same value.November 28, 2019 at 2:45 PM #38382
Sorry, but that’s life when using floating point (non-integer) numbers in a general-purpose computer. It’s not anybody’s bug. It’s down to the fact that the representation of arbitrary floating point numbers in a fixed number of bits in the hardware can result in the computer being unable to represent some values exactly. So, for example, a floating point calculation which should result in 2.000 might instead give 1.99999…December 27, 2019 at 9:39 PM #38386
Yes, this is a standard problem with floating point numbers that values aren’t represented exactly. but are an approximation. Some values approximate well and others, as you’ve seen, end up looking a little weird if you don’t round them. There is an article called What Every Computer Scientist Should Know About Floating-Point Arithmetic which is perhaps a little heavy but is detailed on how floating point numbers are merely an approximation, the general standard is IEEE 754.
Another workaround is to store everything as whole numbers and then elsewhere handle adjusting for display purposes. This allows you to work with whole numbers throughout though it requires at least two fields: one for the value and a second for number of significant digits. You could store it in a single value if and only if you have a fixed number of digits. Then it becomes more a formatting issue. The challenge with this approach is when you need to do maths with the numbers, you need to convert them to an equivalent formats and integer math is restricted in size in many CPUs which is why we have the floating point system we do. These days that is less of an issue, a 64-bit processor can store an unfathomably large number, however a 16-bit processors largest number is 65535, and half that if you want to store negative values.
This is unfortunately less a Tap Forms issue and more an issue with hardware limitations of computers.
You must be logged in to reply to this topic.