ROUND() function produces unexpected results in Spotfire Analyst.
Article ID: KB0079116
Updated On:
| Products | Versions |
|---|---|
| Spotfire Analyst | All Versions |
Description
In some scenarios, Round() does not show the correct value on the numeric scale. Below is an example:
1) 45.25*(1-0.3) = 31.67500 => Correct Value
2) Round(45.25*(1-0.3),2) = 31.67000 => Incorrect Value
3) Round(31.675,2) = 31.68 => Correct Value
1) 45.25*(1-0.3) = 31.67500 => Correct Value
2) Round(45.25*(1-0.3),2) = 31.67000 => Incorrect Value
3) Round(31.675,2) = 31.68 => Correct Value
Issue/Introduction
ROUND() function produces unexpected results in Spotfire Analyst.
Environment
All Supported Operating Systems
Resolution
This is as expected with precision loss using floating-point maths(e.g. the Real data type). If decimal precision is required, then always use the "Currency" data type that doesn't use floating-point maths. It's slower but more accurate with decimal numbers.
Feedback
Yes
No
Powered by
