Floating-point division returning incorrect values

Comments

1 comment

  • Avatar
    Adrian Williams

    Hi Dinesh,

    as you mentioned, it is not always possible to obtain an exact representation of a number using Floating Point arithmetic. Even simple calculations can result in inaccuracies in the result e.g. 

    Total = 0.1 + 0.1 + 0.1 - 0.3

    Which, from a school math perspective should result in a total of 0.0. 

     

    Data3Sixty Analyze does not provide native support for Decimal data types. However, for specific applications that require exact calculations you can leverage the Decimal package of the Python language. See the following link for details of the usage of the Decimal package:

    https://docs.python.org/2/library/decimal.html

    Attached is a data flow that provides some examples of how certain calculations can be implemented in custom nodes. In this case the examples include calculating: a running total; sum of values in a field, average of values in a field; x percent of a value; the percentage of two values.

    The Transform node provides a simple example to show the principles used and the Composite nodes show the use of further features of the module including how you can modify the precision and the rounding method employed (you probably want to use the 'ROUND_HALF_UP' Rounding type rather than the default). The Precision needs to be set appropriately (see hover text for the property).

    A key aspect of using Decimal type calculations is that you should acquire the numeric data as *an exact string representation of the numeric value* e.g. by casting the values to a string rather than outputting the data as a floating point (double) data type - else you may already have introduced an inexact value even before you have made any calculations. The intermediate data should also be passed from one node to another as the string representation of the values for the same reason.

    Regards,

    Adrian

     

    Attached files

    Decimal_Type_Math_Nodes_v1.0--share.lna

     

    0
    Comment actions Permalink

Please sign in to leave a comment.



Powered by Zendesk