[SOLVED] Generate Attributes - Strange issue

rowan_g

Hi All,

I've got several basic calculations in the Generate Attributes operator.

I've got one calculation that calculates correctly for some examples but incorrectly for about 5-10% of them.
The calculation is:

if(Amount==(Rate*Units),1,0)

I've got 2 read database operators and a join. The data types for Amount and Rate is "Real". Units is "Int"

For some reason it calculates correctly for most of them but a certain few come up incorrectly. I checked the decimal places of the numbers and they're identical.

Any help would be much appreciated

Cheers,

Nils_Woehler

Hi,

what do you mean be 'calculating incorrectly'? Could you give an example?

Best,
Nils

rowan_g

Hi Nils,

The calculation is inconsistent.
For example, using the formula if(Amount==(Rate*Units),1,0)
if(7==(7),1,0) will return 1 in some scenarios and 0 in others without a consistent pattern. It should always be 1 because 7==7........

Is it because of the data types? when you multiply an int by a real number does that cause some sort of issue?
Thanks.

Cheers,

rowan_g

I figured this out.....when you multiply real by int something weird happens to the data type.
The way I fixed it is by having a "Format Numbers" operator after my first "Generate Attributes" operator to reformat the result. After that I put another "Generate Attributes" operator to perform my calculation.
Not ideal but it does the job.

Thanks anyway

Nils_Woehler

Hi,

even if you have solved your problem by now I would like to understand what was going on: What datatypes does Amount, Rates and Units have?

Best,
Nils

rowan_g

From my first post:
The data types for Amount and Rate is "Real". Units is "Int"


Generate attributes operator 1:
Amount - Real
Rate - Real
Units - Int
Calculation:
Correct_Amount=Rate*Units


Format Numbers Operator:
Correct_Amount
Amount
Subset formatted to numeric


Generate attributes operator 2:
Correct_Amount - Numeric
Amount - Numeric
Rate - Real
Units - Int
Calculation:
if(Correct_Amount==Amount,1,0)

Nils_Woehler

So I think the problem here is, that you test for equality of real values with '=='.
You can have a look here: http://stackoverflow.com/questions/1088216/whats-wrong-with-using-to-compare-floats-in-java

Best,
Nils