HeroEngine Support > Scripting & Programming

[Clarification] Float math results - precision setting

<< < (2/2)

EO-CaSpivey:
I am confused as to how this is working....This is what I used to test.


--- Code: ---val as Float  = 18.0 / 10.0
println("Floating Point Test: "+ val )
valString as String = FormatFloat(val , 2)
println("Floating Point Test String: "+ valString )
val2 as Float = stof(valString)
println("Floating Point Test After Conversion: "+ val2 )

--- End code ---

And this is what was returned:

[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test: 1.799999952
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test String: 1.80
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test After Conversion: 1.799999952

How is it getting the 1.799999952 from the string 1.80?

This is not really an issue for me, i am just confused as to how it happened.

Thanks,
Charles

HE-HERB:
It's the nature of floating point math.

You can google "floating point precision", but here's one good article: http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Cheers

Navigation

[0] Message Index

[*] Previous page

Go to full version