HeroEngine Forums
HeroEngine Support => Scripting & Programming => Topic started by: brainache on Mar 25, 13, 03:54:53 PM
-
Is there anyway to adjust Hero's floating point precision ( which is what I assume is causing the following)
The following code:
val as Float = 17.0 / 10.0
println("Floating Point Test: "+ val )
val = 18.0 / 10.0
println("Floating Point Test: "+ val )
produces these results:
[hsl_debug] 9223372056477021789,3: SCRIPT[cmdMob]: Floating Point Test: 1.700000048
[hsl_debug] 9223372056477021789,3: SCRIPT[cmdMob]: Floating Point Test: 1.799999952
in other languages, or in a calculator, this would show: 1.7 and 1.8
-
I am not sure about the precision yet. But I have seen a few functions, like truncate, roundUp, roundDown, and round. Check the _ExternalFunctions in client side in the engine code for some that might help.
-
Thanks, but I am not trying to round floats to integers... which is what all those do...
If there was a round to 1 decimal that would work.. but there isnt.. and the standard math way of doing that (ie round ( x*10) / 10 ) doesnt work since the division just sends it back to the incorrect figures it started with..
-
I tried all the methods I know off and all ended up with the same result, it would seem that HeroEngine's auto conversion is causing a bit of grief here....it looks like they will have to add an external function for this in the C++ source as HSL does not have this functionality and the auto conversion makes it impossible to do manually.
Sorry =(
-
// Returns human-readable string representation of a float value, using standard rounding.
external function FormatFloat( value as Float, places as Integer) as String
-
I am confused as to how this is working....This is what I used to test.
val as Float = 18.0 / 10.0
println("Floating Point Test: "+ val )
valString as String = FormatFloat(val , 2)
println("Floating Point Test String: "+ valString )
val2 as Float = stof(valString)
println("Floating Point Test After Conversion: "+ val2 )
And this is what was returned:
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test: 1.799999952
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test String: 1.80
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test After Conversion: 1.799999952
How is it getting the 1.799999952 from the string 1.80?
This is not really an issue for me, i am just confused as to how it happened.
Thanks,
Charles
-
It's the nature of floating point math.
You can google "floating point precision", but here's one good article: http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
Cheers