HeroEngine Forums

HeroEngine Support => Scripting & Programming => Topic started by: brainache on Mar 25, 13, 03:54:53 PM

Title: [Clarification] Float math results - precision setting
Post by: brainache on Mar 25, 13, 03:54:53 PM

Is there anyway to adjust Hero's floating point precision ( which is what I assume is causing the following)


The following code:
  val as Float = 17.0 / 10.0
  println("Floating Point Test: "+ val )
  val  = 18.0 / 10.0
  println("Floating Point Test: "+ val )

produces these results:

[hsl_debug] 9223372056477021789,3: SCRIPT[cmdMob]: Floating Point Test: 1.700000048
[hsl_debug] 9223372056477021789,3: SCRIPT[cmdMob]: Floating Point Test: 1.799999952

in other languages, or in a calculator, this would show:  1.7 and 1.8

Title: Re: Float math results - precision setting
Post by: Thazager on Mar 25, 13, 07:13:13 PM

I am not sure about the precision yet. But I have seen a few functions, like truncate, roundUp, roundDown, and round. Check the _ExternalFunctions in client side in the engine code for some that might help.

Title: Re: Float math results - precision setting
Post by: brainache on Mar 25, 13, 07:59:57 PM

Thanks, but I am not trying to round floats to integers... which is what all those do...

If there was a round to 1 decimal that would work.. but there isnt.. and the standard math way of doing that (ie  round ( x*10) / 10 ) doesnt work since the division just sends it back to the incorrect figures it started with..

Title: Re: Float math results - precision setting
Post by: EO-CaSpivey on Mar 26, 13, 01:38:20 PM

I tried all the methods I know off and all ended up with the same result, it would seem that HeroEngine's auto conversion is causing a bit of grief here....it looks like they will have to add an external function for this in the C++ source as HSL does not have this functionality and the auto conversion makes it impossible to do manually.

Sorry =(

Title: Re: Float math results - precision setting
Post by: HE-CHRISTOPHER on Mar 27, 13, 08:46:44 AM

Code: [Select]

// Returns human-readable string representation of a float value, using standard rounding.
external function FormatFloat( value as Float, places as Integer) as String

Title: Re: [Clarification] Float math results - precision setting
Post by: EO-CaSpivey on Mar 27, 13, 09:26:37 AM

I am confused as to how this is working....This is what I used to test.

Code: [Select]

val as Float  = 18.0 / 10.0
println("Floating Point Test: "+ val )
valString as String = FormatFloat(val , 2)
println("Floating Point Test String: "+ valString )
val2 as Float = stof(valString)
println("Floating Point Test After Conversion: "+ val2 )

And this is what was returned:

[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test: 1.799999952
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test String: 1.80
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test After Conversion: 1.799999952

How is it getting the 1.799999952 from the string 1.80?

This is not really an issue for me, i am just confused as to how it happened.

Thanks,
Charles

Title: Re: [Clarification] Float math results - precision setting
Post by: HE-HERB on Mar 27, 13, 09:29:34 AM

It's the nature of floating point math.

You can google "floating point precision", but here's one good article: http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Cheers