HeroEngine Support > Scripting & Programming

[Clarification] Float math results - precision setting

(1/2) > >>

brainache:
Is there anyway to adjust Hero's floating point precision ( which is what I assume is causing the following)


The following code:
  val as Float = 17.0 / 10.0
  println("Floating Point Test: "+ val )
  val  = 18.0 / 10.0
  println("Floating Point Test: "+ val )

produces these results:

[hsl_debug] 9223372056477021789,3: SCRIPT[cmdMob]: Floating Point Test: 1.700000048
[hsl_debug] 9223372056477021789,3: SCRIPT[cmdMob]: Floating Point Test: 1.799999952

in other languages, or in a calculator, this would show:  1.7 and 1.8

Thazager:
I am not sure about the precision yet. But I have seen a few functions, like truncate, roundUp, roundDown, and round. Check the _ExternalFunctions in client side in the engine code for some that might help.

brainache:
Thanks, but I am not trying to round floats to integers... which is what all those do...

If there was a round to 1 decimal that would work.. but there isnt.. and the standard math way of doing that (ie  round ( x*10) / 10 ) doesnt work since the division just sends it back to the incorrect figures it started with..

EO-CaSpivey:
I tried all the methods I know off and all ended up with the same result, it would seem that HeroEngine's auto conversion is causing a bit of grief here....it looks like they will have to add an external function for this in the C++ source as HSL does not have this functionality and the auto conversion makes it impossible to do manually.

Sorry =(

HE-CHRISTOPHER:

--- Code: ---// Returns human-readable string representation of a float value, using standard rounding.
external function FormatFloat( value as Float, places as Integer) as String
--- End code ---

Navigation

[0] Message Index

[#] Next page

Go to full version