HeroEngine Forums
Welcome, Guest. Please login or Register for HeroCloud Account.

Author Topic: [Clarification] Float math results - precision setting  (Read 3083 times)

brainache

  • World Owner
  • ****
  • Posts: 121
    • View Profile
[Clarification] Float math results - precision setting
« on: Mar 25, 13, 03:54:53 PM »

Is there anyway to adjust Hero's floating point precision ( which is what I assume is causing the following)


The following code:
  val as Float = 17.0 / 10.0
  println("Floating Point Test: "+ val )
  val  = 18.0 / 10.0
  println("Floating Point Test: "+ val )

produces these results:

[hsl_debug] 9223372056477021789,3: SCRIPT[cmdMob]: Floating Point Test: 1.700000048
[hsl_debug] 9223372056477021789,3: SCRIPT[cmdMob]: Floating Point Test: 1.799999952

in other languages, or in a calculator, this would show:  1.7 and 1.8

« Last Edit: Mar 27, 13, 09:13:58 AM by HE-HERB »
Logged

Thazager

  • General Accounts
  • *
  • Posts: 1144
  • Never stop learning
    • View Profile
    • Heroes and Villains MMORPG
Re: Float math results - precision setting
« Reply #1 on: Mar 25, 13, 07:13:13 PM »

I am not sure about the precision yet. But I have seen a few functions, like truncate, roundUp, roundDown, and round. Check the _ExternalFunctions in client side in the engine code for some that might help.
Logged

brainache

  • World Owner
  • ****
  • Posts: 121
    • View Profile
Re: Float math results - precision setting
« Reply #2 on: Mar 25, 13, 07:59:57 PM »

Thanks, but I am not trying to round floats to integers... which is what all those do...

If there was a round to 1 decimal that would work.. but there isnt.. and the standard math way of doing that (ie  round ( x*10) / 10 ) doesnt work since the division just sends it back to the incorrect figures it started with..
Logged

EO-CaSpivey

  • General Accounts
  • *
  • Posts: 16
  • HSL Engineer
    • View Profile
    • Exile Online
Re: Float math results - precision setting
« Reply #3 on: Mar 26, 13, 01:38:20 PM »

I tried all the methods I know off and all ended up with the same result, it would seem that HeroEngine's auto conversion is causing a bit of grief here....it looks like they will have to add an external function for this in the C++ source as HSL does not have this functionality and the auto conversion makes it impossible to do manually.

Sorry =(
Logged
HSL Engineer | Exile Gaming Studio
Website: www.exile-online.com
Follow Us: www.indiedb.com/games/exileonline

HE-CHRISTOPHER

  • HeroEngine
  • *****
  • Posts: 424
    • View Profile
Re: Float math results - precision setting
« Reply #4 on: Mar 27, 13, 08:46:44 AM »

Code: [Select]
// Returns human-readable string representation of a float value, using standard rounding.
external function FormatFloat( value as Float, places as Integer) as String
Logged
Christopher Larsen
CTO
HeroEngine

EO-CaSpivey

  • General Accounts
  • *
  • Posts: 16
  • HSL Engineer
    • View Profile
    • Exile Online
Re: [Clarification] Float math results - precision setting
« Reply #5 on: Mar 27, 13, 09:26:37 AM »

I am confused as to how this is working....This is what I used to test.

Code: [Select]
val as Float  = 18.0 / 10.0
println("Floating Point Test: "+ val )
valString as String = FormatFloat(val , 2)
println("Floating Point Test String: "+ valString )
val2 as Float = stof(valString)
println("Floating Point Test After Conversion: "+ val2 )

And this is what was returned:

[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test: 1.799999952
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test String: 1.80
[hsl_debug] 4611686018427637930,3: SCRIPT[skippy]: Floating Point Test After Conversion: 1.799999952

How is it getting the 1.799999952 from the string 1.80?

This is not really an issue for me, i am just confused as to how it happened.

Thanks,
Charles
Logged
HSL Engineer | Exile Gaming Studio
Website: www.exile-online.com
Follow Us: www.indiedb.com/games/exileonline

HE-HERB

  • HeroEngine
  • *****
  • Posts: 530
    • View Profile
    • HeroEngine
Re: [Clarification] Float math results - precision setting
« Reply #6 on: Mar 27, 13, 09:29:34 AM »

It's the nature of floating point math.

You can google "floating point precision", but here's one good article: http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Cheers
Logged
herb marselas
graphics guy