HeroEngine Forums
Welcome, Guest. Please login or Register for HeroCloud Account.

Author Topic: [Fixed]Localization Questions  (Read 4342 times)

GlorianLanTarini

  • General Accounts
  • *
  • Posts: 148
    • View Profile
[Fixed]Localization Questions
« on: Jan 20, 15, 11:50:46 PM »

Hello all,

Yes, i know about HeroWiki localization pages, and read it many times. But, at implementation still have errors. So, my main question: how and where i should initialize and setup localization system (server and client part respectively). Especially, if I prefer synchronous getting of localized strings from LRC (don't want to hit bandwidth with text requests).

Thanks for any tips and information.
« Last Edit: Jan 29, 15, 07:21:53 AM by GlorianLanTarini »
Logged

FI-ScottZ

  • General Accounts
  • *
  • Posts: 1407
    • View Profile
    • Forever Interactive, Inc.
Re: Localization Questions
« Reply #1 on: Jan 21, 15, 01:22:25 PM »

To what extent have you implemented it?

You have put terms in the Localization Table Editor?

Using the $LOCALIZE node to access the terms?

Where and when do you get errors?

As far as synchronous getting of localized strings, the client-side $LOCALIZE node has a method _GetLocalizedStringValueSync(), but that is not guaranteed to succeed.
« Last Edit: Jan 21, 15, 02:29:06 PM by FI-ScottZ »
Logged
Scott Zarnke
Lead Programmer, Visions of Zosimos
CTO, Forever Interactive, Inc.

FI-ScottZ

  • General Accounts
  • *
  • Posts: 1407
    • View Profile
    • Forever Interactive, Inc.
Re: Localization Questions
« Reply #2 on: Jan 21, 15, 02:08:09 PM »

Incidentally, this thread prompted me to investigate the localization script code in depth, and I found what I believe is a typo in server _LocalizationClassMethods:

Code: [Select]
method _LOC_GetDefaultBaseLocale() as String
  handled as Boolean
  if hasMethod( me, "HE_LOC_GetDefaultBaseLocale" )
    handled = me._LOC_GetDefaultBaseLocale()
  .
  if not handled
    return "en-US"
  .
 
.
After checking for the existence of HE_LOC_GetDefaultBaseLocale, it should be calling that method rather than the base one.
Logged
Scott Zarnke
Lead Programmer, Visions of Zosimos
CTO, Forever Interactive, Inc.

FI-ScottZ

  • General Accounts
  • *
  • Posts: 1407
    • View Profile
    • Forever Interactive, Inc.
Re: Localization Questions
« Reply #3 on: Jan 21, 15, 07:01:31 PM »

Also, when I tried calling server external function LOC_RTGetLocalizedStringSynchronously() from any server script, I get the error:
Quote
SCRIPT ERROR: External function LOC_RTGETLOCALIZEDSTRINGSYNCHRONOUSLY not found

Is there a better way to get a localized string on the server side?
(I'm wanting to get the string in _GetSpecHeaderValuesForSpec() of a spec oracle class so that I can customize the appearance of a LocalStringNumber field in the spec editor and have it appear along with its string).
Logged
Scott Zarnke
Lead Programmer, Visions of Zosimos
CTO, Forever Interactive, Inc.

GlorianLanTarini

  • General Accounts
  • *
  • Posts: 148
    • View Profile
Re: Localization Questions
« Reply #4 on: Jan 21, 15, 09:30:43 PM »

Hi Scott,

To what extent have you implemented it?

You have put terms in the Localization Table Editor?

Using the $LOCALIZE node to access the terms?

Where and when do you get errors?

Well, I decided using localization system to storage all text data of my game. One of reason - game must support 2 languages at release (EN and RU). Also, i used to use strings table for storage game texts in past.
So, I add new locale (RU) to Localization Table and create some game text (basically for buttons) with LTE. Then I try to recive text for button from CSS and CCS using $LOCALIZE. And get error - Localization System don't started. (sorry, don't have opportunity to copy-paste message of this error just right now). Then I try to start loc system with initializing method from $LOCALIZE script, but its don't work properly - because config file loaded always after holding area. I try different scripts, and one time suceed to load config before CSS, but universe don't loaded at all.
So, my question still same. How to setup and initialize Localization system properly. This is not considered in the wiki  :(

As far as synchronous getting of localized strings, the client-side $LOCALIZE node has a method _GetLocalizedStringValueSync(), but that is not guaranteed to succeed.

Yes, i know it. And i know why that is not guaranteed to succeed. It's fine, just need to check and if don't work - go asynchronous. But, maybe there is a way to make chance of succeed higher?

P.S.: The team has a professional translator, but his responsibility is not a translation forum threads. So don't worry, the game text in English will be much better than this.  ;)
Logged

FI-ScottZ

  • General Accounts
  • *
  • Posts: 1407
    • View Profile
    • Forever Interactive, Inc.
Re: Localization Questions
« Reply #5 on: Jan 21, 15, 11:32:16 PM »

OK.  As of this evening I got localization strings to work for some of our GUI labels.  I'll give you a run down of our process.  Keep in mind this was just done today. It has been tested and works for us, but there could be better ways to do this.  Sorry for the long post, but I wanted to be thorough.



One thing I did, which you may have seen in the list of GUI controls, is to use the _LabelLocalized control in place of normal labels.  It already has all of the code set up to request the strings so that all you need to do is assign it a string number and it sets its text automatically.  Very useful.  I did modify the script code a little bit:
  • In OnControlBuild() before calling $LOCALIZE._GetLocalizedStringValueAsync() the normal code checks if the number is 0, since 0 will cause an error.  But the FieldChangeNotification() function did not have that check, so I added the check there.
  • I was finding that even though _GotLocalizedStringValueAsync() is required, it did not seem to get called when the string was loaded; only LOC_OnRTGetLocalizedString() was getting called.  Hence, _localizationStringValue was not being set. Also, when an invalid string number was entered, _GetLocalizedStringValueAsyncFailed() was not being called, only LOC_OnRTGetLocalizedStringFailed().  So I just changed LOC_OnRTGetLocalizedStringFailed() to call _GetLocalizedStringValueAsyncFailed().  I am not sure if that is the totally correct approach.
  • Lastly, I wrapped a lot of the commonly used code into local functions so they are private.  Also, if the string number is 0, rather than do nothing it sets the text to an empty string.  That way we can use it in a node editor cell and be able to assign 0 to make the string empty.

This is what the script looks like for us now:
Code: [Select]
shared function FieldChangeNotification( fieldName as String )
  when tolower(fieldName)
    is "_localizationstringnumber"
      _LocalizationStringNumberToText()
    .
  .
.

method onControlBuild()
  _LocalizationStringNumberToText()
.

method _SetLocalizationStringNumber(string_number as Integer)
  me._localizationStringNumber = string_number
.

method _GetLocalizedText() as String
  return me.text
.
method _GotLocalizedStringValueAsync( localize_request_id as ID, localize_request_string_number as Integer, localize_string_value as String)
  _SetLocalizedText(localize_string_value)
.
method _GetLocalizedStringValueAsyncFailed(localize_request_id as ID, localize_string_number as Integer, fail_reason as String)
  _OnGetLocalizedStringFailed(fail_reason)
.

//callbacks
method LOC_OnRTGetLocalizedString( async_request_id as ID, localized_string as String )
  _SetLocalizedText(localized_string)
.

method LOC_OnRTGetLocalizedStringFailed( async_request_id as ID, errmsg as String )
  _OnGetLocalizedStringFailed(errmsg)
.

method LOC_OnRTError( external_function_name as String, errmsg as String )
  println("$R$R !!! ERROR reported by " + external_function_name + " !!!$R " + errmsg + "$R$R")
.

//private common code
function _LocalizationStringNumberToText()
  where me is kindof _GUILabelLocalized
    if me._localizationStringNumber != 0
      $LOCALIZE._GetLocalizedStringValueAsync(me,  me._localizationStringNumber)
    else
      _SetLocalizedText("")
    .
  .
.

function _SetLocalizedText(text as String)
  where me is kindof _GUILabelLocalized
    me._localizationStringValue = text
    me.text = me._localizationStringValue
  .
.
function _OnGetLocalizedStringFailed(fail_reason as String)
  where me is kindof _GUILabelLocalized
    me._localizationStringValue = "failed: " + fail_reason
    $LOCALIZE.Debug("error retrieving " + me._localizationStringNumber + " " + me._localizationStringValue )
    me.text = me._localizationStringValue
  .
.

Next, I found like you did that I was not able to do things such as set the current locale in HE_FirstChance() since at that point the configuration file is not yet loaded.  I looked around and saw this method in _LocalizeClassMethods: LOC_OnRTLoadFile().  It is a notification of when the configuration file is loaded, but there is no code in there to notify the custom code.  What I did then was Glom a class onto the $Localize system node via the System Nodes Configuration GUI* and in our VOZ_Localize script we defined this method:
Code: [Select]
method HE_OnRTLoadFile(key as ID) as Boolean
  if key == 1
    //config file is loaded.
   
    //also use this method elsewhere to change language by user request in the future:
    $LOCALIZE.LOC_SetCurrentLocale("en-US")
  .
  return true
.
and rewrote LOC_OnRTLoadFile as follows:
Code: [Select]
method LOC_OnRTLoadFile( key as ID )
  //
  // LOC_OnRTLoadFile is called in response to successful asynchronous download of config files and universe files.
  //
  handled as Boolean
  if HasMethod(me, "HE_OnRTLoadFile")
    handled = me.HE_OnRTLoadFile(key)
  .
  if not handled
    if (key == 1) // this is a config file...
 
      println( "$R$R *** CONFIG FILE DOWNLOADED SUCCESSFULLY ***$R$R" )

    else // this is a universe file...
   
      println( "$R$R *** UNIVERSE FILE " + key + " DOWNLOADED SUCCESSFULLY ***$R$R" )
    .
  .
.
You could also write a similar handler to be notified of when LOC_OnRTLoadFileFailed() is called if you wanted to know that.  Of course, it can be risky modifying Clean Engine code, but if you do be sure to record which files you changed so those changes can be added back in if they are wiped out by engine updates.  That is what we do, anyways.

And finally note that LOC_RTLoadConfiguration() is only called in _LOC_InitializeClientLocalizationSystem(), which is itself only called when:
  • The LTE is opened.
  • LOC_SetCurrentLocale() is called.**
  • _GetLocalizedStringValueAsync() is called. OR
  • _GetLocalizedStringValueSync() is called.
So those are the only things that trigger the configuration file to be loaded.



* Of course, LOCALIZE does not appear in the default Client list of the System Nodes Configuration GUI, so you would need to glom an override class onto SYSTEMNODES via that same GUI, then define in that override script this:
Code: [Select]
method HE_GetRegisteredClientSystemNodeNames( systemNodeNames references List of String ) as Boolean
// Used by $SYSTEMNODES to add client system node names to the list in the Control Panel.
//
// Return true if you do NOT want the default system node names added
// to the List by _GetRegisteredClientSystemNodeNames(), but instead only the ones in this method.
  add back "COMMANDHANDLER" to systemNodeNames
  add back "LOCALIZE" to systemNodeNames
 
  return false
.
(COMMANDHANDLER is another system node I found not in that list.)

**LOC_SetCurrentLocale() still fails the first time even though it is calling _LOC_InitializeClientLocalizationSystem(), because LOC_RTSetCurrentLocale() is called immediately after, which usually does not give enough time for the configuration file to be loaded.
Logged
Scott Zarnke
Lead Programmer, Visions of Zosimos
CTO, Forever Interactive, Inc.

GlorianLanTarini

  • General Accounts
  • *
  • Posts: 148
    • View Profile
Re: Localization Questions
« Reply #6 on: Jan 22, 15, 01:12:23 AM »

Thanks a lot, Scott.
You give me great information. I should try it today, and will post about result.
Logged

GlorianLanTarini

  • General Accounts
  • *
  • Posts: 148
    • View Profile
Re: Localization Questions
« Reply #7 on: Jan 23, 15, 12:30:21 AM »

It took more time than I had assumed. But, in the end, almost everything works as I wanted. At least it looks. Further testing will show if it is. I describe all what done in the tutorial style, omitting all attempts and mistakes.

First of all, as a system node, $LOCALIZE too error prone, we will create our own clientside system node $LOCALIZER to perform our tasks. The basis of the prototype will be a class with fields: game_currentLocale (type: String), game_LocalizedRequests (type ID), game_LocalizedStringTarget (type: lookuplist indexed by ID of NodeRef) and game_LocalizerStarted (type: Boolean).
(Of course, you need to replace game in the name of fields on your own prefix, if you follow this instruction)
His script is follows:
Code: [Select]
method StartItUp()
  success as Boolean = LOC_RTStartup ($LOCALIZER)
  if success
    me.game_LocalizerStarted = true
  .
  LOC_RTLoadConfiguration ($LOCALIZE._LOC_GetRuntimeConfigurationFilePath())
.
method LOC_OnRTLoadFile( key as ID )
  //
  // LOC_OnRTLoadFile is called in response to successful asynchronous download of config files and universe files.
  //
    if (key == 1) // this is a config file...
      println( "$R *** CONFIG FILE DOWNLOADED SUCCESSFULLY *** $R" )
     
      loc as String = $LOCALIZER.game_currentLocale
    if loc = ""
      println("Empty locale field. Set en-US")
      loc = "en-US"
    else
      println("Get locale field = " + loc)
    .
    if LOC_RTSetCurrentLocale(loc)
      request_id as ID = me.GetReqID()
      StrID as Integer = 1
      LOC_RTGetLocalizedString(StrID, request_id, me)
    .
    else // this is a universe file...
   
      println( "$R *** UNIVERSE FILE " + key + " DOWNLOADED SUCCESSFULLY ***$R" )
    .
  .

// ----------------------------------------------------------------------------------------------------------------------------

method LOC_OnRTLoadFileFailed( key as ID, errmsg as String )
  //
  // LOC_OnRTLoadFileFailed is called in response to failed asynchronous download of config files and universe files.
  //
  if (key == 1)
    println( "$R$R !!! ERROR: CONFIG FILE DOWNLOAD FAILED !!!$R " + errmsg + "$R$R" )
  else
    println( "$R$R !!! ERROR: UNIVERSE FILE " + key + " DOWNLOAD FAILED !!!$R$R" )
  .
.

// ----------------------------------------------------------------------------------------------------------------------------

method LOC_OnRTLoadStringTable( key as ID )
  println("$R$R *** STRING TABLE " + key + " DOWNLOADED SUCCESSFULLY ***$R$R")
.

// ----------------------------------------------------------------------------------------------------------------------------

method LOC_OnRTLoadStringTableFailed( key as ID, base_locale_name as String, base_locale_load_status as String, base_locale_errmsg as String,
current_locale_name as String, current_locale_load_status as String, current_locale_errmsg as String )
 
  ok as Boolean = false
  if (current_locale_load_status == "LOADED")
    ok = true
  .
  if (base_locale_load_status == "LOADED")
    ok = true
  .

  msg as String
  if (ok == true)
    msg = "$R$R *** STRING TABLE " + key + " DOWNLOADED WITH WARNINGS: "
  else
    msg = "$R$R !!! ERROR: STRING TABLE " + key + " DOWNLOAD FAILED:$R"
  .

  if (current_locale_load_status == "UNDEFINED")
    msg += "Current locale has not been defined."
  else if (current_locale_load_status == "NO FILE")
    msg += "String table " + key + " does not specify a string table filename for current locale $Q" + current_locale_name + "$Q. "
  else if (current_locale_load_status == "LOADED")
    msg += "String table file for current locale $Q" + current_locale_name + "$Q loaded successfully. "
  .
 
  if (base_locale_load_status == "NO FILE")
    msg += "String table " + key + " does not specify a string table filename for base locale $Q" + base_locale_name + "$Q.$R"
  else if (base_locale_load_status == "LOADED")
    msg += "String table file for base locale $Q" + base_locale_name + "$Q loaded successfully.$R"
  else
    msg += "$R"
  .
 
  if (current_locale_load_status == "LOAD FAILED")
    msg += "$RString table file load for current locale $Q" + current_locale_name + "$Q failed:$R" + current_locale_errmsg + "$R$R"
  .
 
  if (base_locale_load_status == "LOAD FAILED")
    msg += "$RString table file load for base locale $Q" + base_locale_name + "$Q failed:$R" + base_locale_errmsg + "$R$R"
  .
 
  println(msg)
.

// ----------------------------------------------------------------------------------------------------------------------------

method LOC_OnRTGetLocalizedString(request_id as ID, LocString as String )
  if me.game_LocalizedStringTarget has request_id
    tgt_gui as NodeRef = me.game_LocalizedStringTarget[request_id]
    SetText(tgt_gui, locString)
    remove request_id from me.game_LocalizedStringTarget destroy
  else
    println("Got localized string without target. String is:" + LocString)
  .
.

// ----------------------------------------------------------------------------------------------------------------------------

method LOC_OnRTGetLocalizedStringFailed(request_id as ID, fail_reason as String )
  ScriptErrorAndContinue("Can't get localized string. Reason = " + fail_reason)
.

// ----------------------------------------------------------------------------------------------------------------------------

method LOC_OnRTError( external_function_name as String, errmsg as String )
  println("$R$R !!! ERROR reported by " + external_function_name + " !!!$R " + errmsg + "$R$R")
.

method LocalizeMe(tgt_gui as NodeRef, StrID as Integer)
  where tgt_gui
    is kindof GUILabel
      locString as String
      if not LOC_RTGetLocalizedStringSynchronously(StrID, locString)
        request_id as ID = me.GetReqID()
        LOC_RTGetLocalizedString(StrID, request_id, me)
        me.game_LocalizedStringTarget[request_id] = tgt_gui
      else
        SetText(tgt_gui, locString)
      .
    .
    default
      ScriptErrorAndContinue("Recived node is not a GUILabel")
    .
  .
.

function SetText(tgt_gui as NodeRef of Class GUILabel, LocString as String)
 
  tgt.text = LocString
.

method GetReqID() as ID
  me.game_LocalizedRequests += 1
  return me.game_LocalizedRequests
.

Next, in game overriding for system node $BASECLIENT in method HE_Area_Preload add following code:

Code: [Select]
  if not $LOCALIZER.game_LocalizerStarted
    $LOCALIZER.StartItUp()
  .
  RepLocFQN as String = "locale_setting.ini"
  os_loc as String
  if not DoesLocalRepositoryFQNExist(RepLocFQN)
      if LOC_RTGetOSLocale(os_loc)
        when os_loc
          is "ru-RU"
            os_loc = "ru"
          .
          default
            os_loc = "en-US"
          .
        .
      else
        println("Can't get OS locale. Set en-US")
        os_loc = "en-US"
      .
    if SaveLocalRepositoryData(RepLocFQN, SYSTEM.EXEC.THISSCRIPT, os_loc)
      println("Repository locale_setting.ini created.")
    .
    $LOCALIZER.game_currentLocale = os_loc
  else
    println("Repository locale_setting.ini find. Read settings.")
    LoadLocalRepositoryData(RepLocFQN,SYSTEM.EXEC.THISSCRIPT)
  .

and create new function in this script with this code:

Code: [Select]
function RepositoryDataDownloaded(name as String, requestTag as ID, successful as Boolean, data as String) as Boolean
  if successful
    if data = "ru" or data = "en-US"
      $LOCALIZER.game_currentLocale = data
    .
  return true
  .
return false
.

Of course you can comment or delete all println, if you don't need tracing any more.

So, now you can set localized string for any label just using $LOCALIZER.LocalizeMe(any_GUILabel as NodeRef, StrID as Integer)

And of course, it's can be modified to work with another GUI controls which have text inside.


Now I have only one problem with localization: when I run the Localization Table Editor, I get an error
Quote
!!! ERROR: CONFIG FILE DOWNLOAD FAILED !!!
 "/Localization/Development/Default/Configuration.loc_run_configuration" is already loaded. It must be unloaded before another run-time configuration can be loaded.

and LTE don't work correctly, or even don't work at all. But, I work on it. Glad to read any ideas about ways to correct this.
« Last Edit: Jan 23, 15, 12:54:43 AM by GlorianLanTarini »
Logged

GlorianLanTarini

  • General Accounts
  • *
  • Posts: 148
    • View Profile
Re: Localization Questions
« Reply #8 on: Jan 29, 15, 07:21:13 AM »

Okey, fine. I lost some time for another thing, but now back to localization and fix last problem (i hope).

For stop getting error from Localization Window, just need to add one "if" to _GUILocalizationWindowClassMethods script to the method onControlBuild()

Code: [Select]
method onControlBuild()
 
  if $LOCALIZE._guiLocalizationWindow != None
    DestroyNode(me)
    return
  .
 
  //_RepopulateLocalizationTableRows()
  $LOCALIZE._LOC_SetLocalizationWindow(me)
 
  if not $LOCALIZER.LocStarted
  $LOCALIZE._LOC_InitializeClientLocalizationSystem()
  .
 
  //$LOCALIZE._LOC_GetLocalizationTableViewFilter()._LocalizationSetTableViewFilterBufferPosition(-1)
 
  //request that a view and filter be created (server call)
  $LOCALIZE._LOC_RequestReplicateTableEntriesToClient()
 
.

Yes, I know this is clean engine script. But this changes necessary.
So, now, it's work fine.  :)
Logged

FI-ScottZ

  • General Accounts
  • *
  • Posts: 1407
    • View Profile
    • Forever Interactive, Inc.
Re: [Fixed]Localization Questions
« Reply #9 on: Jun 04, 15, 12:15:10 AM »

I found a new bug that was introduced in the script change to server script "_LocalizationClassMethods" on April 21, 2015:

That change added the ENABLE_AUTHORIZATIONCHECKS definition to check if a user is authorized to work with the strings. However, in a few of the methods called from the client, they will check if they are in the localization area and, if not, recursively call themselves in that area.  The problem arises because the methods check SYSTEM.REMOTE.CLIENT at the beginning of the function, which works fine when called by the client, but when called server-to-server that value is empty, thus failing the repeated authorization check.  It can be fixed by only checking the remote client value when not in the localization area (assuming clients will never be calling it when in that area).

This affects these methods:
_LOC_SetString
_LOC_SetContext
_LOC_CreateNewString
_LOC_RequestReplicateTableEntriesToClient
_LOC_RemoveLocalizationTableViewFromManager
_LOC_UpdateLocalizationTableViewForClient

In our implementation, I also made _LOC_RemoveString be untrusted so the client could remove strings, as well, and so I put the authorization check there, too.

The old code:
Quote
  #if ENABLE_AUTHORIZATIONCHECKS
  if not ( IsPlayerUsingEditClient( SYSTEM.REMOTE.CLIENT ) and _GetAuthorizationValue( SYSTEM.REMOTE.CLIENT, "__IsAllowedToModifyGOM" ))
    return
  .
  #endif

 
  if $LOCALIZATION._LOC_CalledFromAnotherScript()
    #if DEBUG_LOCALIZATION
    $LOCALIZATION.Debug( "This is an internal method and is not callable externally." )
    #endif
    return
  else if not $LOCALIZATION._LOC_InLocalizationSystemAreaEditInstance()
   
    call area $LOCALIZATION._LOC_GetLocalizationSystemAreaID() instance 0 $LOCALIZATION._LOC_SetString( request_id, string_num, locale, new_value )
    return
  .

becomes this:
Quote
  if $LOCALIZATION._LOC_CalledFromAnotherScript()
    #if DEBUG_LOCALIZATION
    $LOCALIZATION.Debug( "This is an internal method and is not callable externally." )
    #endif
    return
  else if not $LOCALIZATION._LOC_InLocalizationSystemAreaEditInstance()
    #if ENABLE_AUTHORIZATIONCHECKS
    if not ( IsPlayerUsingEditClient( SYSTEM.REMOTE.CLIENT ) and _GetAuthorizationValue( SYSTEM.REMOTE.CLIENT, "__IsAllowedToModifyGOM" ))
      #if DEBUG_LOCALIZATION
      $LOCALIZATION.Debug( "Client "+SYSTEM.REMOTE.CLIENT+" is not an authorized user in HeroBlade." )
      #endif
      return
    .
    #endif

    call area $LOCALIZATION._LOC_GetLocalizationSystemAreaID() instance 0 $LOCALIZATION._LOC_SetString( request_id, string_num, locale, new_value )
    return
  .
« Last Edit: Jul 08, 16, 10:50:30 AM by FI-ScottZ »
Logged
Scott Zarnke
Lead Programmer, Visions of Zosimos
CTO, Forever Interactive, Inc.