HeroEngine Forums
Welcome, Guest. Please login or Register for HeroCloud Account.

Author Topic: Logging system  (Read 2352 times)

zarmin

  • World Owners
  • ****
  • Posts: 22
    • View Profile
Logging system
« on: Jul 23, 13, 11:23:54 AM »

Hello!

How I can implement an event logging system (i'd like to use it from HSL)? I known I can define and subscribe to debug channels, but I need some persistent, searchable database to save the messages.

Thanks!
Logged

FI-ScottZ

  • General Accounts
  • *
  • Posts: 1407
    • View Profile
    • Forever Interactive, Inc.
Re: Logging system
« Reply #1 on: Jul 23, 13, 05:39:41 PM »

If you haven't already, you'll want to peruse this page:
Data Storage options
Logged
Scott Zarnke
Lead Programmer, Visions of Zosimos
CTO, Forever Interactive, Inc.

zarmin

  • World Owners
  • ****
  • Posts: 22
    • View Profile
Re: Logging system
« Reply #2 on: Jul 25, 13, 11:12:44 AM »

Yes I've checked that, but it does not provide a data storage option that's suitable for logging.
Logged

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: Logging system
« Reply #3 on: Jul 25, 13, 11:50:26 AM »

You could prolly do it with an arn and multiple other options, but i would most likely do it like this:

-create an area, keep the edit instance up at all times
-for each event create an event node (which would most likely contain timestamp and name) and hard associate event node to the edit instance area root node.

and there you have it, very simple
Logged

FI-ScottZ

  • General Accounts
  • *
  • Posts: 1407
    • View Profile
    • Forever Interactive, Inc.
Re: Logging system
« Reply #4 on: Jul 25, 13, 12:11:35 PM »

Sounds like that could be creating an awful large number of nodes.  But one persisted node that keeps the data would be a way to go.

You can also write to a text file(s) stored in the repository, then that can be downloaded via the repo browser to have an offline copy.
Logged
Scott Zarnke
Lead Programmer, Visions of Zosimos
CTO, Forever Interactive, Inc.

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: Logging system
« Reply #5 on: Jul 25, 13, 12:27:17 PM »

I belive that you cannot prove me that one node would be significantely more efficient then awful large number of nodes ;)   or ?

We'd probably need Herb or Christopher for that.   8)
Logged

keeperofstars

  • General Accounts
  • *
  • Posts: 998
    • View Profile
    • StarKeeper Online
Re: Logging system
« Reply #6 on: Jul 25, 13, 02:41:24 PM »

Well for one I wouldn't make it an "area" node, but a system area to begin with, that way it can be global.

I would probably build it in a similar fashion as the chat system, but add a persistent aspect for the log files.
If you are wanting to do different options of querying, then possibly use lists, or lookup lists.

That way you can have key(s) to do the lookup for. So if you wanted to see the log files for the player. You would look through the look-up list for the playerNodeID, and find all the logs that have that id, and pull them into whatever format you needed. Since a look-up list can hold any other type of DOM data, there is lots of options for storage there.

Depending on the log you could store it to the player node, for player related data (could get deleted if the player deletes the character). Or to a system area, or a child node of a system area, that is specific to each player to help with querying.
Could have multiple system areas, keeping track of different things, so a combat log one, a npc interaction, a long term chat log, bank logs, etc.
Logged
[img]http://screencast.com/t/x7btcSSyp3h0[\img]

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: Logging system
« Reply #7 on: Jul 25, 13, 02:51:09 PM »

Well for one I wouldn't make it an "area" node, but a system area to begin with, that way it can be global.

From my expirience, in this particular case making it a system area would be unnecessary. Can you explain what do you actually mean by "that way it can be global"?

You can make a remote call from one area to another, regardless of it being a system area or not.
« Last Edit: Jul 25, 13, 02:53:57 PM by AWW_boss »
Logged

FI-ScottZ

  • General Accounts
  • *
  • Posts: 1407
    • View Profile
    • Forever Interactive, Inc.
Re: Logging system
« Reply #8 on: Jul 25, 13, 08:31:13 PM »

True, "system area" can mean different things.  I think in the broadest sense it is simply an area which is not for playing in  but performs some sort of service or services.  So for that, you don't really need to "set" anything up.

But often they register for World or Player info.

You might need that, depending on what you are logging.  And then it is typically made to have at least one instance be always up, so the services are available 24/7 without needing to spin an instance up.
Logged
Scott Zarnke
Lead Programmer, Visions of Zosimos
CTO, Forever Interactive, Inc.

keeperofstars

  • General Accounts
  • *
  • Posts: 998
    • View Profile
    • StarKeeper Online
Re: Logging system
« Reply #9 on: Jul 25, 13, 11:48:03 PM »

True, "system area" can mean different things.  I think in the broadest sense it is simply an area which is not for playing in  but performs some sort of service or services.  So for that, you don't really need to "set" anything up.

But often they register for World or Player info.

You might need that, depending on what you are logging.  And then it is typically made to have at least one instance be always up, so the services are available 24/7 without needing to spin an instance up.

Right, system areas can register with world and player information, regardless of where either are. So if you are looking to get player info you can spin up just the one system area and have access to data in all the areas and all of the players in every area or instance. If you use remote calls from an area, you have to call the world area to get the data from other areas, and that puts extra load on the world area.

Remember standard areas only know what is in them and that is it. so if you need to preform or gather information that is a bit global in nature, a system area is the way to go. System areas aren't actual physical areas but more like system process nodes. So basically it's nothing more than a script that has it's own node, that can reference data from world.
Logged
[img]http://screencast.com/t/x7btcSSyp3h0[\img]

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: Logging system
« Reply #10 on: Jul 26, 13, 02:07:59 AM »

Quote
True, "system area" can mean different things.  I think in the broadest sense it is simply an area which is not for playing in  but performs some sort of service or services.  So for that, you don't really need to "set" anything up.

But often they register for World or Player info.

You might need that, depending on what you are logging.  And then it is typically made to have at least one instance be always up, so the services are available 24/7 without needing to spin an instance up.

Well, i always considered area to be a "system area" only if you register it for World or Player info, but i see your point and i agree. :)
Logged

zarmin

  • World Owners
  • ****
  • Posts: 22
    • View Profile
Re: Logging system
« Reply #11 on: Jul 26, 13, 04:15:07 AM »

I think the HE does not provide a good solution for logging.

My idea is, to write an external server-side module (DLL) to handle the logs, and after that I can use an external logging system to the HE. The biggest disadvantage of that solution, that you need to own the source code license and running your own environment...

Logged

keeperofstars

  • General Accounts
  • *
  • Posts: 998
    • View Profile
    • StarKeeper Online
Re: Logging system
« Reply #12 on: Jul 26, 13, 09:39:00 AM »

Hero has everything you would need to build a complex logging system, just need to take the design and think on it.

If you can provide a bit more detail on what your are wanting to log, I (we) can probably brain storm a good system.

Also you can write custom .dll files for the client side. HSEI allows you to do that and when you move to your own world servers I believe you can do .dll ones for the server side.

Once again depends on what you are wanting to do. If it's keep track of client data, could write a custom .dll have it grab the client information and preform the log to another system if you wanted.
Logged
[img]http://screencast.com/t/x7btcSSyp3h0[\img]

MarlonB

  • General Accounts
  • *
  • Posts: 38
    • View Profile
Re: Logging system
« Reply #13 on: Aug 09, 13, 01:55:17 PM »

You might want to take a look at the character registry system from HJ.

It stores player data in a system area. I extended this system to also cater for game-area's and hold logging for players and areas. Around it I made an api to log to characterRegistry, or read from it.
This way you also have information for offline players available.


It's far from ideal, but does the job if you don't own the source licence.


Logged
Cerberus of The Repopulation.

keeperofstars

  • General Accounts
  • *
  • Posts: 998
    • View Profile
    • StarKeeper Online
Re: Logging system
« Reply #14 on: Aug 09, 13, 03:36:29 PM »

But if it's an always spun up system area that is holding the data, shouldn't need to off load it to retrieve it, just anchor the data you want to the system area.
Be it a player's data, an area, combat, etc.

The bigger question abounds though,

VOLUME even the smallest log file can explode into nightmare, when you look at lets say 1 million users generating 1 million logs a second, even at 1kb per log that is what 119 MB per second.

So if you are long term / temp term storying make sure you know why and the reasons behind it.
Probably best to only log oddities detected by the system. Then to log a ton and look through it.
Logged
[img]http://screencast.com/t/x7btcSSyp3h0[\img]