HeroEngine Forums
Welcome, Guest. Please login or Register for HeroCloud Account.

Author Topic: [Resolved| automated testing?  (Read 5201 times)

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
[Resolved| automated testing?
« on: Mar 04, 12, 01:41:49 PM »

Hi!

I'm looking for a possibility to automate the testing of some of my server side implementations.
What i would like to have is java/junit, tcl/expect or similar framework connected to the chat console of multiple HeroEngine clients.
I would then send commands from a framework to the chat window, and also get feedback messages from chat window back to the framework.

Has anyone done something similar and is it possible?

Kind regards
« Last Edit: Oct 21, 12, 10:41:03 PM by HE-Cooper »
Logged

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: automated testing?
« Reply #1 on: Mar 06, 12, 04:58:41 PM »

Hi again,

I am shamelessly bumping this thread with several additional questions:
1. If there is no possibility to connect to the chat console of multiple HeroEngine clients, will you pretty pretty please consider implementing such a feature?
2. What are my other options for test automation and regression testing(except hiring a lot of testers)?
3. If i haven't been clear enough on what i am trying to achieve here, please say and i'll provide an example.

Can you please answer, since it's such a critical task in my project.

King regards
Logged

HE-HERB

  • HeroEngine
  • *****
  • Posts: 530
    • View Profile
    • HeroEngine
Re: automated testing?
« Reply #2 on: Mar 06, 12, 05:16:43 PM »

Have you reviewed the HeroScript Extension info on the wiki?

Cheers
Logged
herb marselas
graphics guy

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: automated testing?
« Reply #3 on: Mar 07, 12, 12:45:25 AM »

Hi Herb,

thanks for the answer. I started reading about plugins and will get back to you if i run into an obstacle.
Logged

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: automated testing?
« Reply #4 on: Mar 07, 12, 10:48:42 AM »

Hi!

I have reviewed the HeroScript Extension info, but i didn't completely managed to wrap my head around it.
I guess i will have to try out the demo plugin.

But, before i start my endeavour to learn c++ (of which i have only basic knowledge), for the love of god and everything that is holy, can you please answer - from your expirience - is there even a remote possibility to do this with the plugin framework? Do you have any hints on how would you do it?

(as a reminder: what i am trying to get is some kind of an interface between an external application and heroblade clients chat window for test automation purposes)
Logged

HE-Cooper

  • *****
  • Posts: 2221
    • View Profile
Re: automated testing?
« Reply #5 on: Mar 07, 12, 11:30:14 PM »

Boss, you're going to need engineers on your team, so I would ask them, but Herb pointed you to extension interface for a reason. So yes, it's possible, but we don't know what game you're making or what game systems you have, etc.
Logged

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: automated testing?
« Reply #6 on: Mar 08, 12, 02:48:58 AM »

Hi Cooper,
Thank you for your answer and sorry for being vague about my game system (it's more of a testing framework than a game system). Now you convinced me that this can be done with the extension plugin and i will try to do it.

Just for the reference, i will try to explain it on an example, if someone from the community ever wishes to use something similar i'll be glad to hook em up.

HeroEngine by default comes with the cmdTell script which exposes the /tell command. So let's say i want to test if this command works as intended.
I would have to:
1. open one client and log with the character named Alice
2. open second client and log with the character named Bob
3. with Alice i would type "/tell Bob hello" in hers chat window
4. with Bob i would check if i get a "hello" message from Alice in his chat window
5. test passed

What i'm trying to achieve here is have an automated way of doing this.


Code: [Select]

        interface exposed via extension plugin
                |                   
                |                   
 ________       |      ______________
|         |     |     |  alice's chat|
|         |-----|-----|  window in a |
|   my    |     |     |__client_1____|
| testing |                           
|framework|                           
|         |            ______________
|         |     |     |  bob's chat  |
|         |-----|-----|  window in a |
|________ |     |     |__client_2____|
                |                     
                |                     
                |                     
        interface exposed via extension plugin




       
SOOOOooo, instead of me typing all these commands and looking for results, i would just sit back and watch my framework opens up clients, sends commands, and check if the commands result with expected data.
This would be great for regression testing, stress testing etc.


Curently as a testing framework i am using ExpectJ which is free and open source, but i am also planning to hook it up with a very sofisticated model based testing framework.
Currenly the only missing link is the interface, which i am planning to do with the extension system.
 
« Last Edit: Mar 08, 12, 10:35:01 AM by AWW_boss »
Logged

HE-HERB

  • HeroEngine
  • *****
  • Posts: 530
    • View Profile
    • HeroEngine
Re: automated testing?
« Reply #7 on: Mar 08, 12, 09:26:29 AM »

Since HeroBlade itself is written in C#, and the chat and console windows therefore are, you can probably use C# automation to do automatic GUI or program driven test automation through those

Cheers

Logged
herb marselas
graphics guy

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: automated testing?
« Reply #8 on: Mar 08, 12, 10:28:33 AM »

Since HeroBlade itself is written in C#, and the chat and console windows therefore are, you can probably use C# automation to do automatic GUI or program driven test automation through those

Cheers

Hi!

This post confused me a bit, but sounds interesting. Care to elaborate? Did you mean if i had source license, that i could modify the HeroBlade client code to do what i want to, or you ment something else?

Kind regards
Logged

HE-HERB

  • HeroEngine
  • *****
  • Posts: 530
    • View Profile
    • HeroEngine
Re: automated testing?
« Reply #9 on: Mar 08, 12, 11:56:54 AM »

No, this does not require source

Go research C# automation on the web

Cheers
Logged
herb marselas
graphics guy

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: automated testing?
« Reply #10 on: Mar 12, 12, 03:46:47 AM »

Hi! If anyone using these forums has any expirience with automation of the HeroBlade client please contact me.

Specifically, i am looking for a C# class which represents the HeroBlade Client (if one exists) in manner similar to Word or Excel automation:

//word
Word.Application oWord = new Word.Application();

//excel
Excel.Application oExcel = new Excel.Application();

« Last Edit: Mar 12, 12, 06:08:15 AM by AWW_boss »
Logged

RicoTheMad

  • General Accounts
  • *
  • Posts: 40
  • Project Lead/Software Developer
    • View Profile
Re: automated testing?
« Reply #11 on: Mar 13, 12, 08:50:20 AM »

I did not have a chance to reply to you before you left IRC but this is honestly what I'd recommend:

1) This is something you may want to tackle after you've written your game scripts. While automated testing would be a good idea to do before release, I don't see how a new project would benefit much from tackling it this early on rather than working on core systems.

2) The HE folks pointed you in the right direction. I would be surprised if you found some pre-written framework or library on the web that specifically does what you want to do, let alone specifically for HeroEngine. You can hook into the engine using the resources they mentioned. That means you can write whatever testing framework you want, but you'll have to do it yourself.


Testing is good, but you may be jumping the gun here if your project is still in the early stages. If you're unfamiliar with C++ or C#, then you will not be able to do this yourself unless you learn them.
Logged

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: automated testing?
« Reply #12 on: Mar 13, 12, 12:33:36 PM »

The problem is that i got hooked on an idea and i just can't let it out of my mind easily. I sincerely hope the moderators won't ban me for agressive posting on this thread.

Anyways, i've made some progress in last several days.

1. downloaded visual C# 2010 express
2. created a console application project
3. in project References added HeroBlade.exe from heroblade client folder
4. added "using HeroBlade" namespace;
5. in main function wrote the following function call:
Console.WriteLine(HeroBlade.Heroic.UnsafeNativeMethods.GetHeroBladeDeployVersion());
6. compiled and copied the console application exe to heroblade client folder
7. run application and it wrote the number 14
8. danced a monkey jig


then i added a following line:
HeroBlade.Heroic.UnsafeNativeMethods.HeShutdown();
and it crashed my app with the EXCEPTION_ACCESS_VIOLATION.

Then i cried. It's kinda tough to work with only function signatures, a few lines of example code wouldn't hurt (winkwinknudgenudge), like how to use HeStartup and HeShutdown properly.
 
Logged

HE-HERB

  • HeroEngine
  • *****
  • Posts: 530
    • View Profile
    • HeroEngine
Re: automated testing?
« Reply #13 on: Mar 13, 12, 09:20:07 PM »

You're just thrashing around, and you're going to be frustrated.

I'll put it on the list of things to document, but no promises for timeframe.

Cheers

Logged
herb marselas
graphics guy

AWW_boss

  • General Accounts
  • *
  • Posts: 846
  • gdfgd
    • View Profile
Re: automated testing?
« Reply #14 on: Mar 14, 12, 02:41:24 AM »

Yes! Go Herb! I'm sending you my love.  :-*
Logged