bdcribbs
Level: Smiter
Rank Points: 390
Registered: 04-08-2003
IP: Logged

Has any work been done on writing automated tests for DROD code?

I know in the middle of a beta no one is likely to delve into this, but I just started poking around the code shortly before the beta testing started, and I was thinking maybe writing unit tests as a way to get my head around the code.

Unit testing basically means faking an environment for the smallest testable unit (In an OO design the object) and programmatically running it through it's paces, testing that it behaves as expected.

Proven unit tests can also serve as regression tests (test that new features haven't broken existing features).

Has any work along these lines been done?

-Bryan


[Edited by bdcribbs on 06-27-2003 at 08:01 AM GMT: just added email notification]

ErikH2000
Level: Legendary Smitemaster

Rank Points: 2794
Registered: 02-04-2003
IP: Logged

bdcribbs wrote:
Has any work been done on writing automated tests for DROD code?

Yes, but not enough, and please help us if you have the time.

There is a program called DRODUTIL and you can use it to run demos without the UI. A command something like this:

c:> DRODUTIL TEST -c19245974432 23

...runs demo#23 and compares the ending state with the original ending state that was saved with the demo as a checksum. If the demo no longer produces the same ending state, there is an error. Actually, this command was temporarily removed in the last version of DRODUTIL, but the code is still available.

Another good area of unit-testing would be importing and exporting. We could inventory an object like a hold or player. Export, delete, and reimport the object. Then check the imported object against the previously made inventory.

The project is organized such that it is easy to link in the game logic and data access functionality into an executable without the game UI. Unit-testing possibilities are endless, and would show rewarding results. The only problem is getting enough time to work on it.

So by all means, if you've got the inclination, let's start up a DRODTEST project and give it to you. I would provide as much help as you need to make it work. Believe me, you'd be my new hero, since I've been wanting an automated testing suite for a long time now.

-Erik

____________________________
The Godkiller - Chapter 1 available now on Steam. It's a DROD-like puzzle adventure game.
dev journals | twitch stream | youtube archive (NSFW)

bdcribbs
Level: Smiter
Rank Points: 390
Registered: 04-08-2003
IP: Logged

ErikH2000 wrote:Yes, but not enough, and please help us if you have the time.

I'm definitely interested. I didn't have any time away from working at work today
Thanks for the tips, I'd disabled the Util project and never looked in it, I'll look for that.
When I've got something to show or something to ask, I'll followup here.

-Bryan

ErikH2000
Level: Legendary Smitemaster

Rank Points: 2794
Registered: 02-04-2003
IP: Logged

bdcribbs wrote:
I'm definitely interested. I didn't have any time away from working at work today
Thanks for the tips, I'd disabled the Util project and never looked in it, I'll look for that.
When I've got something to show or something to ask, I'll followup here.

Cool, cool, cool. Note that the demo testing code for the DRODUtil project can be found in the "Build_14"-tagged revision of util1_5.cpp.

-Erik

____________________________
The Godkiller - Chapter 1 available now on Steam. It's a DROD-like puzzle adventure game.
dev journals | twitch stream | youtube archive (NSFW)

bdcribbs
Level: Smiter
Rank Points: 390
Registered: 04-08-2003
IP: Logged

ErikH2000 wrote:
Note that the demo testing code for the DRODUtil project can be found in the "Build_14"-tagged revision of util1_5.cpp.

Errr... cvs [server aborted]: no such tag Build_14

I can proceed without it, when it's convienent for you, you can send me a copy. thanks.


[Edited by bdcribbs on 06-29-2003 at 08:59 AM]

ErikH2000
Level: Legendary Smitemaster

Rank Points: 2794
Registered: 02-04-2003
IP: Logged

bdcribbs wrote:
Errr... cvs [server aborted]: no such tag Build_14

Okay, see PrintTestOneDemo() in this file:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/drod/Caravel/DRODUtil/Util1_5.cpp?rev=1.72&content-type=text/vnd.viewcvs-markup

It's a function that checks if a demo still produces the same result it did when it was recorded. Things that are checked are position of swordsman, whether room gets conquered, and the number of monsters left int the room. If these things don't match up, then game behavior has changed and we probably need to do something about it.

-Erik

____________________________
The Godkiller - Chapter 1 available now on Steam. It's a DROD-like puzzle adventure game.
dev journals | twitch stream | youtube archive (NSFW)

bdcribbs
Level: Smiter
Rank Points: 390
Registered: 04-08-2003
IP: Logged

I've been focusing on playtesting (which seems more useful), but I have been trying to make headway on unit testing and have been focusing on the export/import code.

The more I get into it, the easier it is, but it does have a steep learning curve, this is a complicated project. I'm taking the approach of testing the XML, assuming the encoding and decoding of the dat files is stable (is it?).

It looks like the import is dependent entirely on the validity of the export?

And the exports structure should be something like:
Holds:
-> Players
-> Levels
-> Demos
Levels:
-> Players
-> Rooms
Players:
-> Demos
-> Saved Games
Rooms:
-> Levels
Saved Games:
-> Players
-> Rooms


Is comparing the XMl a reasonable test? Or is the DOM changing?

-Bryan

mrimer
Level: Legendary Smitemaster

Rank Points: 5056
Registered: 02-04-2003
IP: Logged

This week I just checked in a bunch of changes to the import/export, making the import (hopefully) much more robust. The export xml must be correct for the import to work, but since we're the only one generating it, we assume it's correct. (If you happen to see any mistakes, let me know!) It might change in the future from time to time, as new data gets added to the db.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

mrimer
Level: Legendary Smitemaster

Rank Points: 5056
Registered: 02-04-2003
IP: Logged

This week I just checked in a bunch of changes to the import/export, making the import (hopefully) much more robust. The export xml must be correct for the import to work, but since we're the only one generating it, we assume it's correct. (If you happen to see any mistakes, let me know!) It might change in the future from time to time, as new data gets added to the db. Did this answer some of your questions?

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.