schep
Level: Smitemaster

Rank Points: 865
Registered: 03-01-2005
IP: Logged

The DROD Unconquer Tool is a program that may help DROD players deal with the sorts of bugs that make the game act as though you've explored and/or conquered some rooms which you haven't. Using this tool might help players fix incorrect Restore screen stats, fix Hold Mastered statuses, and find conquered rooms where the player is missing a demo.

Before I release this creation over on General, I'd like to find out whether it works for various people other than me. Would you like to be a beta tester?

Qualifications:
* You MUST know how to make a backup of all your DROD data files.
* You may have either the full TCB, the TCB demo, or both installed. You can even have multiple versions of TCB installed, if you're a developer sort or something.
* Ideally, you've been playing DROD TCB (3.*) for quite a while, and have multiple holds installed.

Downloads:
Source code: unconquer-src-0.1.tar.gz
Windows executable: unconquer-win-0.1.zip

The Unconquer tool is licensed to all downloaders under GPL 3.0 or later. The source code requires Qt 4.4 or 4.5 and Metakit to compile.

Wanted: Some trustworthy forumite with MacOS X know-how, to compile a MacOS X version easy for casual users to just download and run.

Although this program attempts to prevent simultaneous access from DROD and from the tool, and opens all DROD data files in read-only mode except for the final Save button, this is beta software. So make a backup of your data before you try it out.

I hope the tool makes enough sense that I don't need to explain too much.

At one point, you will probably come across a list of rooms which DROD can't tell whether it should unconquer or not. These rooms can come up for three reasons:
1. You did conquer the room, but the option to automatically save victory demos was disabled at the time.
2. You did conquer the room, and you wanted automatic victory demos, but due to some DROD bug, a victory demo was not saved.
3. You found (explored) the room, but never did conquer it. A DROD bug made a mistake and marked it as conquered.

If you do try out the tool, please report back, either on this thread or in PM to me:
1. What Operating System are you using?
2. Do you have the full TCB, the TCB demo, or something more complicated?
3. Did you encounter any bugs?
4. Were there any times when you were rather confused about what the tool was asking or what its effects would be?
5. When you first start up the Unconquer tool, does it select reasonable directories for the two inputs on the initial page? If not, do you have DROD installed in a "default" location for your OS?
6. If DROD is open when you press Next from the first page, does the tool ask you to close DROD? (If not, close the Unconquer tool immediately to reduce risk of data corruption.)
7. If you attempt to open DROD while on the page asking you to select a Player, does DROD refuse to start up normally? (If not, close the Unconquer tool immediately to reduce risk of data corruption.)
8. Any changes or additions you think would make the Unconquer tool better? No promises here, but if I agree something would be good and it wouldn't be too hard, I may do it.

Thanks!
-- schep

---

[Last edited by schep at 04-07-2009 01:21 AM : email notification on]

RoboBob3000
Level: Smitemaster

Rank Points: 1980
Registered: 10-23-2003
IP: Logged

Continuing the trend of welcome returns! Good to see you again, schep!

____________________________
http://beepsandbloops.wordpress.com/

Samuel
Level: Delver

Rank Points: 69
Registered: 07-08-2007
IP: Logged

How do you back up all your DROD data files?

____________________________
I believe in everything until it's
disproved. So I believe in fairies,
the myths, dragons. It all exists,
even if it's in your mind. Who's to
say that dreams and nightmares
aren't as real as the here and now?
John Lennon

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

Samuel wrote:
How do you back up all your DROD data files?

Copy or ZIP all the files with extension ".dat" in the "Data" folder beneath the directory you installed DROD to - except for drod3_0.dat, which only contains the game's graphics and sounds and can be fixed by simply re-installing the game.

Store those copies or that ZIP file on an external drive for extra safety, but unless something shreds your hard disc a place in some other folder should also be safe for the purpose of testing this tool. Just don't forget where you put those copies, and don't mess with them.

In case something goes horribly wrong in DROD copying or extracting the .dat-files back or (in the worst case) re-installing DROD then copying them back should fix things - just don't forget that all progress between when you copied the files and now will obviously be lost, so making regular copies and keeping more than one version to be safe surely isn't a bad idea...

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

Samuel
Level: Delver

Rank Points: 69
Registered: 07-08-2007
IP: Logged

Well then, it sounds like i've got what it takes to be a beta tester.(in my opinion)

____________________________
I believe in everything until it's
disproved. So I believe in fairies,
the myths, dragons. It all exists,
even if it's in your mind. Who's to
say that dreams and nightmares
aren't as real as the here and now?
John Lennon

Syntax
Level: Smitemaster
Rank Points: 1218
Registered: 05-12-2005
IP: Logged

I finally got Qt 4.5 and the latest Metakit installed on Ubuntu Ibex. The former was easy enough (apt-get) but Metakit took me a while as I haven't installed from source for a rather long time.

Anyways...

schep wrote:
If you do try out the tool, please report back, either on this thread or in PM to me:
1. What Operating System are you using?

Linux Ubuntu Ibex 2.6.27-11-generic

2. Do you have the full TCB, the TCB demo, or something more complicated?

Full TCB with 2 player files. Can't wait to test on my XP as all my progress was attributed to the 2nd player account. This one is pretty clean though.

3. Did you encounter any bugs?

None

4. Were there any times when you were rather confused about what the tool was asking or what its effects would be?

"The rooms below are marked as unconquered in the database, but look suspiciously like unconquered rooms."
->
"Erm... is the database on my end, or Caravel's end? And who's suspicious anyway... me or Caravel? Because if I have done them, I'd rather not lose my progress..."

5. When you first start up the Unconquer tool, does it select reasonable directories for the two inputs on the initial page?

No problems there... both players recognised too (starting by checking my own player file)

6 & 7

I'll test this later - though it did correctly ask me to shut down drod first when I clicked "Reexamine db".

8. Any changes or additions you think would make the Unconquer tool better? No promises here, but if I agree something would be good and it wouldn't be too hard, I may do it.

Music?

--

Anyways, here's my report on checking various rooms. I'll skip reporting any duplicate types and certainly won't individually check them all (I have about 100).


--

I'll happily test this further but so far I guess I'm confused about the purpose of "unconquering" rooms. I'm all for wiping scores for rooms which have since been marked un-highscorable, but don't want to lose valid points for rooms I've completed.

Having said that, I'm impressed by the analysis unconquer achieves and its ease of use. If this gets fully tested and can reliably remove invalid demos, I would love to see it running server side.

Cheers,

Peter

[EDIT]

I realise I'm making a cross-over between progress and demos, but surely this kind of analysis could aid in both...

---

[Last edited by Syntax at 04-07-2009 11:40 PM]

schep
Level: Smitemaster

Rank Points: 865
Registered: 03-01-2005
IP: Logged

Okay, so I probably need to add something to the tool to explain exactly what checking rooms in the unconquer list does.

Your demos and saved games are never modified at all, and using the tool will have no effect on your CaravelNet high scores. Also, whether a room can or should be unconquered is not related to whether it is marked as "No high scores" on CaravelNet.

The only effects "unconquering" a room will have are:
1. If you encountered the "Master Walls drop too soon" bug, and you unconquer any secret room in that hold, Master Walls in that hold will be up again. In addition, your "Secrets conquered" stats on the Restore screen will be corrected.
2. If you unconquer one or more rooms and then manually re-upload scores to CaravelNet: On your High Scores page here on the forum, under "Explored but Unconquered Holds", the "Rooms Conquered" percentage might be corrected.
3. Danforth's estimation of your skill might drop slightly, if he still has any interesting conversations with you.

Syntax mentioned some examples of things which happened which are, I think, typical:

1. The Unconquer tool listed a number of unconquerable rooms. At some point in your DROD history, some sort of bug marked these rooms as conquered, somewhat randomly. I would check and unconquer these.

2. The tool noted a room which was conquered but was missing a victory demo. On visiting the room again, DROD created a victory demo, and then on pressing the "Reexamine DB" button, the tool removed that room from the list of suspects.

This is exactly why the tool provides the ability at this point to open DROD, play a bit, close DROD, and press that button.

Not so typical:
A hold by any other name : The Crypt of the Lazy : Entrance

I get the feeling I also came across something like this, and I had something which appeared to be a victory demo but had issues. It was related to a known bug in older versions of DROD which came up in level entrances and/or demos which exit via a staircase. If you start a new game in that hold, go to that room and exit (which is easy), do you get a new victory demo? Does the room disappear from the Unconquer tool?

TCB : Estengard
These rooms may be marked as no-high-scores, but they are still conquerable in the sense this tool cares about. I have no idea why these rooms would show up in your list, since all saved games from TCB after Estengard should record that you did conquer them. Do you have victory demos there?

CuriousShyRabbit
Level: Smitemaster
Rank Points: 3170
Registered: 10-14-2006
IP: Logged

I just did something horrible that caused a bug in DROD to "unconquer" a whole load of secret rooms that I have conquered and have created victory demos for. Does this new tool work the other way too? That is, can I use it to reconquer all those secret rooms without going back and replaying them all? If so, I'll give it a try...

schep
Level: Smitemaster

Rank Points: 865
Registered: 03-01-2005
IP: Logged

CuriousShyRabbit wrote:
I just did something horrible that caused a bug in DROD to "unconquer" a whole load of secret rooms that I have conquered and have created victory demos for. Does this new tool work the other way too? That is, can I use it to reconquer all those secret rooms without going back and replaying them all?

Nope, it wasn't really designed for that, but only because I didn't expect anyone to have "missing" conquered statuses. Is this bug understood, and has it affected or is it likely to affect anyone else? It shouldn't be too hard to add in some automatic reconquer logic.

CuriousShyRabbit
Level: Smitemaster
Rank Points: 3170
Registered: 10-14-2006
IP: Logged

schep wrote:

CuriousShyRabbit wrote:
I just did something horrible that caused a bug in DROD to "unconquer" a whole load of secret rooms that I have conquered and have created victory demos for. Does this new tool work the other way too? That is, can I use it to reconquer all those secret rooms without going back and replaying them all?

Nope, it wasn't really designed for that, but only because I didn't expect anyone to have "missing" conquered statuses. Is this bug understood, and has it affected or is it likely to affect anyone else? It shouldn't be too hard to add in some automatic reconquer logic.

What I did was: export my player, move my old .dats so DROD would start again with new ones, reimport all the holds I had before, and reimport my player. This unconquers the last secret room conquered in a bunch of the holds, as reported by Remlin here. If anyone understands this bug, it would be you, because you helped me with this one, where importing a new version of a hold unconquered the last secret room.

I'd say, yes, it's likely to affect a few other people, who, for whatever reason, find themselves starting over with nothing but a player file and a clean install of DROD. But this situation would not be as ubiquitous as the falsely conquered secret rooms in everyone's .dats. In my case, since my little operation didn't end up accomplishing its goal, I'll just revert to my backup .dats. I didn't want to miss the opportunity to test your tool first...

CuriousShyRabbit
Level: Smitemaster
Rank Points: 3170
Registered: 10-14-2006
IP: Logged

I see you fixed the bug where secrets become unconquered on player import. Thanks a whole lot!

I reverted to my old .dats, and tried out your unconquer tool on them. I'm a completist, and so without many unconquered rooms, I'm not a very good test case for this tool. I know I have exactly one falsely conquered secret room in my reverted .dats, El Dorado : Finale : 1S1E.

The unconquer tool gave me a list of five rooms to be unconquered, which did not include El Dorado : Finale : 1S1E. Perhaps this is because I have BoyBlue's demo for this room? Three of the rooms it did list were in WIPs where I had arrived at a level entrance in an empty room with no exits. The game marks those as conquered right away, but never saves a victory demo because I couldn't exit. The other two rooms were in beta holds that I had conquered in early versions but not yet in the current version of the hold. The fact that updated rooms stay conquered is not a random error in DROD, it's deliberate (or at least it happens that way every single time).

Then the unconquer tool gave me a list of "unexplored secrets". There was a checkbox for displaying room coordinates, but checking it or unchecking it made no difference. It showed me room coordinates each time. I did not understand what this list meant. For example, for TCB, it listed Holding Vats 1E, 1S1E, 1S2E, 2S1E, and 2S2E. These rooms do not exist. [EDIT: Scratch that, I just went back and saw a cutscene I'd missed before!] It also listed a couple rooms in WIPs that I had not visited because they are not accessible in game. I checked in the editor and found that neither of these rooms was marked secret. [EDIT 2: So is the reason I saw the coordinates for all these rooms because none of them are secret? In that case, it's just the heading for this list of rooms that confused me.]

So I didn't elect to save any changes to my .dats, but since trying this didn't break anything, I'd be happy to try it again.

---

[Last edited by CuriousShyRabbit at 04-09-2009 08:24 AM]

schep
Level: Smitemaster

Rank Points: 865
Registered: 03-01-2005
IP: Logged

Yeah, the "Show coordinates of unexplored secrets" checkbox isn't meant to be a label for the results box, but I can see it sort of would look like that. I'll add an actual label like "Changes to be made" or something, and move that checkbox below the list. I think I'll also disable it when there are no "unexplored secrets" for it to affect.

CuriousShyRabbit wrote:
The unconquer tool gave me a list of five rooms to be unconquered, which did not include El Dorado : Finale : 1S1E. Perhaps this is because I have BoyBlue's demo for this room?

Hmm, other players' demos shouldn't make the tool think you conquered it. Dunno here.

Three of the rooms it did list were in WIPs where I had arrived at a level entrance in an empty room with no exits. The game marks those as conquered right away, but never saves a victory demo because I couldn't exit.

What "conquered" means for rooms like this is sort of fishy. It would be a rather tricky for the tool to try to detect conquer-on-entrance rooms, though. I guess it won't really matter if you unconquer these or not.

The other two rooms were in beta holds that I had conquered in early versions but not yet in the current version of the hold. The fact that updated rooms stay conquered is not a random error in DROD, it's deliberate (or at least it happens that way every single time).

I'm going to call this a feature. I imagine some beta testers might want to unconquer these rooms, especially if one happened to be a significantly changed Secret Room.

And in any cases of "why in the world is room X listed / not listed?", I'm willing to look at users' *.dats and try to figure it out, if the user wants to arrange to send them. [Never post your *.dats publicly; they may contain a CaravelNet password.] PM if you'd like to send some dats my way.

---

[Last edited by schep at 04-09-2009 03:06 PM]

CuriousShyRabbit
Level: Smitemaster
Rank Points: 3170
Registered: 10-14-2006
IP: Logged

schep wrote:

CuriousShyRabbit wrote:
The unconquer tool gave me a list of five rooms to be unconquered, which did not include El Dorado : Finale : 1S1E. Perhaps this is because I have BoyBlue's demo for this room?

Hmm, other players' demos shouldn't make the tool think you conquered it. Dunno here.

Hmmm indeed... This is actually the only room I wanted your tool to find in the falsely conquered list. It's cases like that which confuse players who try to use the restore screen to find secrets that still need to be conquered. For all the rooms it did list, I already know that I didn't conquer them, and I already know the reasons why I didn't conquer them.

The other list - once I understood it - was indeed very helpful. It directed me to some cool nonhighscorable story rooms in a couple holds.

And in any cases of "why in the world is room X listed / not listed?", I'm willing to look at users' *.dats and try to figure it out, if the user wants to arrange to send them. [Never post your *.dats publicly; they may contain a CaravelNet password.] PM if you'd like to send some dats my way.

I'm sorry I can't send .dats at the moment. They're whopping HUGE, and they're full of the upcoming SmS holds.

Syntax
Level: Smitemaster
Rank Points: 1218
Registered: 05-12-2005
IP: Logged

Just send the .dits.

It's all one of the same.

JoMax
Level: Delver

Rank Points: 65
Registered: 12-28-2005
IP: Logged

Thanks Schep

Have just run this on my "dats" - of the rooms incorrectly marked as conquered (according to the tool) many were 'current' rooms (both normal and secret) ie Rooms I've started but not completed, and then changed location.

Only had 3 secret rooms marked as conquered related to the conquered secrets bug - which I've fixed.

Also had some unscorable rooms incorrectly marked conquered - exiting these rooms fixed the DB entry as expected.

Finally had 2 holds (Tacker Trilogy and Long Quest) where several rooms were incorrectly marked, but the game correctly sees them as conquered, on both these holds all my demos had been deleted - possibly from this bug - Guess I'll replay these holds...

HTH

Edit - Info as requested from your 1st post:-

Win XP, TCB v3.2.0.83 (full), tool picks up demo version installed to default, not full version (non-default) - my bad

---

[Last edited by JoMax at 04-10-2009 03:36 AM]

Syntax
Level: Smitemaster
Rank Points: 1218
Registered: 05-12-2005
IP: Logged

Ok, it was a bad pun. I was alluding to "this and that".

Anyways, close to recreating the scenario in Vista which I had on XP. Basically, the 2nd player gets all of my progress transferred to her upon importing the players at the same time.

I'd like to see what unconquer does with that, though I realise it's a bug in DROD's player management code.

I should have updates on this by Tuesday.