mrimer
Level: Legendary Smitemaster

Rank Points: 5149
Registered: 02-04-2003
IP: Logged

Pilchard VIII wrote:
Okay, I've got the demo right here showing the no sound effect bug. I've tried it three times on two different computers, and it worked every time. But it seems it's only when the fuse is burning. Listen very carefully and you should catch where the bug is.

Ah, I understand now. I think I do hear this...but it might be some sound artifact of moving fast. I checked, and the game is playing the sounds correctly during these moves, so I'm not sure what's going on or whether something should be done to fix it.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

mrimer wrote:
Good news. I've reworked the import process (performing a lot more code maintenance than I had expected) and now DROD is much better behaved when importing huge .player files like yours. Loading and saving of game data is a bit faster. The initial import parsing (i.e. the first 50% of the progress bar, before demos+saved games begin to be verified) is probably significantly faster than it was before, too. Please redo your imports and let me know if everything seems to be working correctly.

Just tried rev6, and Virtual Size topped out at 642MB, which is nice...

The game's just checking demo integrity; I wonder what'll happen if I import the same player file again after it is done...

EDIT: To answer my own question - 677MB.

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

---

[Last edited by Briareos at 07-30-2006 10:35 AM]

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

By the way - is the game supposed to take 10+ seconds to import a downloaded demo with this version? I somehow remember this taking a lot less time in earlier versions... *scratches head*

np: SCSI-9 - Little Leaves Fall (The Line Of Nine)

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

do you mean importing a .demo file? Or downloading a highscore demo? Since the latter hasn't been significantly slower for me but I haven't tried the former with the beta patch.

____________________________
I got my avatar back! Yay!

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

eytanz wrote:
do you mean importing a .demo file? Or downloading a highscore demo? Since the latter hasn't been significantly slower for me but I haven't tried the former with the beta patch.

I guess it doesn't matter which you do.

The download finishes in less then a second for me - broadband will do this to you...

It's the "Importing demo" part that stalls 10+ seconds. Moreover, the "change location" screen stalls 5 or so seconds before it spits out the "100%" for Journey to Rooted Hold; I wonder what takes so long there? Shouldn't the amount of mastering for a hold be cached somewhere?

Also, come to think of it - I've just changed location to JtRH, and upon hitting "Restore" the game also froze for 10+ seconds (with it hogging cpu, but not redrawing the window) until the restore screen came up.

That's with a recent fresh install of the game where I imported all holds I have high scores for followed by importing my player export, so you could call it rather pristine...

np: Sturm - Track 7 (Sturmgesten)

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

---

[Last edited by Briareos at 08-05-2006 03:55 PM]

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

I'm not experiencing any of these slowdowns. I experience slowdowns before the "restore" screen shows up, and when pressing F6 until I see the demo list, but that's not new.

Anyway, whatever the issue is, it's not universal.

____________________________
I got my avatar back! Yay!

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

eytanz wrote:
I'm not experiencing any of these slowdowns. I experience slowdowns before the "restore" screen shows up, and when pressing F6 until I see the demo list, but that's not new.

Anyway, whatever the issue is, it's not universal.

Yeah, now I managed to import a demo for Magic Show in 2 or 3 seconds. Still, I wonder what happened...

It's just that the UI becoming unresponsive for several seconds is scary, as you never know what's happening...

np: Vladislav Delay - Karha (Multila)

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

coppro
Level: Smitemaster
Rank Points: 1309
Registered: 11-24-2005
IP: Logged

After changing the main entrance to the level, the choices when associating stairs don't change to reflect this until you change which room you're editing.

Pilchard VIII
Level: Smiter
Rank Points: 392
Registered: 01-21-2006
IP: Logged

I think the patch us still unstable. I tried importing The Underground Civilisation, which is a great hold. But when it imported, the whole of text.dat was wiped. So I was stuck playing Return of The Proximity Mines 2E. I do feel a bit dumb though for:

1. Noit making a recent backup.
2. Forgetting that all writing is in text.dat and replacing data.dat and player.dat. (>%/

____________________________
The DROD Dragon

Pilchard VIII
Level: Smiter
Rank Points: 392
Registered: 01-21-2006
IP: Logged

I was designing a room for KPVIIID3, when the weirdest thing happened. I was resizing a snake, and when I placed it, two tiles off the end were missing!!! Of course I thought it was cool, but when I tested the room and moved (surprise surprise) it crashed! I'll try to duplicate it again, but if I can't, I exported the room so you can see for yourself. It said also that there was an error in lines 155 and 157 of serpent.cpp.

____________________________
The DROD Dragon

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

I wrote:
It's just that the UI becoming unresponsive for several seconds is scary, as you never know what's happening...

Well, since I've just had DROD suck up a full CPU core for over a minute when leaving a level (~30 seconds before the end level text, and the same after hitting a key), I used Process Explorer to look at the stack dump of the thread that was eating my CPU for breakfast, and this is the result:

ntoskrnl.exe!ExReleaseResourceLite+0x2b4
ntoskrnl.exe!IoPageRead+0xd8a
hal.dll!HalClearSoftwareInterrupt+0x342
ntdll.dll!RtlReAllocateHeap+0xe96
ntdll.dll!RtlReAllocateHeap+0x83b
ntdll.dll!RtlInitializeCriticalSection+0x149
msvcrt.dll!free+0x1ae
msvcrt.dll!free+0x1cc
msvcrt.dll!operator new+0xf
drod.exe+0x6ac07

I made a new stackdump every few seconds, but it was always the same... could it be that the last optimizations, while saving on memory use, hurt normal operation a bit much?

(I mean, it's always suspended in operator new - what causes so many objects to be created when leaving a level? Also, just leaving a room I've already solved and entered again sometimes causes the same half-minute pause...)

np: Monolake - Deadbeat Remix (Plumbicon Versions)

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

---

[Last edited by Briareos at 08-19-2006 11:43 AM]

michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged

Briareos wrote:
could it be that the last optimizations, while saving on memory use, hurt normal operation a bit much?

My guess is that there's still too much memory allocation/deallocation/reallocation going on. It looks to me like your CPU is feverishly writing to virtual memory. I've also found that DROD uses far more memory than seems necessary, causing it to use large amounts of virtual memory, which is painfully slow. I'm not exactly right on top of the code, but from what I've seen, two general areas where memory usage might be reduced are:

Say you're entering a procedure that's going to be reading a lot of demos. Allocate a buffer that should be large enough to hold most typical demos. Instead of disposing and reallocating it after each demo, reuse it. So memory is allocated only once, unless that ridiculous 65000 move room happens to come up. This way you reduce fragmentation, so it's less likely that virtual memory will be allocated, and if it is, it doesn't happen constantly. Either way, you're reducing the amount of memory management done, which is slow at the best of times.

I'm not sure about this, but it could be that DROD sometimes reads unnecessarily large amounts of data into memory all at once, instead of reading it bit by bit. If this does happen, fixing it would make an enormous difference. I reckon the code wasn't written with the amazing number of currently available user-made holds in mind, so where this wouldn't have mattered in the past, it's becoming a problem.

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

michthro wrote:
My guess is that there's still too much memory allocation/deallocation/reallocation going on. It looks to me like your CPU is feverishly writing to virtual memory.

If you mean "thrashing around in my page file" by "writing to virtual memory" that'd be a no. There's no disk activity while this is going on. A lot of heap re(al)locations, probably. A lot of allocations of objects on the heap, definitely.

One of these days I'm going to look for a good profiler and go over it, but I don't have time for that currently...

EDIT: Oh yeah, while I'm on a roll - it seems as if a hold wasn't marked as updated if you brought up the Change Location dialog before it was updated (or something like that), i.e. a beta hold was surely updated while I was playing it, and the hold list doesn't show it - one restart of DROD later I suddenly see it.

Could it be that the check for new holds is only done once per launch? And is that a bug or a feature?

np: Monolake - Deadbeat Remix (Plumbicon Versions)

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

---

[Last edited by Briareos at 08-19-2006 06:12 PM]

michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged

Briareos wrote:
If you mean "thrashing around in my page file" by "writing to virtual memory" that'd be a no. There's no disk activity while this is going on. A lot of heap re(al)locations, probably. A lot of allocations of objects on the heap, definitely.

Wow. A minute of 100% cpu and no disk activity. I assumed it must at least involve a lot of disk activity, which is usually the case for me when DROD gets stuck for a while. There has to *really* be a lot of heap allocations then.

mrimer
Level: Legendary Smitemaster

Rank Points: 5149
Registered: 02-04-2003
IP: Logged

File: DROD_2_0_15_rev7_FixBadData.zip (227.6 KB)
Downloaded 47 times.
License: Public Domain

For people who have tried the DROD 2.0.15 patch candidates here and apparently lost data (such as the hold list) after importing something, here's an .exe that might fix it. Just unzip it into the directory that contains drod.exe and run it (while DROD is not running). It will tell you whether anything got fixed.

How this works: although it looks like your hold files are gone, they are probably still there, but some bad data introduced during import is now "hiding" it. This fix removes the bad data, so you should be able to see everything as before. (However, if your .dat files were actually scrambled somehow, as some people have reported in the past due to power failures, etc., this won't fix it.)

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

---

[Last edited by mrimer at 08-22-2006 12:55 AM]

Kevin_P86
Level: Smitemaster

Rank Points: 535
Registered: 06-28-2005
IP: Logged

I put it in the old broken install directory, and ran it, and a message came up that said

DROD's data files did not need patching and were not modified.

However, upon running the drod.exe in that directory, the holds list is now restored. So it must have done something...?

____________________________
+++++[>+++++<-]>[>+++>++++>+++++<<<-]>.>+.>-------.<++++.+++++.

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

Kevin_P86 wrote:
However, upon running the drod.exe in that directory, the holds list is now restored. So it must have done something...?

I guess that first message came from the patcher to tell you it really didn't touch your DAT-files, which is the main point - i.e. not destroying them further...

The actual fixing was then done by the new DROD executable, which probably isn't covered by the previous message...

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

mrimer
Level: Legendary Smitemaster

Rank Points: 5149
Registered: 02-04-2003
IP: Logged

Kevin_P86 wrote:
I put it in the old broken install directory, and ran it, and a message came up that said

DROD's data files did not need patching and were not modified.

However, upon running the drod.exe in that directory, the holds list is now restored. So it must have done something...?

Woah! I guess that's good news, but still rather odd. I'd really like to get ahold of your .dats (drod2_0.dat especially, but also player.dat and text.dat archived with it also, if possible) with the missing hold list, if you still have a copy around.

Edit:

The actual fixing was then done by the new DROD executable, which probably isn't covered by the previous message...

Heh. There is no new DROD .exe out yet.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

---

[Last edited by mrimer at 08-21-2006 11:02 PM]

Kevin_P86
Level: Smitemaster

Rank Points: 535
Registered: 06-28-2005
IP: Logged

Ok, here is a .zip containing drod2_0.dat, player.dat and text.dat from before the problem was fixed.

Edit: Well that's annoying - it spent about 5 minutes uploading, and then it didn't end up in the message. Is 15.3 MB too big to be attached...?

____________________________
+++++[>+++++<-]>[>+++>++++>+++++<<<-]>.>+.>-------.<++++.+++++.

---

[Last edited by Kevin_P86 at 08-21-2006 11:10 PM]

Kevin_P86
Level: Smitemaster

Rank Points: 535
Registered: 06-28-2005
IP: Logged

Let's try that again... maybe?

____________________________
+++++[>+++++<-]>[>+++>++++>+++++<<<-]>.>+.>-------.<++++.+++++.

Briareos
Level: Smitemaster

Rank Points: 3516
Registered: 08-07-2005
IP: Logged

mrimer wrote:
Edit:

The actual fixing was then done by the new DROD executable, which probably isn't covered by the previous message...

Heh. There is no new DROD .exe out yet.

Eeep... I was assuming what you attached was a patcher like for all the earlier revisions - didn't really take a closer look at it...

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

Jeff_Ray...
Level: Smitemaster

Rank Points: 962
Registered: 05-16-2005
IP: Logged

It doesn't work for me!

____________________________
I make Let's Plays too!
Click here to find out my LPs' progress, and find out what I plan to tackle next!

Currently playing:


My Holds:
-Completed:

-Work in Progress:

mrimer
Level: Legendary Smitemaster

Rank Points: 5149
Registered: 02-04-2003
IP: Logged

Kevin_P86 wrote:
Let's try that again... maybe?

Excellent! Thanks. I confirmed the problem in your .dats and that this .exe fixes them. However, when I ran it on your .dats, I got the ".dats were patched successfully message", so I have no idea why you got the other message. In any event, I'm glad they're fixed now.

It doesn't work for me!

What do you mean?

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

mrimer
Level: Legendary Smitemaster

Rank Points: 5149
Registered: 02-04-2003
IP: Logged

Without further ado, I've posted a rev8 that should not cause the hold list to become empty any more, or other apparent data loss. Also fixed new crashes from bad data and a new problem with incomplete player import (experienced by Eytan), and the crash with tailless serpents.

As before, if you want to try this patch, please backup your four .dat files first. I'd recommend making an additional, separate backup for each day or however often you feel comfortable with. If the empty-hold-list bug is still around, please try to revert to your backed up .dats and retrace your steps to try to reproduce the problem. If you can reproduce it, then post your .dats along with steps for reproducing the apparent data loss...and, BAM! I'll be able to fix it right away. But here's to hoping that it's permanently fixed already.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

Jeff_Ray...
Level: Smitemaster

Rank Points: 962
Registered: 05-16-2005
IP: Logged

mrimer wrote:

It doesn't work for me!

What do you mean?

Well, i get an error about SDL.dll. Then anohter error occurs when i put it in that folder. It's something about SDL_strlcpy.

____________________________
I make Let's Plays too!
Click here to find out my LPs' progress, and find out what I plan to tackle next!

Currently playing:


My Holds:
-Completed:

-Work in Progress:

mrimer
Level: Legendary Smitemaster

Rank Points: 5149
Registered: 02-04-2003
IP: Logged

Jeff_Ray... wrote:

mrimer wrote:

It doesn't work for me!

What do you mean?

Well, i get an error about SDL.dll. Then anohter error occurs when i put it in that folder. It's something about SDL_strlcpy.

Then it looks like you're not unzipping the file to the directory containing DROD.exe, which is where you should be putting it to run it. (Note that the directory where drod.exe is does indeed have SDL.dll in it already.)

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

---

[Last edited by mrimer at 08-22-2006 12:39 AM]

Jeff_Ray...
Level: Smitemaster

Rank Points: 962
Registered: 05-16-2005
IP: Logged

Oh! I thought it was in the Data folder >_<

____________________________
I make Let's Plays too!
Click here to find out my LPs' progress, and find out what I plan to tackle next!

Currently playing:


My Holds:
-Completed:

-Work in Progress:

Pilchard VIII
Level: Smiter
Rank Points: 392
Registered: 01-21-2006
IP: Logged

File: To Mrimer.hold (1.4 KB)
Downloaded 43 times.
License: Public Domain

The snake bug I report earlier doesn't seem to be quite fixed yet. I was experimenting with the bug, and I came up with these experiments. I know I really shouldn't of, but it was for a good cause surely. If you see the description, you can see the effects of what happened to me when I ran them. I hope this helps fix the bugs a bit more.

____________________________
The DROD Dragon

mrimer
Level: Legendary Smitemaster

Rank Points: 5149
Registered: 02-04-2003
IP: Logged

Pilchard VIII wrote:
The snake bug I report earlier doesn't seem to be quite fixed yet. I was experimenting with the bug, and I came up with these experiments. I know I really shouldn't of, but it was for a good cause surely. If you see the description, you can see the effects of what happened to me when I ran them. I hope this helps fix the bugs a bit more.

Thank you. You're right -- I haven't fixed the serpent placement bug yet. I haven't checked your hold yet and I'll ask in advance -- do you describe what steps you performed to make this problem happen? I need to understand what things you do in the editor to make these problems happen in order to fix the bug. Just seeing that the problem can happen isn't enough to fix this bug.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

Pilchard VIII
Level: Smiter
Rank Points: 392
Registered: 01-21-2006
IP: Logged

It's just what I did for the first serpent bug, but I just experimented to find how deep the bug goes. It just randomly happened when I was resizing the serpents. And then for these I just changed the tail position, and changed the monster type from serpent to rattlesnake.

____________________________
The DROD Dragon