disoriented
Level: Smitemaster

Rank Points: 2401
Registered: 08-07-2007
IP: Logged

Assertion error in line 1650 of DbXML.cpp: "importBuf.isReset()"
Assertion error in line 1592 of DbXML.cpp: "importBuf.isReset()"
Assertion error in line 158 of DbXML.cpp: "!uncompressedBuffer"
Assertion error in line 181 of DbXML.cpp: "!compressedBuffer"
Assertion error in line 182 of DbXML.cpp: "!compressedSize"
Assertion error in line 217 of DbXML.cpp: "!d_stream"
Import Parse Error: junk after document element at line 16:0
Import Parse Error: junk after document element at line 16:0
Import Parse Error: junk after document element at line 16:0
Import Parse Error: junk after document element at line 4355:0
Import Parse Error: junk after document element at line 8709:0
Import Parse Error: junk after document element at line 13078:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0
Import Parse Error: junk after document element at line 17176:0


____________________________
34th Skywatcher

Best to PM me, since I might miss your message on CaravelNet chat.

---

[Last edited by disoriented at 08-06-2022 08:14 PM]

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

Does this occur when attempting to update any hold?

Unfortunately it seems this hasn't been reported for the Windows build, so it could be Mac-only. This looks like an issue with the changes Mike made to improve importing, so he's probably the best person to be looking at this.

____________________________
[Insert witty comment here]
Qzvlkx?

disoriented
Level: Smitemaster

Rank Points: 2401
Registered: 08-07-2007
IP: Logged

It does seem to occur with every hold I have tried.

____________________________
34th Skywatcher

Best to PM me, since I might miss your message on CaravelNet chat.

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

I was able to update a hold in the Windows version, so if you can't update any holds, I think it's more likely that is a Mac-only issue.

Two additional follow-up questions:

1. Does this affect both methods of updating, that is via CNet and through importing a file into the game?
2. Does the crash occur when importing a new hold into the game?

____________________________
[Insert witty comment here]
Qzvlkx?

disoriented
Level: Smitemaster

Rank Points: 2401
Registered: 08-07-2007
IP: Logged

1 appears to be yes. I download an update to King Hesper's and it crashed the program, although the .err log was a bit different:

Import Parse Error: junk after document element at line 137:0
Assertion error in line 1995 of DbXML.cpp: "bytesRead <= XML_PARSER_BUFF_SIZE"

2 appears to be no. I just imported your Scrumbly Hollow from a file without issues.

____________________________
34th Skywatcher

Best to PM me, since I might miss your message on CaravelNet chat.

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

New weird follow up question: Does the imported hold get updated in spite of the crash?

Trying to work out all the angles here.

____________________________
[Insert witty comment here]
Qzvlkx?

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

Double post: Decided to import some more holds via CNet and oh no it's not just a Mac issue.

Doing a CNet update with released holds leads to the Assertion error in line 1995 of DbXML.cpp: "bytesRead <= XML_PARSER_BUFF_SIZE" error.

The hold still updated, but saves got obliterated, so that's a problem.

____________________________
[Insert witty comment here]
Qzvlkx?

disoriented
Level: Smitemaster

Rank Points: 2401
Registered: 08-07-2007
IP: Logged

That does sound like what I was seeing.

____________________________
34th Skywatcher

Best to PM me, since I might miss your message on CaravelNet chat.

mrimer
Level: Legendary Smitemaster

Rank Points: 5138
Registered: 02-04-2003
IP: Logged

PR

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

As reported by disoriented, this issue, or a very similar one, is still occuring in the Mac builds.

____________________________
[Insert witty comment here]
Qzvlkx?

slimm tom
Level: Smitemaster

Rank Points: 1240
Registered: 10-14-2006
IP: Logged

Is this now a Mac only issue? Or does it still occur on Windows as well?

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

Currently disoriented is the only person who has report this issue still occuring, so it might be a Mac issue. I updated some holds on the latest Windows build and the issue didn't appear.

____________________________
[Insert witty comment here]
Qzvlkx?

disoriented
Level: Smitemaster

Rank Points: 2401
Registered: 08-07-2007
IP: Logged

It seems to happen only sometimes.

Two new holds were released today, Challenge Crafting Compilation and Architectural Bad Habits: Backtracking. They both showed as (Updated!) in the hold list in-game.

From the hold list, I was able to download the update for one of the holds, but the other one failed to update. I deleted the failed hold, and re-downloaded it, and that worked.

Can't remember which one failed of the two.

EDIT: I'm using the latest beta that slimm tom posted, I think it may be 5.1.1.454-speech-fix although the game's title screen says 5.1.1.409 for some reason.

____________________________
34th Skywatcher

Best to PM me, since I might miss your message on CaravelNet chat.

---

[Last edited by disoriented at 12-15-2022 07:44 PM]

Kalin
Level: Master Delver

Rank Points: 216
Registered: 01-25-2016
IP: Logged

I also installed the two new holds today.

Windows, non-Steam (GoG) version 5.1.1.436x, no CN subscription

Both holds were already installed from voting and testing.

Backtracking crashed when checking if demos were compatible. Restarting the game and trying again, it said I already had that version installed.

Crafting updated without problems. Did not mention needing to check saves, so I guess no changes were made since I last downloaded it.

____________________________
he/him
~170th Skywatcher

---

[Last edited by Kalin at 12-15-2022 11:42 PM]

Dragon Fogel
Level: Smitemaster
Rank Points: 2467
Registered: 06-21-2014
IP: Logged

There were changes made, I adjusted the scripting of a couple of rooms during the HA process.

disoriented
Level: Smitemaster

Rank Points: 2401
Registered: 08-07-2007
IP: Logged

Sounds like the remaining hold-import issues might be shared between Windows and Mac.

____________________________
34th Skywatcher

Best to PM me, since I might miss your message on CaravelNet chat.

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

Unfortunately, Kalin's datapoint is for 5.1.1.436x, which predates the original hold import fix. The latest available version for Windows is 5.1.1.459.

If anyone encounters this issue again, in the latest build, please remember and provide the following information:

1. The hold you tried to update
2. If it was via CNet or a local import
3. If the hold was updated when you reopned DROD
4. If your saved games were still there after reopening
5. Optionally, content from the drod.err file.

____________________________
[Insert witty comment here]
Qzvlkx?

Dragon Fogel
Level: Smitemaster
Rank Points: 2467
Registered: 06-21-2014
IP: Logged

Got a crash when updating on the Steam build for TSS.

Tried to update Entry Point, over CNet since it's a beta hold.
When I reopened DROD, the hold updated but my saves were gone.

drod.err has this line for today:
Assertion error in line 1999 of DbXML.cpp: "bytesRead <= XML_PARSER_BUFF_SIZE"

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

My current diagnosis for this issue is that there's something wrong with the data importing algorithm, but only when performing an internal import of saves and demos during a hold update.

To explain, updating a hold goes like this:

1. Get the hold data ready for importing
2. Export existing saves and demos for the hold to an internal holding space
3. Delete the existing version of the hold, and any associated saves and demos. (This is required due to database things)
4. Import the hold data
5. Reimport saves and demos
6. Check that demos are still valid

The process is going wrong either on step 2 or 4, causing a crash at step 4. This means the hold is successfully updated, but saved games and demos are lost.

Not sure how to fix this at the moment.

____________________________
[Insert witty comment here]
Qzvlkx?

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

Did some more investigating, and discovered the problem is in the reimport step. It's a problem with the details of how hold importing now works.

To fix problems with holds that are Too Big to fit into memory, Mike changed hold importing to decompress imported holds in chunks. There's a maximum chunk size, which is important. When importing holds, the data always starts compressed and is incrementally decompressed.

However, when saves are reimported, the game will load the entire uncompressed blob of data at once. If the amount of data is less than the maximum chunk size, everything is fine. However, if it's bigger than that, problems occur. The game assigns less memory to an buffer than it needs, leading to Bad Things, and the game crashes. (This is what the "bytesRead <= XML_PARSER_BUFF_SIZE" assert means. The statement is false, so the game complains.)

I'm not quite sure how to fix this, other than doing the perhaps drastic move of compressing the internally exported data so that algorithm will just do it's thing normally.

____________________________
[Insert witty comment here]
Qzvlkx?

---

[Last edited by hyperme at 12-20-2022 07:07 PM]

disoriented
Level: Smitemaster

Rank Points: 2401
Registered: 08-07-2007
IP: Logged

hyperme wrote:
I'm not quite sure how to fix this, other than doing the perhaps drastic move of compressing the internally exported data so that algorithm will just do it's thing normally.

Was this the ultimate solution?

____________________________
34th Skywatcher

Best to PM me, since I might miss your message on CaravelNet chat.

hyperme
Level: Smitemaster

Rank Points: 1173
Registered: 06-23-2006
IP: Logged

Luckily no compression was needed. Since the functions that import data into the database ultimately work on uncompressed data, new functions could be written to handle uncompressed data from internal sources.

____________________________
[Insert witty comment here]
Qzvlkx?

mrimer
Level: Legendary Smitemaster

Rank Points: 5138
Registered: 02-04-2003
IP: Logged

PR

Thank you so much for the fix, hyperme!

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.