mrimer
Level: Legendary Smitemaster

Rank Points: 5198
Registered: 02-04-2003
IP: Logged

...and I applied a heuristic hack that doesn't allocate memory so aggressively, and it actually worked in this case to allow re-importing your exported player file into your dats. (Yay!)

It's probably a better strategy to have in place in general, so I'm going to add it to my PR.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

mrimer
Level: Legendary Smitemaster

Rank Points: 5198
Registered: 02-04-2003
IP: Logged

I was struck with inspiration and devised a way to minimize the memory footprint used during import to manage data uncompression. This should resolve any inability to import large player profiles.

Until they get super duper large, anyway (i.e., at the point that a 32-bit process isn't good enough...and then we'd need to figure out 64-bit builds on Windows).

PR

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

disoriented
Level: Smitemaster

Rank Points: 2424
Registered: 08-07-2007
IP: Logged

You’ve got 64-bit builds on Mac at least

____________________________
34th Skywatcher

Best to PM me, since I might miss your message on CaravelNet chat.

mrimer
Level: Legendary Smitemaster

Rank Points: 5198
Registered: 02-04-2003
IP: Logged

Yep. I managed to do that. And trick did this on Linux too, iirc.

Unfortunately, how to rebuild all of the third-party libs as 64-bit in VS is an opaque process to me thus far.

If anyone wants to hack away on that one, be my guest. Although, I'm optimistic that optimizing the export/import memory usage pushes off the need for this a good ways.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

---

[Last edited by mrimer at 10-27-2020 10:33 PM]

RabidChild
Level: Smitemaster
Rank Points: 720
Registered: 01-15-2006
IP: Logged

Confirmed. I successfully exported and reimported my player from the latest alpha. Yay!

mrimer
Level: Legendary Smitemaster

Rank Points: 5198
Registered: 02-04-2003
IP: Logged

RabidChild wrote:
Confirmed. I successfully exported and reimported my player from the latest alpha. Yay!

Hooray! So glad this issue is resolved for you.

Caveat:
FYI -- there's a slightly modified file format used for exports, in this build of 5.1.1 and going forward, which I needed to transition to in order to get streamlined exports working. The files won't be recognized by earlier 5.x builds of the game.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

mrimer
Level: Legendary Smitemaster

Rank Points: 5198
Registered: 02-04-2003
IP: Logged

On further consideration, I'm backing out the export optimization for 5.1.1, as it is a format change that's incompatible with previous 5.x builds.

We'll get 5.1.1 out the door in a good state first, and then I'll re-introduce the export optimization immediately afterwards as I begin providing 5.2 dev builds. It will be trivial to re-include.

FWIW, the import optimization and new format support will remain in place, so 5.1.1 can continue to be used to import the new export format (until there's a hold version bump in 5.2).

PR

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

---

[Last edited by mrimer at 11-02-2020 08:28 PM]

robin
Level: Smitemaster

Rank Points: 852
Registered: 09-01-2004
IP: Logged

I just want to add, that it kinda works/worked to get a full exporting bar.
(It used to crash when the exporting bar was at 80% or something.)
But now I get a message, that it couldn't export the player because, I don't have enough space, or I don't have admin "acces".

I do have enough memory, but if someone can help me, how to give me full admin control?
I sometimes have some issues with this with other applications too, since this is my computer, I probably did something wrong in the past

____________________________

mrimer
Level: Legendary Smitemaster

Rank Points: 5198
Registered: 02-04-2003
IP: Logged

robin wrote:
I just want to add, that it kinda works/worked to get a full exporting bar.
(It used to crash when the exporting bar was at 80% or something.)
But now I get a message, that it couldn't export the player because, I don't have enough space, or I don't have admin "acces".

I do have enough memory, but if someone can help me, how to give me full admin control?
I sometimes have some issues with this with other applications too, since this is my computer, I probably did something wrong in the past

No, the issue your experiencing is not your fault, and it's not an admin access issue. The game has a 32-bit memory fragmentation issue, which the user shouldn't have to worry about. Hence my fix for it (which I deferred in the latest 5.1.1 dev build to 5.2; but you can still use the previous 5.1.1 dev build to export your player profile, with the caveat that you need to import it using version 5.1.1 or later).

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

---

[Last edited by mrimer at 11-06-2020 06:01 PM]

martz
Level: Smiter
Rank Points: 416
Registered: 08-19-2005
IP: Logged

Unfortunately this problem has reoccurred for me and it's a bit of an issue now because my old desktop PC is on it's last legs and I've got a replacement incoming in a few daysThe last time I successfully exported my profile was last August (this looks to coincide with my getting back into playing after a small hiatus). Not sure what version I was running at this point.

I patched the game a couple of weeks ago to 5.1.0.6505 (after another hiatus) and I am now unable to export (to clarify this, both full export and saves-only export generate the Windows runtime error and close the program).

I've tried:
1. Installing the 5.1.1 beta - this also fails but with no error at all - the program just closes.
2. Downgrading to the earlier available patch on the downloads page (5.0.1.6060) - this produces the same runtime error.

Worst case, I can use my last year profile when I install on the new machine, and I'll have to replay the 13 holds I've done since then, but it would be great if I could actually get my profile exported beforehand.

One other possibility - is the cloud sync fit for purpose? I recall trying that previously but didn't have much luck with it?

____________________________
Without struggle, there is no progress, so get up, get out, and go make the rest of your life the best of your life; you deserve to live the life that YOU want to live...

Progress - Gavin G

---

[Last edited by martz at 09-01-2021 04:56 PM]

Dragon Fogel
Level: Smitemaster
Rank Points: 2574
Registered: 06-21-2014
IP: Logged

There are five .dat files which contain all the important data. If you're just transferring between computers, you can move those directly to the new computer and not have to worry about exporting.

martz
Level: Smiter
Rank Points: 416
Registered: 08-19-2005
IP: Logged

Update in a new post for increased visibility

I noticed a setting in the ini file:

QuickPlayerExport=0

I've changed this to 1 and now get a prompt on export "Export less player data for quick import?". Selecting "Yes" does successfully export my profile although the file size is around 50% smaller than my previously exported profile.

A quick search shows this was added to JtRH in 2007: 2.0.16 JtRH patch candidate

The relevant details are:

+ Added "Startup:QuickPlayerExport" option to the INI. Set it to "1" to more quickly get saved games from one installation to another. It skips exporting demos and checkpoint saves to player profiles and "saved games only" exports. Should be much faster if you just want to retain general room explored/conquered progress.

Hopefully this should work for me - I'll update once I'm set up on my new PC

____________________________
Without struggle, there is no progress, so get up, get out, and go make the rest of your life the best of your life; you deserve to live the life that YOU want to live...

Progress - Gavin G

martz
Level: Smiter
Rank Points: 416
Registered: 08-19-2005
IP: Logged

Dragon Fogel wrote:
There are five .dat files which contain all the important data. If you're just transferring between computers, you can move those directly to the new computer and not have to worry about exporting.

Ah - that will make things easier for me, although there are still issues with the export which my post(s) above may assist with.

____________________________
Without struggle, there is no progress, so get up, get out, and go make the rest of your life the best of your life; you deserve to live the life that YOU want to live...

Progress - Gavin G

martz
Level: Smiter
Rank Points: 416
Registered: 08-19-2005
IP: Logged

Now all set up on my new PC, few minor issues but nothing too bad - I'm documenting it below for reference:

1. Fresh install of TSS 5.1.0.6512
2. Imported quick profile from old PC
3. Downloaded ALL holds (including all official)
4. Tried re-importing profile but TSS produced "Out of Memory" error, presumed edge-case due to downloading all holds
5. Restarted TSS and re-imported profile, reached final hold and closed without error
6. Restarted PC and tried re-importing profile again - this time successfully!
7. 8 holds previously completed not showing up as (100%) in holds list - went down exit stairs for each of these to rectify (list of holds below)
8. Now able to export full profile with no errors

List of holds:

City of Queens
Cludo's arena
Delver's Demise
Dinfro Budkin's Wine Cellar
ImpossibleCubes
Seeds Dungeon
Simplicity
Temple of Kurath

____________________________
Without struggle, there is no progress, so get up, get out, and go make the rest of your life the best of your life; you deserve to live the life that YOU want to live...

Progress - Gavin G

mrimer
Level: Legendary Smitemaster

Rank Points: 5198
Registered: 02-04-2003
IP: Logged

Thank you for the documented process. Glad you were able to migrate your progress to the new machine. I think everything you described makes sense and is expected (a known issue) in this circumstance.

Version 5.1.1 will begin addressing the import/export issues on Windows, and 5.2 should have them completely resolved in cases like yours.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.