Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

Hey dudes --

Let's say I want to dump some time into a pet feature that I'd like to see in the DRoD games. What's the most reasonable path to get this feature into the official DRoD source tree? As far as I can tell there is no "official" repository that I can clone and patch against.

Also, in this case, the feature in question is CaravelNet related, and it seems that all net code has been stripped from the public release. Is that right, or am I looking in the wrong place?

-- John

Jatopian
Level: Smitemaster
Rank Points: 1842
Registered: 07-31-2005
IP: Logged

It's open-source in the sense that you are allowed to see the source, redistribute it, alter it, compile altered versions. "Everyone can make changes" isn't a prerequisite for the label.

I'm not sure how they're legally getting away with not putting the net code in, but I'm sure it's nothing shady.

____________________________
DROD has some really great music.
Make your pressure plates 3.0 style!
DROD architecture idea generator

Briareos
Level: Smitemaster

Rank Points: 3517
Registered: 08-07-2005
IP: Logged

Jatopian wrote:
It's open-source in the sense that you are allowed to see the source, redistribute it, alter it, compile altered versions. "Everyone can make changes" isn't a prerequisite for the label.

I'm not sure how they're legally getting away with not putting the net code in, but I'm sure it's nothing shady.

The code you can download is licensed under the MPL (Mozilla Public License), but as long as Caravel holds the copyright on all used code they can release it including non-open code (like CaravelNet) any way they want, as well as under any license they see fit.

____________________________
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
R.I.P. Robert Feldhoff (1962-2009)

Schik
Level: Legendary Smitemaster

Rank Points: 5527
Registered: 02-04-2003
IP: Logged

Hoff,

Yeah, we don't have a public svn server or anything, you'd need to base your code off of the latest release at http://forum.caravelgames.com/viewsitepage.php?id=98165 . Unfortunately the 3.2.1.89 code is not up there yet. I'll see if Mike can easily get that together.

While the actual communication with the server is in the CaravelNet library, most everything else is in the public source - i.e. to view high scores, the interface was a lot more work than the function call to grab scores from a given room.

The most reasonable way to get your idea into an official release would be:
1) Let us know what the idea is, either here or privately.
2) If we agree to add it to CaravelNet, a more detailed design - user interface, how and when it will be used, server requirements, etc.
3) If it's what I think it is, you could code the entire thing, adding methods to NetInterface that would save/load data locally instead of from CaravelNet just for testing, and give us a patch. If it's something else, we'll figure it out once I know what it is.
4) We'd write the actual CaravelNet methods and server-side code.


____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

Schik --

Okay, that all sounds reasonable.

I was thinking it would be nice to have player profiles sync to CaravelNet for several reasons:

1) It's a pain to manually backup profiles across OS reinstalls/new machines. I lost my main profile a couple times mostly because I just plain didn't think to back it up, and have hence done levels 1-15 in JtRH several times now.

2) It makes playing on multiple machines much easier.

Now, I think there are some questions that need to be answered before implementation -- namely, is it automatic and transparent or is there an "Import/Export via CaravelNet" button in the UI? I can think of problems with both, but the latter is probably a good starting point.

Anywhoo, if this sounds like a reasonable idea I certainly wouldn't mind taking a stab at it. I know the Caravel team is limited in time and that this would (rightfully so) be pretty far down on the list.

-- John

Schik
Level: Legendary Smitemaster

Rank Points: 5527
Registered: 02-04-2003
IP: Logged

I agree that synching your player to CaravelNet would be awesome. I mentioned wanting that on a private board recently, but haven't written any code to support it.

Some thoughts:
There's a couple ways to go about this.
1) The client does all merging of players. "Sync with CaravelNet" means download CaravelNet player, merge, upload CaravelNet player.
2) The server does all merging of players. Progress could be uploaded continuously or batched. "Sync" would upload anything not sent yet and download the CaravelNet player and merge it.

Number 1) is much easier on the server - it just accepts and serves a file. However, the wait to build and upload and entire .player to upload is probably substantial.
Number 2) is easier on the client - uploading partial .player files could be done transparently to the user. However, server load and development time would be much greater.

I'm not sure which we should do. If 1) is "good enough" then it's certainly the simple solution. How big is a really big player file? The biggest one I have around is 1.8MB - I can try running that through my parser to see how long the server takes to parse it.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

Excellent! I think in general I feel better about 1), although I hope 1.8MB per player is the exception and not the rule. What's taking up most of the space, all the demos and savegames?

If there are no objections, I'll start working on 1) and add a per-player option to sync with CaravelNet on DRoD startup and exit. My only real concern is how to handle merges, although a quick look at the code implies it might be as simple as importing every available savegame and such.

-- John

Blondbeard
Level: Smitemaster

Rank Points: 1507
Registered: 03-31-2005
IP: Logged

Wow! This would be wonderful. In most cases I am unable to help in H&S, since I don't have access to the room in question, or my demos. I would really apreciate being able to access every room I've ever played!

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

Okay, I just put up an initial implementation at my BitBucket account. The source is hosted with mercurial; if you're not familiar with mercurial you can download the repo with "hg clone http://bitbucket.org/box/drod ". You can view the major changeset here, but there are a couple other minor changes I made to get it to compile on my machine. Note that at the moment this will only build on POSIX systems (the janky CaravelNet-on-local-filesystem uses POSIX functions to modify file timestamps); if you guys normally test on Windows I can probably get this all building in Windows too.

So far I have download/upload working on game enter/exit (respectively) and controlled by a per-player option on the Settings screen. It will only update players that it thinks have changed by tracking the LastUpdated field in each player. DRoD seems to do the Right Thing when merging saves (i.e., progress is not lost when an older save is imported), which is really awesome. And by "download/upload" I mean write files to the DRoD data directory, as Schik suggested.

The methods I added to CNetInterface are:
UploadPlayer
DownloadPlayer
GetPlayerList
UploadModifiedPlayers
DownloadModifiedPlayers

The last two don't do any direct CaravelNet logic per se, and can be refactored to wherever they actually belong. I added {Upload/Download}ModifiedPlayers at approximately the right spots in Main.cpp, but I'm sure they aren't ideal.

Things left to do are (at least) add the proper localization strings (does DRODUtil build on Linux?) and add UI notifications while working. And we probably want some way to delete players from CaravelNet, I just realized.

So, does this all sound good so far? If so, I'll finish up the UI stuff and we can go from there!

-- John

Schik
Level: Legendary Smitemaster

Rank Points: 5527
Registered: 02-04-2003
IP: Logged

Hey John,

First off, great work so far! A couple issues though:

- I think this should just do a sync on the current player.... so if multiple players have CaravelNet accounts (or one CaravelNet user has multiple player profiles) each would be sent/retrieved when that player profile is being used and only then.

- The import needs to change all imported data to be owned by the current player. For instance, I have two copies of DROD on different computers, and on each I've played various holds. With the code as-is, if I go to computer A and upload my player to CaravelNet, and then go to computer B and download my player, computer B now has two distinct players, both named Schik, because I created the two players individually (as I'm guessing most people do). Then when I upload from computer B, I overwrite the CaravelNet player from computer A, losing it.

Does this make sense?

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

---

[Last edited by Schik at 03-28-2011 04:07 AM]

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

- Okay, I'll move the download to after player selection and find a place to put the upload.

- Yep, makes perfect sense -- sounds like the issue is that DRoD uses an integral ID while my stuff is using the player name.

Thanks for your help!

-- John

Tim
Level: Smitemaster

Rank Points: 1979
Registered: 08-07-2004
IP: Logged

Please, please do not ever use the player name to do anything.
Especially merging.

I've had so many people who had a player name like mine, it's not funny.

____________________________
The best way to lose customers is to let little kids running loose on a forum with too many mod points.

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

Please, please do not ever use the player name to do anything.
Especially merging.

I've had so many people who had a player name like mine, it's not funny.

It only looks for the same name across your individual CaravelNet account, so you should be fine there.

-- John

Schik
Level: Legendary Smitemaster

Rank Points: 5527
Registered: 02-04-2003
IP: Logged

Hoff wrote:
It only looks for the same name across your individual CaravelNet account, so you should be fine there.

Does it even need to do that? Can't it just upload/download a player file for the current CaravelNet player? Do people need or want to synchronize *parts* of their player based on the name of the in-game profile they're using?

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

Schik wrote:

Hoff wrote:
It only looks for the same name across your individual CaravelNet account, so you should be fine there.

Does it even need to do that? Can't it just upload/download a player file for the current CaravelNet player? Do people need or want to synchronize *parts* of their player based on the name of the in-game profile they're using?

Maybe I'm confused as to exactly what a player profile is. I'm taking it to be all the information that is in the "name.player" as generated by the Export button in the UI. I might have two players, say John and JohnJr, associated with my CaravelNet account login Hoff. When syncing, it would say, "JohnJr is playing, so give me the player profile with that name if you have it."

Is that not right the right way to think about it?

-- John

Penwielder
Level: Smitemaster
Rank Points: 628
Registered: 09-12-2009
IP: Logged

Does this mean that (when I get CaravelNet again, as I said I would do shortly before the next game comes out) I will be able to import my old AE profile into TCB, connect it to my CaravelNet account, synchronize, and thus finally get all my progress on AE holds back into my main profile again? That would be really nice. I didn't realize the consequences at first when I started using TCB and created a new player profile instead of going through the proper upgrade process (that is, importing all the holds I had done and then importing my old profile).

____________________________
Penwielder's Palace, Detention Complex, Archipelago, Cube of Memories

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

Maybe, Penwielder? If I understand the player handling code it normally will only merge two players if it thinks they are "equal", as defined by (name, time-of-creation). This would sidestep that test, so it probably would do what you're looking for.

Schik, to put it another way, it seems our misunderstanding is that I thought there was a many-to-one relationship between player profiles and CaravelNet accounts, whereas it's really intended to be one-to-one?

-- John

Schik
Level: Legendary Smitemaster

Rank Points: 5527
Registered: 02-04-2003
IP: Logged

Hoff wrote:
Maybe, Penwielder? If I understand the player handling code it normally will only merge two players if it thinks they are "equal", as defined by (name, time-of-creation). This would sidestep that test, so it probably would do what you're looking for.

Yeah, it should. You would synchronize with the AE profile, then sync with the TCB profile, and they'd be merged.

Schik, to put it another way, it seems our misunderstanding is that I thought there was a many-to-one relationship between player profiles and CaravelNet accounts, whereas it's really intended to be one-to-one?

It is intended to be one-to-one. That's how it is with high scores, for example - all high scores are associated with your CaravelNet ID, not your in-game name. So if John and JohnJr both upload demos for scores, they're all tallied as Hoff's scores.

The hard part will just be making the merge work. I'll have to think this over more before deciding for sure if we should allow backups of multiple profiles or not.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

That's a good point about the highscores, that argues for one-to-one pretty strongly. Anyway, I took a first pass at the merging (changeset) -- from what I can tell DRoD will remap player IDs in an imported file if it can match up (name, time-of-creation) to one in the local database. While importing from CaravelNet I changed this to just name, which would cause players of the same name to be synced between machines (this prevents the two-players-named-Schik syndrome you brought up earlier).

It appears demos and savegames are smart about not importing duplicates of themselves, although I haven't really tested this yet.

I'm also not sure if there are side effects to doing this, such as causing all player references with coincidentally the same name in imported demos and such to point to the local player instead.

-- John

Blondbeard
Level: Smitemaster

Rank Points: 1507
Registered: 03-31-2005
IP: Logged

Is this possible to use now, and if so how?

Schik
Level: Legendary Smitemaster

Rank Points: 5527
Registered: 02-04-2003
IP: Logged

No - I've done some work on this but it's been stalled for a while. I'm hoping that this will make it into the first release of the Next Big Project.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

Hoff
Level: Delver
Rank Points: 42
Registered: 08-23-2004
IP: Logged

Yeah, I haven't touched it in a little while either -- my hack in DbPlayers.cpp turns out to have not done what I thought it did, as the multiple-players-with-same-name problem is still present. Is CDbPlayer::GetLocalID not the right place to address this? It seems it was made to remap remote->local IDs, which is exactly what we want.