kieranmillar
Level: Smitemaster
Rank Points: 3668
Registered: 07-11-2014
IP: Logged

Given we are getting closer to DROD RPG 2's release, we probably don't want to rock the boat too much, however while developing on the engine it's clear we do really need to think about issues with DROD's dependencies and language version etc. I have this in my head right now so figured it was better to write down thoughts while I have them, even if we delay doing anything until after RPG 2's release.

DROD's github repo has two outstanding issues, so we might as well start there:

FMOD
https://github.com/CaravelGames/drod/issues/576
FMOD isn't open source, it's the only DROD dependency that isn't. Also, according to that issue, one of the other platforms has to swap it out for SDL_mixer anyway. Therefore it makes sense to me that we should consider swapping over. I don't know if we owe FMOD any money for using it in a paid product, if so then it probably makes sense to strip it out in time for DROD RPG 2's release.

Dependencies too old to build
https://github.com/CaravelGames/drod/issues/602
Two of the dependencies can no longer be built in the free versions of Visual Studio, they are just too old. Most (all? I haven't checked) of the dependencies have CMAKE available as an option to build them, so it might be worth taking a look if we can get the build script to use CMAKE as a way to build the dependencies.

But this also leads on to a more general concern about supporting old tools vs support from modern tools. At them moment we try to e.g. not use C++ 11 features, in order to support older compilers. C++ 11 came out in 2011, that's 14 years ago. While I can see the value in letting things potentially build easily on more systems, I think things can get so old that you eventually start to run into a new set of issues. 1) Modern IDEs, especially free versions for use by hobbyists, simply stop supporting older versions of various tech, making it much harder to get the code up and running by a new person. 2) When you look online for figuring out how to fix issues, more and more the solutions you find may make use of features you can't use in DROD's codebase.

I think there's always a tradeoff to be made between supporting the existing tech stacks of the old guard, and easier onboarding of new people that want to help out, and I feel like the DROD engine is increasingly falling on the wrong side of that tradeoff. Soon it will be the case that if somebody wants to help out with DROD development and build the game, they need to find tools that will support a tech stack that is older than they are!

Outdated dependencies
Some of the dependencies are no longer available to download on the internet, as the versions are so old and it's no longer recommended to use them. Updating dependencies is never as simple as you'd hope, I remember the issues we had with SDL_ttf and how an older version of Tom's New Roman had an incorrect flag that caused rendering issues.

However I do think that it is worth reviewing if some dependencies should have updating prioritised. For example, curl stands out to me as an issue, the version we have is over 10 years old and we this to talk over the internet, presumably for CaravelNet. The internet is a dangerous and scary place, I don't like the idea of our internet communication code being over a decade out of date! But there might actually be no real cause for concern.

-----

Back in 2017 I wanted to work on something in C++. I actually tried the DROD source first. This was before it was on Github. Well it was just simply too hard for a newbie like me, I had no idea how to get it to work. In the end I worked on something else and it went well, but it could have been DROD. Also, last year I got a new computer, and the truth is I have never been able to build it properly on this machine, I had to copy over the built dependencies from my old machine. To make it worse, on my old machine I can't actually build DROD anymore there either. I updated my Visual Studio there at one point to work on something else and something about the build tools or whatever was just incompatible with DROD and I simply couldn't build the source any more. It's only because of the new computer that I have been able to make PRs again and fix a load of bugs in the engine.

You may have noticed hyperme produces all of the DROD RPG 2 development builds in the private forum. That's no coincidence, when I build it, it mostly works but various things would crash the game frequently. I spent a while yesterday tracking one of those down and put in a PR, but it appears to be as far as I can tell a newer version of the standard library no longer putting up with something that was always wrong in the code but was simply not picked up by using an older version.

I am a little concerned about the impact for other platforms. There's been a bunch of linux users over the years having serious difficulty trying to build, and while slimm tomm has done great work maintaining a Mac build, it is slowly becoming more and more unstable, especially around importing and hold updating and I am a little concerned that nobody else will be able to help out there because it's too difficult to build.

---

[Last edited by kieranmillar at 01-11-2025 11:13 AM]

kieranmillar
Level: Smitemaster
Rank Points: 3668
Registered: 07-11-2014
IP: Logged

It's come to my attention that the main reason for the crashing when running local debug builds is not only a longstanding issue that everyone experiences, not just me, but is actually unfixable. It's the dreaded broken priority queue used in pathfinding. Well, I was able to fix it for RPG at least.

mrimer
Level: Legendary Smitemaster

Rank Points: 5430
Registered: 02-04-2003
IP: Logged

kieranmillar wrote:
FMOD
https://github.com/CaravelGames/drod/issues/576
FMOD isn't open source, it's the only DROD dependency that isn't. Also, according to that issue, one of the other platforms has to swap it out for SDL_mixer anyway. Therefore it makes sense to me that we should consider swapping over. I don't know if we owe FMOD any money for using it in a paid product, if so then it probably makes sense to strip it out in time for DROD RPG 2's release.

I understand we (Caravel) purchased a blanket license a long time back (~2005?) to use FMOD in all our products and OS versions (when such an option was available with the FMOD version we were using, and we're holding on to that). If SDL_mixer provides all the functionality we're using in FMOD, however, I'd be fine in principle with standardizing on SDL_mixer.

---

[Last edited by mrimer at 01-12-2025 05:10 PM]

bbb
Level: Master Delver
Rank Points: 218
Registered: 10-07-2013
IP: Logged

I think over the last few years we finally fixed the main issues which were causing problems on Linux.

I don't think I've had issues recently which were related specifically to my self compiled Linux builds.

My builds are usually with system libraries, which means they work with the latest (or almost latest) versions.

The main issue I have with building DROD is metakit. Replacing that would be a major change, and not just an upgrade. The easiest change, would probably be to convert everything to sqlite, though it may be better to completely reconsider the architecture of DROD data storage.

hyperme
Level: Smitemaster

Rank Points: 1458
Registered: 06-23-2006
IP: Logged

The main issue with metakit is that the guy who wrote it kept moving it around breaking links and generally being annoying. At some point we made a Caravel fork of it, which is included as a submodule in the main repo.

The sticking point with replacing metakit anyway is that DROD uses one of metakit's funky quirk: you can have fields in a table which are themselves tables. Or a view, as metakit calls it. You can definitely rewrite things to not require subviews, but definitely adds some extra work to it.

In terms of C++ versions, being stuck on the version before C++11 is pretty frustrating. There are a non-zero number of places where, for example, lambdas would be useful, but oh no that's a C++11 feature. (then in C++20 land we have std::ranges which is so wonderful). Moving up at least to C++11 would be very good.

For FMOD my main worry is the availability one, since the version DROD uses isn't directly provided by FMOD.

____________________________
[Insert witty comment here]
Qzvlkx?

kikuchiyo
Level: Master Delver
Rank Points: 166
Registered: 04-01-2012
IP: Logged

I just wanted to raise the same point myself: metakit is abandoned software, meaning not just that it's not actively maintained anymore but that it's missing from all common software repositories. There are multiple versions of unclear provenance and status on the internet and the build instructions in the DROD sources are not clear on which one to use (and there is contradicting information about it on the forum).

I realize that it may be outside the means of the current dev team to replace it with something else, but at the very least a properly working version of it should be vendored into the codebase.

Edit: Oh, I just saw that you already did that.

curl

IIRC I compiled DROD with the native libcurl I had on the system, so theoretically there should not be any issue of using a current version of it on other platforms. However... while I was trying to figure out why Caravelnet was not working (before realizing that it's actually missing from the sources) I noticed that DROD uses unencrypted HTTP to communicate with the Caravelnet servers, including requests that contain usernames and passwords - shouldn't you use HTTPS in this day and age?

---

[Last edited by kikuchiyo at 01-15-2025 08:22 PM]

skell
Level: Legendary Smitemaster

Rank Points: 3859
Registered: 12-28-2004
IP: Logged

Metakit might no longer be developed but I am not convinced it's a reason to switch, especially that (from what I understand) we have a fork and if anything needs fixing/changing we can handle it. It can be compiled too.

And replacing would be an insane undertaking. It's not just used for storage in the filesystem, it's used as an active database while the game is running. Replacing it would touch pretty much every class, would be super risky and at the end of the day you'd still need to keep linking to it for a seamless migration from an old version to a new one. It's really not worth the effort.

I noticed that DROD uses unencrypted HTTP to communicate with the Caravelnet servers, including requests that contain usernames and passwords

It's not passwords that are sent, it's the one-time keys. It is insecure in that if someone intercepted that they could abuse the API in your name but, beyond that, there doesn't seem to be anything risky or automated that can happen with it (and it's not like DROD is super popular and we have to worry about someone actively targeting the community).

Would be great to change it to HTTPS though, agreed.

____________________________
My website

vittro
Level: Smiter

Rank Points: 489
Registered: 04-17-2005
IP: Logged

kieranmillar wrote:
At them moment we try to e.g. not use C++ 11 features, in order to support older compilers. C++ 11 came out in 2011, that's 14 years ago.

Just wanted to add that this drives away potential contributors -- it is personally the reason why I never contributed anything to DROD's codebase.
No offense, but sticking to C++03 in 2025 is just archaic. After using Modern C++ for more than a decade both professionally and as a hobby, I can't see myself using C++03 unless I was being paid really well.

____________________________
http://vittorioromeo.com

mrimer
Level: Legendary Smitemaster

Rank Points: 5430
Registered: 02-04-2003
IP: Logged

Good points on leveraging C++ 11 and beyond.

I personally don't have a hard stance on avoiding C++ 11 and later; it's just a discussion point I bring up, as it's generally easy to avoid for typical maintenance. I understand it can be grating to stick to the archaic stuff, though, and I certainly see the benefits of using newer language features.

If the dev community wants to use newer stuff in the codebase, I won't stand in the way, as long as it's not including anything that's too bleeding edge that would present an issue with keeping the code maintained on all the current platforms.

Also, I'd prefer we think about limiting C++ language upgrades to where newer features touch the codebase, as opposed to rewriting swaths of preexisting code, just so we could say we rewrote it, as that carries not insignificant risk. I could be swayed on this point if there's a compelling argument made.

----

Re: Metakit

As skell said, I agree refactoring out this implementation is probably not ever going to be worth it from a practical standpoint. Even though the original implementation itself may be considered abandoned, in some ways, we could consider the Metakit code fork as being just another part of DROD's codebase now, for all intents and purposes. It functions as another internal set of classes. It works well, for what it does, and we don't really need to touch it. If there's a compelling argument for refactoring out *all* the Metakit implementation and all the client code for something else, I'm opening to hearing what that is.

hyperme
Level: Smitemaster

Rank Points: 1458
Registered: 06-23-2006
IP: Logged

The thing with staying on C++03 is it probably leads to a Catch-22 where you can't move onto C++11 because of the problems building the third party libraries, and you can't update the third-party libraries because you're stuck on C++03. That's not great. (although that's probably not that great an issue to break out of)

vittro's point is more pressing, since I'm willing to bet most C++ programmers are somewhat unaware of which features are from C++11, only that they've been taught to program in C++ with the assumption those things are there. Obviously, going back and elevenifying the codebase is silly, but it would be nice to be able to use it going forward.

---

In terms of metakit, I think the difficulty of replacing it is being a little oversauced. It's still not worth it, but IIRC all the use of metakit is restricted to specific database management and interaction functions. And to be frank "we'd need to keep it for backwards compatibility" reads like making up a guy to get mad at, but for dependencies. The only case where you'd want to rip out and replace metakit is during a major version update, where back-reading of older dat files isn't supported.

____________________________
[Insert witty comment here]
Qzvlkx?

slimm tom
Level: Smitemaster

Rank Points: 1279
Registered: 10-14-2006
IP: Logged

Just to give some historical context: https://github.com/CaravelGames/drod/pull/234

tl;dr: My current Mac build setup, as finicky as it is, doesn't support any C++11 features.

I'm not at all against upgrading things. But it would probably mean I would need to invest time in getting things working properly in a new setup.