Schik
Level: Legendary Smitemaster

Rank Points: 5447
Registered: 02-04-2003
IP: Logged

Okay, I'm working on a pretty major change to the forums. After 1.6 is released, there will be a new forum for user made holds. It will work a lot like the normal forums, except every topic must be regarding one final, released hold. If you want to get people to test your hold, I think that should go somewhere else - once you put a hold here, you won't be able to get rid of it or modify it, and it will *possibly* be put into the "Insert DROD Room image" database.

After a new hold (thread) is posted, users will be able to rate it. This will work similar to voting, but different. We've had a little (very little) private discussion about this, but I want to get a lot of opinions.

How would you like to rate a hold? I think the consensus will be that various categories should be used instead of one single 1-10 rating.

But what categories to rate?
Difficulty? Novelty? Thought required? Style? There are a ton of possible choices. We need to weed it down to a managable set of criteria.

I know we won't all be able to agree, but hopefully some discussion will help us make a good choice that will make most people happy.


____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

bdcribbs
Level: Smiter
Rank Points: 390
Registered: 04-08-2003
IP: Logged

Everything I can think of is so subjective it's hard to say what should be rated, I don't know how most of these could be rated meaningfully. A "similiar to..." option might be cool (e.g. if you enjoyed --- try ---), but such recommendations could be implicit spoilers.


One idea on hold finalization:

So that architects could get feedback and tweak their hold, perhaps there will be a "Playtesters" forum. There anyone could submit an unfinalized (able to be edited) hold for people to play with. These volunteers could play the holds at their own risk, knowing that the holds may be changed.

And, there may be a playability rating there. The playtesting forum would not judge style, difficulty, novelty, etc... just playability. This rating could serve the architect as an indication of when to release their hold, but a low rating needn't restrict someone from releasing.



[Edited by bdcribbs on 07-10-2003 at 06:15 AM]

bdcribbs
Level: Smiter
Rank Points: 390
Registered: 04-08-2003
IP: Logged

bdcribbs wrote:
And, there may be a playability rating there.

Or... maybe all the ratings happen at playtesting stage?

Maybe holds are allowed to be released only by popular vote, or it could just be informal feedback at this stage.

[Edited by bdcribbs on 07-10-2003 at 06:54 AM]

Oneiromancer
Level: Legendary Smitemaster

Rank Points: 2936
Registered: 03-29-2003
IP: Logged

Schik wrote:
But what categories to rate?
Difficulty? Novelty? Thought required? Style? There are a ton of possible choices. We need to weed it down to a managable set of criteria.

Any voting is going to be subjective, you can't really help it. And just a number doesn't tell you much. For example, if you have a "Fun" category, and I vote "5", does that mean that I found the hold to be totally average or I only had fun on half of the rooms within it? Those aren't quite the same thing, although perhaps they are close enough to some people that it doesn't matter.

I think that rating holds on difficulty and fun factor is important. But perhaps combine the two also? That is, is the entertainment the hold provides worth the time it took you to solve it? This kind of rating means that both short, easy, fun holds can get high marks, as well as long, difficult, but satisfying holds.

Oh, and any reason why you can't add the new forum now, at least for playtesting? Right now there are holds posted in the Bugs forum and the General forum (and possibly even other places) and I can never remember which one I am looking for. You could always just require that the post state that the hold was made using the beta version, just for in the future when the final edition has come out. And each poster will have to update their post when the final version is put in the main hold forum.

That's all I can think of right now.

Game on,

[Edited by Oneiromancer on 07-10-2003 at 07:32 AM]

____________________________
"He who is certain he knows the ending of things when he is only beginning them is either extremely wise or extremely foolish; no matter which is true, he is certainly an unhappy man, for he has put a knife in the heart of wonder." -- Tad Williams

Schik
Level: Legendary Smitemaster

Rank Points: 5447
Registered: 02-04-2003
IP: Logged

Oneiromancer wrote:
Any voting is going to be subjective, you can't really help it.

Of course it will be subjective - otherwise I'd just come up with an algorithm to rate them. This is all about what the users think of the holds.

And just a number doesn't tell you much. For example, if you have a "Fun" category, and I vote "5", does that mean that I found the hold to be totally average or I only had fun on half of the rooms within it? Those aren't quite the same thing, although perhaps they are close enough to some people that it doesn't matter.

Well, then perhaps there should be some sort of "consistency" rating. If half the levels are brilliant and half are crap, that would be nice to know.

Oh, and any reason why you can't add the new forum now, at least for playtesting? Right now there are holds posted in the Bugs forum and the General forum (and possibly even other places) and I can never remember which one I am looking for. You could always just require that the post state that the hold was made using the beta version, just for in the future when the final edition has come out. And each poster will have to update their post when the final version is put in the main hold forum.

For one, Erik doesn't want to do add it now. Neither do I. Oh yeah - and I'm not done coding it.

I don't want this to be used for playtesting - I think that should be kept somewhere else, and probably shouldn't be quite so formal. Also, once the release is out, I'd personally prefer to NOT playtest every hold - I don't want the finished product to be ruined for me because I already know half of it. (And I know I don't have the willpower to NOT download it if it's here )

Once DROD and the forum mods are done, all holds in this "User made holds" forum will be automagically shown in some location on the main drod.net site. Exactly how it's done is up to Maffy. When we have new users visiting drod.net, and they see a file downloads section with a bunch of new holds, I don't think we really want them to see a bunch of beta quality holds. I've seen my share of gaming sites out there with user made levels, and never have I seen anything but finished products.

I also don't want to allow users to update the hold once it's uploaded - what happens to all the ratings in that case? Throw them all out? Who knows what has changed. Maybe it was trivial, maybe it wasn't. If you add a hold to this forum, you should be pretty confident that it's in a final state.


____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

krammer
Level: Smitemaster
Rank Points: 904
Registered: 02-12-2003
IP: Logged

The only effective way to get a real rating system is to get someone (like Erik or Malarame) to review new holds and post reviews on drod.net, but I get the feeling so many holds will be coming out this will be nigh on impossible.

Anyway, I don't think you can really rate holds with numbers. I couldn't say that any one of the new levels I've downloaded is better or worse than others - I may like some better, but they all have good and bad points. I couldn't give them marks out of 10, even on different categories. As Oneiromancer said, ratings are subjective. I know reviews need to be, but in a good way. I would put any hold with nothing but goblins as a low fun and high difficulty simply because I hate them and always mess up badly. One person who hates something can really screw up an average.

I think the best system to begin with would be for every hold to be given - by the author - an approximate difficulty rating compared to some official milestone - perhaps whereabouts it wiould fit in Dugan's dungeon (i.e. about Levl8, or post level 25). This could be checked by playtesters before the final release.

And about playtesting - I think the best bet would be to have a Testers forum where pople can indicate that they want testers. Any people who are interested email the author and he emails the hold to them, they can then do all communication via email or PMs. That way people won't accidently download holds that aren't finalised.

Sorry this is a bit long winded, I get carried away with typing. I think that's enough for the moment.

____________________________
+++ Divide by Cucumber Error +++ Please Reinstall Universe and Reboot +++

"Oook!"

Malarame
Level: Master Delver
Rank Points: 220
Registered: 02-04-2003
IP: Logged

krammer wrote:
The only effective way to get a real rating system is to get someone (like Erik or Malarame) to review new holds and post reviews on drod.net, but I get the feeling so many holds will be coming out this will be nigh on impossible.

There will be a new section on DROD.net containing the new holds once the final version has been released. I'll do my best to play through and review each hold, but I'll be realistic and say that it probably won't be feasible by just one person. For that reason, I like the idea for people to be able to rate the holds. However, I think that people should be able to make a few comments in each of the categories. That way, they'd be able to explain their ratings.

____________________________
Matt O'Leary
Webb Institute
Class of 2007

"I never forget a face, but in your case I'll be glad to make an exception."
-Groucho Marx

Schik
Level: Legendary Smitemaster

Rank Points: 5447
Registered: 02-04-2003
IP: Logged

krammer wrote:
The only effective way to get a real rating system is to get someone (like Erik or Malarame) to review new holds and post reviews on drod.net

I'll have to respectfully disagree - with hopefully a LOT of people rating these holds, the net effect should be a well-rounded rating, in contrast to one person's opinion. While I think Erik and Matt could probably give good opinions on new holds, maybe their likes and dislikes are opposite of mine, and thus I shouldn't rely on just their opinion.

With what I'm proposing, everyone (well, everyone who's registered on the forum) will be able to voice their opinion, both by rating certain categories and by a full-fledged review, if they'd so like. This IS a forum after all - you can post a reply saying "This hold is horrible because it has a crapload of serpents, and I hate serpents." Hopefully something a little more verbose, but you get the picture.

Anyway, I don't think you can really rate holds with numbers. I couldn't say that any one of the new levels I've downloaded is better or worse than others - I may like some better, but they all have good and bad points.

Well, hopefully the categories chosen will adequately represent the good and bad points you're thinking of. That's why I posted this - to try to get opinions on what categories people think should be used.

I would put any hold with nothing but goblins as a low fun and high difficulty simply because I hate them and always mess up badly. One person who hates something can really screw up an average.

While "fun" might be a category, it very well might not. I'd rather see things like difficulty, creativity, style, ... etc. And hopefully we'll have enough people rating these that one idiot won't screw it up. Even if I hated a level (because it wasn't fun for me - i.e. I hate serpents) I could still appreciate it's difficulty, creativity, etc.

I think the best system to begin with would be for every hold to be given - by the author - an approximate difficulty rating compared to some official milestone - perhaps whereabouts it wiould fit in Dugan's dungeon (i.e. about Levl8, or post level 25). This could be checked by playtesters before the final release.

I agree that the author should be able to enter these things.

And about playtesting - I think the best bet would be to have a Testers forum where pople can indicate that they want testers. Any people who are interested email the author and he emails the hold to them, they can then do all communication via email or PMs. That way people won't accidently download holds that aren't finalised.

Couldn't that just go in one of the existing forums? Especially if all follow-up will be in PMs or email, as it should be.


____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

Quath
Level: Goblin
Rank Points: 17
Registered: 06-20-2003
IP: Logged

I would say we could start with author giving values for the following:

Difficulity from 1 to 10 (10 hardest 1 easiest)
Size (# rooms)
One sentence summary (e.g. "Simple puzzles for the beginner" or "Hack and slash dungein crawl" or "Story with little action thrown in")

Then have users rate:
Challenging (1 to 10 on how intuitative the puzzle was)
Puzzles: (Score on how well the puzzles were)
Combat: (Score on how well the combat was done)
Overall: (Score takes into effect above plus story, humor, design etc)


Yet another possibility.

Quath

agaricus5
Level: Smitemaster
Rank Points: 1838
Registered: 02-04-2003
IP: Logged

Quath wrote:
Difficulity from 1 to 10 (10 hardest 1 easiest)

Size (# rooms)
One sentence summary (e.g. "Simple puzzles for the beginner" or "Hack and slash dungein crawl" or "Story with little action thrown in")

Then have users rate:
Challenging (1 to 10 on how intuitative the puzzle was)
Puzzles: (Score on how well the puzzles were)
Combat: (Score on how well the combat was done)
Overall: (Score takes into effect above plus story, humor, design etc)


Yet another possibility.

Quath
The problem is what to rate "difficulty" on, for there are many types of things that make a hold difficult. I used this rather basic and not very good system for rating difficulty, but I'm sure you can see what I'm getting at:

Movement Skill (How specialised your movements need to be, e.g. backswiping is fairly specialised)

Patience Skill (How much patience you need)

Route Planning (How much you need to plan the route to take)

Guesswork (How obvious the solution, or what you need to do is (0 is very straightforward))

Frustration Factor (How frustratingly hard I found the level to be).

____________________________
Resident Medic/Mycologist