Announcement: Be excellent to each other.


Caravel Forum : Caravel Boards : The Site : How the heck do you want to rank new holds?
New Topic New Poll Post Reply
Poster Message
Schik
Level: Legendary Smitemaster
Avatar
Rank Points: 5381
Registered: 02-04-2003
IP: Logged
icon How the heck do you want to rank new holds? (0)  
Okay, I'm working on a pretty major change to the forums. After 1.6 is released, there will be a new forum for user made holds. It will work a lot like the normal forums, except every topic must be regarding one final, released hold. If you want to get people to test your hold, I think that should go somewhere else - once you put a hold here, you won't be able to get rid of it or modify it, and it will *possibly* be put into the "Insert DROD Room image" database.

After a new hold (thread) is posted, users will be able to rate it. This will work similar to voting, but different. We've had a little (very little) private discussion about this, but I want to get a lot of opinions.

How would you like to rate a hold? I think the consensus will be that various categories should be used instead of one single 1-10 rating.

But what categories to rate?
Difficulty? Novelty? Thought required? Style? There are a ton of possible choices. We need to weed it down to a managable set of criteria.

I know we won't all be able to agree, but hopefully some discussion will help us make a good choice that will make most people happy.


____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi
07-10-2003 at 05:17 AM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores Quote Reply
bdcribbs
Level: Smiter
Rank Points: 390
Registered: 04-08-2003
IP: Logged
icon Re: How the heck do you want to rank new holds? (0)  
Everything I can think of is so subjective it's hard to say what should be rated, I don't know how most of these could be rated meaningfully. A "similiar to..." option might be cool (e.g. if you enjoyed --- try ---), but such recommendations could be implicit spoilers.


One idea on hold finalization:

So that architects could get feedback and tweak their hold, perhaps there will be a "Playtesters" forum. There anyone could submit an unfinalized (able to be edited) hold for people to play with. These volunteers could play the holds at their own risk, knowing that the holds may be changed.

And, there may be a playability rating there. The playtesting forum would not judge style, difficulty, novelty, etc... just playability. This rating could serve the architect as an indication of when to release their hold, but a low rating needn't restrict someone from releasing.



[Edited by bdcribbs on 07-10-2003 at 06:15 AM]
07-10-2003 at 06:04 AM
View Profile Send Private Message to User Send Email to User Visit Homepage Show all user's posts Quote Reply
bdcribbs
Level: Smiter
Rank Points: 390
Registered: 04-08-2003
IP: Logged
icon Re: Re: How the heck do you want to rank new holds? (0)  
bdcribbs wrote:
And, there may be a playability rating there.

Or... maybe all the ratings happen at playtesting stage?

Maybe holds are allowed to be released only by popular vote, or it could just be informal feedback at this stage.

[Edited by bdcribbs on 07-10-2003 at 06:54 AM]
07-10-2003 at 06:51 AM
View Profile Send Private Message to User Send Email to User Visit Homepage Show all user's posts Quote Reply
Oneiromancer
Level: Legendary Smitemaster
Avatar
Rank Points: 2936
Registered: 03-29-2003
IP: Logged
icon Re: How the heck do you want to rank new holds? (0)  
Schik wrote:
But what categories to rate?
Difficulty? Novelty? Thought required? Style? There are a ton of possible choices. We need to weed it down to a managable set of criteria.

Any voting is going to be subjective, you can't really help it. And just a number doesn't tell you much. For example, if you have a "Fun" category, and I vote "5", does that mean that I found the hold to be totally average or I only had fun on half of the rooms within it? Those aren't quite the same thing, although perhaps they are close enough to some people that it doesn't matter.

I think that rating holds on difficulty and fun factor is important. But perhaps combine the two also? That is, is the entertainment the hold provides worth the time it took you to solve it? This kind of rating means that both short, easy, fun holds can get high marks, as well as long, difficult, but satisfying holds.

Oh, and any reason why you can't add the new forum now, at least for playtesting? Right now there are holds posted in the Bugs forum and the General forum (and possibly even other places) and I can never remember which one I am looking for. You could always just require that the post state that the hold was made using the beta version, just for in the future when the final edition has come out. And each poster will have to update their post when the final version is put in the main hold forum.

That's all I can think of right now.

Game on,

[Edited by Oneiromancer on 07-10-2003 at 07:32 AM]

____________________________
"He who is certain he knows the ending of things when he is only beginning them is either extremely wise or extremely foolish; no matter which is true, he is certainly an unhappy man, for he has put a knife in the heart of wonder." -- Tad Williams
07-10-2003 at 07:23 AM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
Schik
Level: Legendary Smitemaster
Avatar
Rank Points: 5381
Registered: 02-04-2003
IP: Logged
icon Re: Re: How the heck do you want to rank new holds? (0)  
Oneiromancer wrote:
Any voting is going to be subjective, you can't really help it.
Of course it will be subjective - otherwise I'd just come up with an algorithm to rate them. This is all about what the users think of the holds.

And just a number doesn't tell you much. For example, if you have a "Fun" category, and I vote "5", does that mean that I found the hold to be totally average or I only had fun on half of the rooms within it? Those aren't quite the same thing, although perhaps they are close enough to some people that it doesn't matter.
Well, then perhaps there should be some sort of "consistency" rating. If half the levels are brilliant and half are crap, that would be nice to know.

Oh, and any reason why you can't add the new forum now, at least for playtesting? Right now there are holds posted in the Bugs forum and the General forum (and possibly even other places) and I can never remember which one I am looking for. You could always just require that the post state that the hold was made using the beta version, just for in the future when the final edition has come out. And each poster will have to update their post when the final version is put in the main hold forum.
For one, Erik doesn't want to do add it now. Neither do I. Oh yeah - and I'm not done coding it. :)

I don't want this to be used for playtesting - I think that should be kept somewhere else, and probably shouldn't be quite so formal. Also, once the release is out, I'd personally prefer to NOT playtest every hold - I don't want the finished product to be ruined for me because I already know half of it. (And I know I don't have the willpower to NOT download it if it's here :) )

Once DROD and the forum mods are done, all holds in this "User made holds" forum will be automagically shown in some location on the main drod.net site. Exactly how it's done is up to Maffy. When we have new users visiting drod.net, and they see a file downloads section with a bunch of new holds, I don't think we really want them to see a bunch of beta quality holds. I've seen my share of gaming sites out there with user made levels, and never have I seen anything but finished products.

I also don't want to allow users to update the hold once it's uploaded - what happens to all the ratings in that case? Throw them all out? Who knows what has changed. Maybe it was trivial, maybe it wasn't. If you add a hold to this forum, you should be pretty confident that it's in a final state.


____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi
07-10-2003 at 01:26 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores Quote Reply
krammer
Level: Smitemaster
Avatar
Rank Points: 904
Registered: 02-12-2003
IP: Logged
icon Re: How the heck do you want to rank new holds? (0)  
The only effective way to get a real rating system is to get someone (like Erik or Malarame) to review new holds and post reviews on drod.net, but I get the feeling so many holds will be coming out this will be nigh on impossible.

Anyway, I don't think you can really rate holds with numbers. I couldn't say that any one of the new levels I've downloaded is better or worse than others - I may like some better, but they all have good and bad points. I couldn't give them marks out of 10, even on different categories. As Oneiromancer said, ratings are subjective. I know reviews need to be, but in a good way. I would put any hold with nothing but goblins as a low fun and high difficulty simply because I hate them and always mess up badly. One person who hates something can really screw up an average.

I think the best system to begin with would be for every hold to be given - by the author - an approximate difficulty rating compared to some official milestone - perhaps whereabouts it wiould fit in Dugan's dungeon (i.e. about Levl8, or post level 25). This could be checked by playtesters before the final release.

And about playtesting - I think the best bet would be to have a Testers forum where pople can indicate that they want testers. Any people who are interested email the author and he emails the hold to them, they can then do all communication via email or PMs. That way people won't accidently download holds that aren't finalised.

Sorry this is a bit long winded, I get carried away with typing. I think that's enough for the moment. ;)

____________________________
+++ Divide by Cucumber Error +++ Please Reinstall Universe and Reboot +++

"Oook!"
07-11-2003 at 07:21 PM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
Malarame
Level: Master Delver
Rank Points: 220
Registered: 02-04-2003
IP: Logged
icon Re: Re: How the heck do you want to rank new holds? (0)  
krammer wrote:
The only effective way to get a real rating system is to get someone (like Erik or Malarame) to review new holds and post reviews on drod.net, but I get the feeling so many holds will be coming out this will be nigh on impossible.
There will be a new section on DROD.net containing the new holds once the final version has been released. I'll do my best to play through and review each hold, but I'll be realistic and say that it probably won't be feasible by just one person. For that reason, I like the idea for people to be able to rate the holds. However, I think that people should be able to make a few comments in each of the categories. That way, they'd be able to explain their ratings.

____________________________
Matt O'Leary
Webb Institute
Class of 2007

"I never forget a face, but in your case I'll be glad to make an exception."
-Groucho Marx
07-12-2003 at 02:31 AM
View Profile Send Private Message to User Send Email to User Visit Homepage Show all user's posts Quote Reply
Schik
Level: Legendary Smitemaster
Avatar
Rank Points: 5381
Registered: 02-04-2003
IP: Logged
icon Re: Re: How the heck do you want to rank new holds? (0)  
krammer wrote:
The only effective way to get a real rating system is to get someone (like Erik or Malarame) to review new holds and post reviews on drod.net
I'll have to respectfully disagree - with hopefully a LOT of people rating these holds, the net effect should be a well-rounded rating, in contrast to one person's opinion. While I think Erik and Matt could probably give good opinions on new holds, maybe their likes and dislikes are opposite of mine, and thus I shouldn't rely on just their opinion.

With what I'm proposing, everyone (well, everyone who's registered on the forum) will be able to voice their opinion, both by rating certain categories and by a full-fledged review, if they'd so like. This IS a forum after all - you can post a reply saying "This hold is horrible because it has a crapload of serpents, and I hate serpents." Hopefully something a little more verbose, but you get the picture.
Anyway, I don't think you can really rate holds with numbers. I couldn't say that any one of the new levels I've downloaded is better or worse than others - I may like some better, but they all have good and bad points.
Well, hopefully the categories chosen will adequately represent the good and bad points you're thinking of. That's why I posted this - to try to get opinions on what categories people think should be used.
I would put any hold with nothing but goblins as a low fun and high difficulty simply because I hate them and always mess up badly. One person who hates something can really screw up an average.
While "fun" might be a category, it very well might not. I'd rather see things like difficulty, creativity, style, ... etc. And hopefully we'll have enough people rating these that one idiot won't screw it up. Even if I hated a level (because it wasn't fun for me - i.e. I hate serpents) I could still appreciate it's difficulty, creativity, etc.
I think the best system to begin with would be for every hold to be given - by the author - an approximate difficulty rating compared to some official milestone - perhaps whereabouts it wiould fit in Dugan's dungeon (i.e. about Levl8, or post level 25). This could be checked by playtesters before the final release.
I agree that the author should be able to enter these things.
And about playtesting - I think the best bet would be to have a Testers forum where pople can indicate that they want testers. Any people who are interested email the author and he emails the hold to them, they can then do all communication via email or PMs. That way people won't accidently download holds that aren't finalised.
Couldn't that just go in one of the existing forums? Especially if all follow-up will be in PMs or email, as it should be.


____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi
07-12-2003 at 03:55 AM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores Quote Reply
Quath
Level: Goblin
Rank Points: 17
Registered: 06-20-2003
IP: Logged
icon Re: How the heck do you want to rank new holds? (0)  
I would say we could start with author giving values for the following:

Difficulity from 1 to 10 (10 hardest 1 easiest)
Size (# rooms)
One sentence summary (e.g. "Simple puzzles for the beginner" or "Hack and slash dungein crawl" or "Story with little action thrown in")

Then have users rate:
Challenging (1 to 10 on how intuitative the puzzle was)
Puzzles: (Score on how well the puzzles were)
Combat: (Score on how well the combat was done)
Overall: (Score takes into effect above plus story, humor, design etc)


Yet another possibility.

Quath
07-24-2003 at 12:08 AM
View Profile Send Private Message to User Show all user's posts Quote Reply
agaricus5
Level: Smitemaster
Rank Points: 1838
Registered: 02-04-2003
IP: Logged
icon Re: Re: How the heck do you want to rank new holds? (0)  
Quath wrote:
Difficulity from 1 to 10 (10 hardest 1 easiest)
Size (# rooms)
One sentence summary (e.g. "Simple puzzles for the beginner" or "Hack and slash dungein crawl" or "Story with little action thrown in")

Then have users rate:
Challenging (1 to 10 on how intuitative the puzzle was)
Puzzles: (Score on how well the puzzles were)
Combat: (Score on how well the combat was done)
Overall: (Score takes into effect above plus story, humor, design etc)


Yet another possibility.

Quath
The problem is what to rate "difficulty" on, for there are many types of things that make a hold difficult. I used this rather basic and not very good system for rating difficulty, but I'm sure you can see what I'm getting at:

Movement Skill (How specialised your movements need to be, e.g. backswiping is fairly specialised)

Patience Skill (How much patience you need)

Route Planning (How much you need to plan the route to take)

Guesswork (How obvious the solution, or what you need to do is (0 is very straightforward))

Frustration Factor (How frustratingly hard I found the level to be).

____________________________
Resident Medic/Mycologist
07-24-2003 at 10:35 AM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
New Topic New Poll Post Reply
Caravel Forum : Caravel Boards : The Site : How the heck do you want to rank new holds?
Surf To:


Forum Rules:
Can I post a new topic? No
Can I reply? No
Can I read? Yes
HTML Enabled? No
UBBC Enabled? Yes
Words Filter Enable? No

Contact Us | CaravelGames.com

Powered by: tForum tForumHacks Edition b0.98.8
Originally created by Toan Huynh (Copyright © 2000)
Enhanced by the tForumHacks team and the Caravel team.