silver
Level: Smitemaster
Rank Points: 915
Registered: 01-18-2005
IP: Logged

So for me to rate a hold a perfect 10 it needs most of the following (not necessarily all):

* clean, elegant puzzle design
* inspires thought
* excellent story and dialog, ideally with memorable characters
* amusing surprises
* good savepoint placement
* aesthetics in layout and/or room-design
* a lack of annoyances like tar mazes, empty-crumbly-wall-mazes, major bottlenecks, orb puzzles, and not having "final" savepoints by the exit stairs

In general, the above need to be combined artistically and originally.

---

How do you rate holds?


____________________________

---

[Last edited by silver at 09-24-2006 09:13 AM]

Alneyan
Level: Smitemaster
Rank Points: 622
Registered: 07-06-2004
IP: Logged

My criteria for ratings go more or less along these lines: 6 corresponds to an average hold, with little irritation but some interesting rooms. 5 is for holds that are annoying, lack polish... but still have a few interesting rooms. 4 starts being "a couple jewels, if you look hard enough" and then it all goes downhill from here.

Going upwards, 7 is "a fun time, but some problems", 8 is "recommended", with either a strong concept or a strong implementation, 9 is for a very solid design and an host of interesting rooms, and 10 is simply for holds that shine through. Simon's Dungeon will probably get a 10 for me, as soon as I manage to finish up the hold. I don't care at all about storytelling myself, though I do like some comments. Heck, HIJK is probably my favourite hold in that department, simply because of Tscott's comments.

I use a "challenging/frustrating" balance for difficulty; 7 is my best difficulty rating, meaning "challenging only in a good way", with higher ratings increasing on the frustrating scale. Some easy but frustrating holds could get a very low difficulty score instead, but there aren't that many of them.

Pinnacle
Level: Smitemaster

Rank Points: 1129
Registered: 06-10-2004
IP: Logged

Quality Rating
1- Unplayable. 'Nuff said.
2- Bad. Playable, but just barely. May have some potential, though.
3- Poor. All slashfests, few puzzles, trial-and error, etc.
4- Not quite. Amateurish design, and significant flaws.
5- Average. Most holds fall here.
6- Recommended. It has something to it that makes it definately worth playing, flaws are few and minor.
7- Highly recommended. No substantial flaws, and excellent design.
8- Must-have. You can't call yourself a smitemaster without playing this hold.
9- Masterful. Above and beyond all expectations, exceeds KDD and JtRH in overall quality.
10- Perfect. Transcends the "collection of puzzles" and becomes a work of art. Only one hold has ever gotten a 10 from me, and that is Larry Murk's excellent Perfection. (It is Perfect, after all)
Yeah, so a 6 is actually pretty decent.

Brain rating
How often am I stuck?
1- Never
2- Rarely, and not for more than a minute.
3- Rarely or briefly, not both.
4- Not often
5- About average
6- More often than a 5
7- On about half the rooms.
8- On most of the rooms.
9- On almost all of the rooms, but I can pull through with H&S.
10-Beethro's Teacher

____________________________
Once (adv.): Enough.
Twice (adv.): Once too often.
~Ambrose Bierce, The Devil's Dictionary

---

[Last edited by Pinnacle at 09-24-2006 02:49 AM]

NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged

Pinnacle wrote:
1- Unplayable. 'Nuff said.
2- Bad. Playable, but just barely. May have some potential, though.
3- Poor. All slashfests, few puzzles, trial-and error, etc.
4- Not quite. Amateurish design, and significant flaws.
5- Average. Most holds fall here.
6- Recommended. It has something to it that makes it definately worth playing, flaws are few and minor.
7- Highly recommended. No substantial flaws, and excellent design.
8- Must-have. You can't call yourself a smitemaster without playing this hold.
9- Masterful. Above and beyond all expectations, exceeds KDD and JtRH in overall quality.
10- Perfect. Transcends the "collection of puzzles" and becomes a work of art. Only one hold has ever gotten a 10 from me, and that is Larry Murk's excellent Perfection. (It is Perfect, after all)
Yeah, so a 6 is actually pretty decent.

And what about brains?

Blondbeard
Level: Smitemaster

Rank Points: 1486
Registered: 03-31-2005
IP: Logged

I think Pinnacles quality description is pretty good, although I don't get stuck as often as him The only hold I have given a perfect ten is Perfection, but there are a few nines.

Another question is HOW people vote. I think you ought to play through at least half of a hold before you rate it, but I don't think everybody does that. I have seen some comments in some larrymurk holds, and now the voting is quite odd in my brothers hold, The Underground Civilisation.
If I am correct the hold has been rated more or less as follows:
9/7.5 (by me)
10/10
10/10 (by persons I suspect havn't played through the hold)
9/8
9/7.5 (probably by Rabscuttle or michthro, who tested it)
10/10
7/8
7/8 (I don't think these votes of seven comes from peoples who have played the hold. I think they might be originating from peoples who for some reason tries to lower the score a bit to even it out.)

Sorry for going a bit off-topic, but I think this is rather important.

Alneyan
Level: Smitemaster
Rank Points: 622
Registered: 07-06-2004
IP: Logged

I'm very pricky about voting or commenting myself; I normally only do that with the whole of the hold conquered and mastered, period. If I'm a beta-tester, I also tend to wait for quite a while before commenting; I can vote right away, though, if a CaravelNet vote is required (if not, I'll also wait).

Doom
Level: Smitemaster

Rank Points: 3226
Registered: 07-05-2004
IP: Logged

I've given 10 to about 3 holds. (none of which are Smitemaster's Selections, surprisingly)

One of them is KDD (+KDD2, which I didn't count above, because it's essentially the same) which is probably still the best hold ever. A perfect introduction into the game with a great variety of puzzles. I just can't find anything wrong with this.

I also gave Fool's Errand 10, but for different reasons. It used a new concept. That alone isn't enough for a 8+ rating, but what made it get full marks is the perfect execution of it. That is,
Spare/Blue Rooms were great holds, but despite having great puzzles, they weren't worth a 10 from me, because they were just rooms scattered around. Some kind of structure is a good thing. Basically, the rooms could be the same, but progressing in the hold would be more fun. And having fun a hold is a requirement for 10/10.

NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged

Doom wrote:
Basically, the rooms could be the same, but progressing in the hold would be more fun. And having fun a hold is a requirement for 10/10.

Unfortuantly, what is defined as fun differs from person to person.
Do you mind if i ask what makes a fun hold for you?

Doom
Level: Smitemaster

Rank Points: 3226
Registered: 07-05-2004
IP: Logged

I did mention some things in that post.

It's mostly about good or original puzzles, but other things can improve/degrade the whole picture.

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

What I do is basically start from a rough score based on overall considerations like puzzle quality and challenge, and originality (similar to other people above, except that I'm likely to start out with a higher score), and then deduct points for things that irritate me, like lack of checkpoints, scripting-based puzzles, too many bottlenecks, tar mazes, or too many serpent manipulation puzzles.

It ends up that 8 is the score I give most holds I like, 5 is the score I give most holds I dislike, and 3 is the score I give holds I really hate.

____________________________
I got my avatar back! Yay!

NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged

eytanz wrote:
...scripting-based puzzles...

Mind you, I have seen some pretty well done scripting puzzles.

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

True - I was listing things that *tend* to irritate me, not things that necessarily do. If I don't get irritated, I don't have a problem.

And usually, I should point out, I have no problem with scripting puzzles or serpents or tar mazes if they are in holds dedicated to them. I just don't like them when they block my way to other puzzles.

____________________________
I got my avatar back! Yay!

Remlin
Level: Master Delver
Rank Points: 181
Registered: 04-28-2005
IP: Logged

I tend to rate more towards the extremes than the middle. I'd only give a 4, 5 or 6 if the hold has both unusually excellent and terrible qualities at once. Something like Metroid Prime. Otherwise I consider my time playing the hold either well spent or poorly spent, and I rate very high or very low accordingly. There's no real in between.

The most common offenders for things that make me feel like it's time poorly spent are easy but time-consuming rooms (crumbly wall mazes, long serpent timers...), forced backtracking, orb puzzles, or groups of rooms that are all more or less the same. If you avoid all that, you'll probably get at least a 7.

Blondbeard
Level: Smitemaster

Rank Points: 1486
Registered: 03-31-2005
IP: Logged

I just realized that there is another important factor to voting for me, and that is the quality of the hold. If a hold obviously is a piece of devotion, hard work and imagination I would never give it a raiting below 7. There are some examples of this. Real quality holds I just don't think is funny. I don't want to hurt any architect by naming them, but I think I can saftely say that larrymurks Fun Park is one of these holds. That is because I absolutely love most of what Larry have produced.

With Fun Park I found the rooms very tedious, and didn't get any satisfaction from solving them. If someone forced me to complete the hold and then forced me to rate I would probably give it a fun raiting of 3 or 4.

The reason for me not to rate a hold like this is the quality of it. It happens to not suit my tastes, yet I can see that it's a very good hold. Thus, when this happens it seems as if the only fair thing I can do is not to vote.

Tahnan
Level: Smitemaster

Rank Points: 2459
Registered: 11-14-2005
IP: Logged

So, pursuant to the discussion that broke out over here where it doesn't really belong, some of which was my fault, I'll go ahead and post my feelings here:

I don't have criteria. I go on sheer impression: did I like it? Did I enjoy it? Was it well done? (What does "enjoy" mean, anyway? I enjoy Bavato's Dungeon in a very different way than I enjoy Odd Jobs, which I enjoy in a very different way than I enjoy not-dr0d....)

There are factors that go into that impression. Serious spelling and grammar mistakes are likely to leave me with a less positive impression than I would have otherwise had. Bad scripting will as well.

(Again, what's "bad"? Well...a poorly-executed script, such as one that runs again when I re-enter a room even though it shouldn't, or one that fails to even though it should; but also a well-crafted, perfectly-executed script that does exactly what it should, but what it does is something that annoys me. Some people spoke, here and in that thread, of good workmanship; I don't care how well someone writes a script that causes a guard to appear and disappear apparently at random around the screen, killing me if I happen to be in the space where his sword shows up. It would annoy me, and I'd rank the hold lower for it.)

So the upshot is, I ask myself, "What did I think of this hold?" and I rate it accordingly.

And, let me emphasize one more time here, as it was stressed over there: I don't care how anyone else rates holds, or what they rate holds. I call 'em as I see 'em, and I'd be downright startled if people voted the same way as me every time, or even within three rankings of me every time. And I think everyone should feel free to do the same.

---

[Last edited by Tahnan at 09-27-2006 10:18 PM]

Schik
Level: Legendary Smitemaster

Rank Points: 5381
Registered: 02-04-2003
IP: Logged

I'm going to expand the scope of this thread a little, which is just a nice way of saying I'm going off-topic.

Here's my question: Why don't people rate holds?

I just ran a little query to compare the number of holds that people have conquered (well, just those that CaravelNet knows about obviously) and how many holds they've rated. While there are a few notable exceptions who rate most of the holds, I'd say most people rate 10-20% of the holds that CaravelNet knows they've conquered. Presumably they've played more holds than they've conquered, and probably conquered more than CaravelNet knows about.

It's also somewhat humorous to me that many of the people who are most vocal about discussing how you vote and how other people should vote are in that group that only rates 10-20% of the holds they play. The best way to make those outliers less influential is... *gasp*.... to rate the hold yourself. C'mon people, rock the vote! Or something.

Hopefully this will be a moot point when TCB comes out, and people can just rate holds from within the game.

So, those of you who don't rate holds: Why not? If you get TCB with CaravelNet, will you be more likely to rate holds in-game than using the forum? I know there have been requests to have a "My ratings" page, where you can see all of the holds, what you've rated them, and be able to rate/change ratings. Would that help get more ratings? Would something else that I haven't thought about help?

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged

Schik wrote:
[snip]Hopefully this will be a moot point when TCB comes out, and people can just rate holds from within the game.

So, those of you who don't rate holds: Why not? If you get TCB with CaravelNet, will you be more likely to rate holds in-game than using the forum? I know there have been requests to have a "My ratings" page, where you can see all of the holds, what you've rated them, and be able to rate/change ratings. Would that help get more ratings? Would something else that I haven't thought about help?

Well I honestly can say that I have done my best to ensure that I have rated every hold I have played.

The major issue with rating holds, as you have said, is convenience. After I have mastered a hold, I then have to go onto the forum search, search for that topic, find one of the posts in that topic(I wish the topic search would only search the first post in each topic) scroll my way to the top, and go to page one. I then read through all the comments, and then mark the hold.

If I could simply click on the topic from my profile it would be a great help.

Rabscuttle
Level: Smitemaster

Rank Points: 2460
Registered: 09-10-2004
IP: Logged

I rate holds infrequently and inconsistently. A page where I could see and change all my ratings would be good.

---

[Last edited by Rabscuttle at 09-28-2006 05:37 AM]

silver
Level: Smitemaster
Rank Points: 915
Registered: 01-18-2005
IP: Logged

I might rate more holds now, but a year ago, I didn't care about that. So I didn't rate holds. The thing is, I conquered a LOT of holds back then.

Here's the thing: how can I figure out which holds I've rated, and which I haven't? What if I actually thought a hold deserved a 5?

Improve the user experience of rating holds and finding their TODO list of holds played v. holds rated, and the users will follow.


____________________________

Remlin
Level: Master Delver
Rank Points: 181
Registered: 04-28-2005
IP: Logged

I just forget to do it. Especially if the thread isn't on the front page of the Holds board, it won't occur to me to rate it. If TCB is going to have some in-game way of prompting me to rate a hold after I conquer it, I'd end up rating much more often.

Banjooie
Level: Smitemaster

Rank Points: 1645
Registered: 12-12-2004
IP: Logged

I download a hold, I play it.

And then whenever I look at the thread, I immediately scroll down to the bottom, and just..don't really bother to rate it.

Schik
Level: Legendary Smitemaster

Rank Points: 5381
Registered: 02-04-2003
IP: Logged

silver wrote:
Here's the thing: how can I figure out which holds I've rated, and which I haven't? What if I actually thought a hold deserved a 5?

This might not be the most intuitive, but if the button says "Rate" then you haven't rated the hold yet. If you have, it will say "Change Rating".

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

MeckMeck GRE
Level: Smitemaster
Rank Points: 594
Registered: 01-03-2005
IP: Logged

I rate a hold like that....

I play the hold and when I finished it I take a look back at some rooms. Then I rate it. Important criterias for me are :

* Puzzle design
* Originality
* Story/Scripts
* Bugs/Trivial Solutions
* Smaller things

1 - horrible - The hold is just plain boring and makes absolutly no fun. I regret that I downloaded it.
2 - dreadful - The hold is boring and lacks about everything that makes a good hold
3 - weak - The puzzles are not really good and many things could have been done better
4 - average - It's ok and some rooms are quite enjoyable.
5 - nice - The hold is decent, the puzzles are ok, but still : something is missing
6 - good - All in all a good hold. Good job !
7 - very good - I would recommend this hold because it contains good puzzles/ideas/story/...
8 - awesome - This hold rocks !
9 - almost perfect - I can hardly think of a better hold-
10 - perfect - Explains itself

As far as difficulty is concerned :

1/2 : I went trough this dungeon like a knife through butter
3/4 : Sometimes it was a little challenging
5/6 : Challenging but not hard
7/8 : I was hardly able to do it and needed hints
9/10 : Help !

---

[Last edited by MeckMeck GRE at 09-28-2006 01:25 PM]

michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged

Schik wrote:
Here's my question: Why don't people rate holds?

Mostly I just don't get around to it, but:

-A lot of holds I've conquered I won't rate because I don't think it's fair to rate a hold long after it's published. I already made the point in the Bavato's Dungeon thread that frankly, if I were to rate it after replaying it not so long ago, I'd give it a 6, but when I first played it years ago, I remember thinking that it's perfect. It contains a lot of rooms of a type I wasn't sick of yet *at the time it was published*.

-I don't like rating down a hold that someone obviousy put a lot of time and effort into, even if I think it's poor quality despite that. I feel that rating holds is not the same as rating, say, authors (of books), to continue with an idea from the other thread. Architects are not making money or anything out of providing us with their work, which makes a big difference. It's similar to the old free-open-source attitude of if you don't like it, don't use it, and keep your wise-cracks to yourself; constructive criticism welcome, but saying it's worth 2/10 isn't constructive criticism. I even refrain from rating down poor quality holds that were thrown together in half an hour, but that's changing fast.

trick
Level: Legendary Smitemaster
Rank Points: 2580
Registered: 04-12-2003
IP: Logged

But michthro, if everybody rated (or didn't rate) holds based on what they thought they might have deserved, in stead of being honest about their actual experience and rating based on that, the entire rating system would be useless. When sorting by overall score, people would get holds that should have been good, or deserved better, but still might have major flaws that prevent them from being a great experience when actually playing the game. See the problem there ?

Edit: Er, seems I mixed up the threads. Wops.

Anyway, michthro, I do know what you mean.. Maybe there should be more voting categories ?

~ Gerry

---

[Last edited by trick at 09-28-2006 03:00 PM : wops]

Schik
Level: Legendary Smitemaster

Rank Points: 5381
Registered: 02-04-2003
IP: Logged

michthro wrote:
-A lot of holds I've conquered I won't rate because I don't think it's fair to rate a hold long after it's published. I already made the point in the Bavato's Dungeon thread that frankly, if I were to rate it after replaying it not so long ago, I'd give it a 6, but when I first played it years ago, I remember thinking that it's perfect. It contains a lot of rooms of a type I wasn't sick of yet *at the time it was published*.

If you don't want to go back and change your votes if you change your opinion about a hold, that's fine. But I don't see any sense in not rating holds that are older. You still have a valid opinion about how good the hold is. If you feel that newer holds make older holds seem inadequate, then that's a valid opinion that should be shared.

-I don't like rating down a hold that someone obviousy put a lot of time and effort into, even if I think it's poor quality despite that. I feel that rating holds is not the same as rating, say, authors (of books), to continue with an idea from the other thread. Architects are not making money or anything out of providing us with their work, which makes a big difference. It's similar to the old free-open-source attitude of if you don't like it, don't use it, and keep your wise-cracks to yourself; constructive criticism welcome, but saying it's worth 2/10 isn't constructive criticism. I even refrain from rating down poor quality holds that were thrown together in half an hour, but that's changing fast.

You seem to think the ratings are aimed mostly towards the architects, while I think they're for prospective players. Sure, architects want to get high ratings, but the rating system wasn't invented to give architects a pat on the back - it was to give players a way to make a somewhat informed decision about what hold they should try next. Rating something (2/10 or anything else) isn't meant to be constructive criticism. If you want to do that, then make a post in the hold's thread, and give as detailed criticism as you want to. But please rate the hold 2/10, if that's what it's worth, so that others know the hold's not all that great. If the architect learns from your criticism and makes the hold better, you can always change your rating.

I don't know about that book analogy either... you're saying that if someone stands to lose a lot of money (they're trying to sell their book) then it's okay to tell everyone how bad it is. But if they won't lose anything, and other people will potentially waste lots of time by playing a really bad hold, then you should keep quiet?

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

Tahnan
Level: Smitemaster

Rank Points: 2459
Registered: 11-14-2005
IP: Logged

To answer Schick's question: if I don't rate a hold, a large part of it is because I don't remember which holds I've rated; and another large part is that, once I conquer a hold, I'm quite likely to breathe a deep sigh of relief and satisfaction and quit DROD to get back to work, in which case I'm unlikely to remember to go back and rate the hold the next time I'm playing.

I think in-game rating will help immensely with the latter. A summary page would help at least somewhat with the former.

jbluestein
Level: Smitemaster

Rank Points: 1670
Registered: 12-23-2005
IP: Logged

Schik wrote:
So, those of you who don't rate holds: Why not? If you get TCB with CaravelNet, will you be more likely to rate holds in-game than using the forum? I know there have been requests to have a "My ratings" page, where you can see all of the holds, what you've rated them, and be able to rate/change ratings. Would that help get more ratings? Would something else that I haven't thought about help?

I think a "My ratings" page would definitely help me. I try to rate every hold when I complete it, but if I don't do it right away for whatever reason, it's quite tedious to track which holds I've rated and which I haven't.

At this point, I've conquered 65 holds (actually, I think it's more, but I have a persistent problem getting my scores registered on CaravelNet), and while I've as many as I can remember to, I just went through the first handful on my list and found a bunch I hadn't rated. So I have no idea how many I've actually rated.

Seems like it would be easy to incorporate into the My Stats page -- a column indicating the rating, if any, for each hold in the Conquered and Explored categories (I do think it's OK to rate holds that you haven't conquered yet, under certain circumstances...I tend not to do it, but I've probably done a couple for holds that I think will take me a long time to finish but that I still like (Beethro's Teacher and Magic Show spring to mind)).

Josh



____________________________
"Rings and knots of joy and grief, all interlaced and locking." --William Buck

Alneyan
Level: Smitemaster
Rank Points: 622
Registered: 07-06-2004
IP: Logged

"My Ratings" for President! Same story as the above posters: I rank holds once I'm thoroughly done with them, so I often forget doing the rating, and it gets annoying looking at every single hold thread on the boards.

michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged

Schik wrote:
You seem to think the ratings are aimed mostly towards the architects, while I think they're for prospective players. Sure, architects want to get high ratings, but the rating system wasn't invented to give architects a pat on the back - it was to give players a way to make a somewhat informed decision about what hold they should try next.

I wouldn't say mostly, but I do think that architects should be considered a little more by the rating system.

Rating something (2/10 or anything else) isn't meant to be constructive criticism. If you want to do that, then make a post in the hold's thread, and give as detailed criticism as you want to.

Well, I guess I'm way out there, but I'd prefer constructive criticism. It just seems obvious to me that free work goes hand in hand with constructive criticism, as opposed to a mere rating, where the architect doesn't necessarily know why someone doesn't like his hold. In other words, I don't like "give it 2/10, and *if you want to* tell the architect why". I prefer "If you really must give it 2/10, tell the architect why, and be nice about it. It's not your divine right that people should spend months building holds for you to criticise."