Announcement: Be excellent to each other.


Caravel Forum : Caravel Boards : General : How do you rate holds? (What's a perfect 10?)
12
Page 3 of 3
New Topic New Poll Post Reply
Poster Message
michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged
icon Re: How do you rate holds? (+1)  
Banjooie wrote:
It's not your divine right that people should spend months building holds for you to criticise."

It's not your divine right that people should spend months building holds for you to rate it a 10.
That simply makes no sense at all. It's just an attempt at looking clever by lamely playing on words. You know, Banjooie, there's nothing clever about it. It's very easy to do. I, and many others, don't do it because it's cheap, low, annoying and serves no purpose, not because we're not "clever" enough.

Anyway, the point I'm making, in case you really didn't get it, is that you take good architects for granted. I don't, which doesn't mean that I'll rate all their holds a 10. It does mean that I don't turn their work into something all about *me* by seeing it as just another opportunity to try and look clever and funny by way of lame sarcastic comments. (And failing miserably at both, not to mention failing to make an actual contribution.)
It goes both ways, and you've yet to tell me why a high rating is inherently more justified than a low rating.
I never said that. Do you agree that holds should be rated on merit, and not, for instance, rated down because of a post made by someone other than the architect (or the architect, for that matter)?
09-29-2006 at 11:51 AM
View Profile Send Private Message to User Show all user's posts Quote Reply
Ezlo
Level: Smitemaster
Avatar
Rank Points: 1214
Registered: 01-08-2006
IP: Logged
icon Re: How do you rate holds? (0)  
Yes, I do. But I don't think we should vote higher because the guy put a lot of effort into it. That's unfair as well. Overall means the hold's overall, not the architect's overall.
09-29-2006 at 01:07 PM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged
icon Re: How do you rate holds? (0)  
michthro, I agree totally that we should encourage good architechs, I also agree that we should only give constructive criticism of hold, rather than just saying it sucks.

However, the hold rating is NOT for that. The hold rating is not for constructive criticism, nor for making Architech's feel good. The rating system is to help people find holds, with a difficulty or rating of their desire. I don't know how you managed to think otherwise.
There is a place for what you want, and that is the POST REPLY button, which is found almost everywhere. If you think that a hold shows promise, but the execution sucked, mark it honestly, rather than marking it what it should theoretically be if the execution was perfect, and then post a comment telling them why that is the case, which is real constructive criticism.

If everyone marked it like you do, hold ratings would mean absolutly NOTHING! is that what you want? To ruin the rating system?
09-29-2006 at 01:58 PM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
Ezlo
Level: Smitemaster
Avatar
Rank Points: 1214
Registered: 01-08-2006
IP: Logged
icon Re: How do you rate holds? (0)  
You've got it wrong, michthro votes fine. He just wants the guy who voted a two to change it. That's where this all started I think.
09-29-2006 at 02:01 PM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
Schik
Level: Legendary Smitemaster
Avatar
Rank Points: 5381
Registered: 02-04-2003
IP: Logged
icon Re: How do you rate holds? (+1)  
NiroZ wrote:
However, the hold rating is NOT for that. The hold rating is not for constructive criticism, nor for making Architech's feel good. The rating system is to help people find holds, with a difficulty or rating of their desire. I don't know how you managed to think otherwise.
There is a place for what you want, and that is the POST REPLY button, which is found almost everywhere. If you think that a hold shows promise, but the execution sucked, mark it honestly, rather than marking it what it should theoretically be if the execution was perfect, and then post a comment telling them why that is the case, which is real constructive criticism.
This is pretty much exactly what I've been saying.
If everyone marked it like you do, hold ratings would mean absolutly NOTHING! is that what you want? To ruin the rating system?
This, however, is simply bizarre. Maybe there was some communication problem at first, but michthro has made it clear many times since then that he doesn't want people to give extra bonus points for being a new architect, nor does he want people to replicate his vote. Don't try to turn this into a flamefest.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi
09-29-2006 at 02:03 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores Quote Reply
michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged
icon Re: How do you rate holds? (+1)  
Well, yes, I basically agree with everyone on rating in general. This issue was complicated by a separate incident in the TUC thread, which is I suppose where the misunderstanding came in. In particular, NiroZ, I'm actually the one who's been making the point that people shouldn't simply rate a hold 2 without using the post reply button, because that's just saying:"this hold sucks". Rate it 2 and give some constructive criticism.

Here's what I disagree with:
Banjooie's outrageous statement that anyone who's never played DROD has the "right" to vote as they please. Compared to that it would be nothing if, for instance, I:
-rated every hold containing a wubba horde 1, no matter how good I know the hold is otherwise. That's not only unfair, but it really wouldn't help players find good holds if everyone (or a lot of players, at least) did that kind of thing.
-decided it would be appropriate to rate Tim's holds, and by association, everyone with a "t", "i", or "m" in their names, by getting swacked out of my mind and taking it from there.

The other problem I have is that if the fun-factor is taken too literally, difficult holds would get a lot of low ratings, regardless of whether it's difficult for the right or wrong reasons (according to players' opinions), simply because getting stuck is no fun. Difficult holds should be rated knowing that they're meant to be difficult. Players who don't enjoy difficult holds don't need the overall rating to tell them they won't enjoy the hold. To try and cover possible misunderstandings:
-I'm not a "difficulty snob". I feel exactly the same about holds meant to be easy/medium/a story/artistic/whatever.
-I'm not saying difficult holds should be rated higher. Im saying they shouldn't be rated down purely because they're difficult. If you feel it's difficult for the wrong reasons, rate it down.

Otherwise I agree wholeheartedly.
09-29-2006 at 03:48 PM
View Profile Send Private Message to User Show all user's posts Quote Reply
AlefBet
Level: Smitemaster
Rank Points: 979
Registered: 07-16-2003
IP: Logged
icon Re: How do you rate holds? (+2)  
michthro wrote:
Do you agree that holds should be rated on merit, and not, for instance, rated down because of a post made by someone other than the architect (or the architect, for that matter)?
I don't think anyone here is defending the capricious type of vote that you're pointing out. I don't think it needs to be mentioned anymore because (1) we all agree this is a bad vote, (2) continuing to mention it will (I believe) serve as a red herring and divert the discussion without getting anywhere, and (3) whoever made this vote is either not reading this thread or entrenched and not going to change it no matter what is said (or already changed it and we didn't notice).
Here's what I disagree with:
Banjooie's outrageous statement that anyone who's never played DROD has the "right" to vote as they please. Compared to that it would be nothing if, for instance, I:
-rated every hold containing a wubba horde 1, no matter how good I know the hold is otherwise. That's not only unfair, but it really wouldn't help players find good holds if everyone (or a lot of players, at least) did that kind of thing.
-decided it would be appropriate to rate Tim's holds, and by association, everyone with a "t", "i", or "m" in their names, by getting swacked out of my mind and taking it from there.
If you really want to vote that way, I suppose I will defend your right to vote that way. Or rather, more to the point, even if I agree you don't have the right to vote that way, how do you propose we enforce more reasonable votes? Hopefully the number of people voting capriciously is a small enough percentage to not critically impact the overall average (which is one reason we need to encourage everyone with experience on the hold to vote on it).
The other problem I have is that if the fun-factor is taken too literally, difficult holds would get a lot of low ratings, regardless of whether it's difficult for the right or wrong reasons (according to players' opinions), simply because getting stuck is no fun. Difficult holds should be rated knowing that they're meant to be difficult. Players who don't enjoy difficult holds don't need the overall rating to tell them they won't enjoy the hold. To try and cover possible misunderstandings:
-I'm not a "difficulty snob". I feel exactly the same about holds meant to be easy/medium/a story/artistic/whatever.
-I'm not saying difficult holds should be rated higher. Im saying they shouldn't be rated down purely because they're difficult. If you feel it's difficult for the wrong reasons, rate it down.
Here I'm with you. Unfortunately, if you play a hold that isn't at your preferred level of difficulty, it can be rather hard to give it an evaluation based on how fun you think it would be to a person in the target audience. So you can either not vote at all or vote more subjectively. Either of these is fine with me. Hopefully most people who would vote the hold down because of its difficulty will choose to play something else in the first place so that those few who do play difficulty-inappropriate holds will not affect the overall quality score for the hold in question much.

Voting systems like this anticipate bizarre and illogical votes by averaging them over many more sensible votes. Hopefully we won't get stuck fixating on the one or two weird votes that invariably occur every so often.

____________________________
I was charged with conspiracy to commit jay-walking, and accessory to changing lanes without signaling after the fact :blush.

++Adam H. Peterson
09-29-2006 at 05:31 PM
View Profile Send Private Message to User Send Email to User Visit Homepage Show all user's posts Quote Reply
AtkinsSJ
Level: Master Delver
Avatar
Rank Points: 107
Registered: 12-23-2005
IP: Logged
icon Re: How do you rate holds? (0)  
What I find awkward, being not that good at DROD, is that generally harder holds have higher ratings, and easier holds lower ones. I can either go for higher ratings, and get horribly stuck, or play the not-so-well-rated holds, and get a bit bored. Players who like difficult holds need to think about how fun it is for the desired audience as much as not-so-good players.
09-29-2006 at 09:02 PM
View Profile Send Private Message to User Send Email to User Visit Homepage Show all user's posts Quote Reply
eytanz
Level: Smitemaster
Avatar
Rank Points: 2708
Registered: 02-05-2003
IP: Logged
icon Re: How do you rate holds? (+1)  
While I agree that it is true that the easier holds tend to be rated lower, I don't think this is entirely because of how people rate things, but to a large extent because people are putting more time and effort into building difficult holds, so there are just fewer good easy holds out there. This was discussed before (look at the thread for Beethro's Teacher, and the threads it spawned in this forum), and the situation is improving, but I think it's more a problem with how people build holds than with how they rate them.

____________________________
I got my avatar back! Yay!
09-29-2006 at 09:28 PM
View Profile Send Private Message to User Show all user's posts This architect's holds Quote Reply
Banjooie
Level: Smitemaster
Avatar
Rank Points: 1645
Registered: 12-12-2004
IP: Logged
icon Re: How do you rate holds? (-2)  
I still want to understand this doomsday scenario where all the new architects are getting downvoted off the site despite making obscenely good holds, when in this situation, /one guy/ didn't like a hold, so he rated it down.

09-29-2006 at 09:30 PM
View Profile Send Private Message to User Show all user's posts This architect's holds Quote Reply
eytanz
Level: Smitemaster
Avatar
Rank Points: 2708
Registered: 02-05-2003
IP: Logged
icon Re: How do you rate holds? (+2)  
Banjooie wrote:
I still want to understand this doomsday scenario where all the new architects are getting downvoted off the site despite making obscenely good holds, when in this situation, /one guy/ didn't like a hold, so he rated it down.

Banjooie - stop beating a dead horse, seriously. We've moved on past that point, most people discussing this have figured out what we agree on and what we disagree.

And in general, just because there's no huge catastrophe is no reason to avoid discussing what can be done better.

____________________________
I got my avatar back! Yay!
09-29-2006 at 09:32 PM
View Profile Send Private Message to User Show all user's posts This architect's holds Quote Reply
Ezlo
Level: Smitemaster
Avatar
Rank Points: 1214
Registered: 01-08-2006
IP: Logged
icon Re: How do you rate holds? (0)  
I think the scenario is more: Some guy was rude to change his vote to a 2 when people started to complain that it was maybe a tad unfair and that the voter should voice his opinion on whether or not it actually deserved the 4/2..
09-29-2006 at 09:32 PM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
eytanz
Level: Smitemaster
Avatar
Rank Points: 2708
Registered: 02-05-2003
IP: Logged
icon Re: How do you rate holds? (0)  
Ezlo wrote:
I think the scenario is more: Some guy was rude to change his vote to a 2 when people started to complain that it was maybe a tad unfair and that the voter should voice his opinion on whether or not it actually deserved the 4/2..

Look, there are several things going on in this discussion at once; one guy gave a low grade, someone complained, and then another guy gave a low grade. Some of the discussion (esp. by Michthro) was about the second guy who gave the low grade, other parts of the discussion (such as most of my posts) were about the first guy. This partially led to some misunderstandings, and partially the discussion evolved into an interesting debate on rating philosophy. But at this point, the specific series of events that led to the discussion starting in the first place are really not relevant anymore and it makes no sense to keep bringing them up.

____________________________
I got my avatar back! Yay!

[Last edited by eytanz at 09-29-2006 09:40 PM]
09-29-2006 at 09:39 PM
View Profile Send Private Message to User Show all user's posts This architect's holds Quote Reply
Ezlo
Level: Smitemaster
Avatar
Rank Points: 1214
Registered: 01-08-2006
IP: Logged
icon Re: How do you rate holds? (0)  
Ah, okay. As I was posting, your post came up, so I didn't see yours. But yours sort of makes mine irrelavent. Sorry.
09-29-2006 at 09:43 PM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged
icon Re: How do you rate holds? (0)  
AtkinsSJ wrote:
Players who like difficult holds need to think about how fun it is for the desired audience as much as not-so-good players.
You have a point when it comes to SmS holds, but that was discussed to death, and iirc sorted out, in the BT thread and others spawning from it. Otherwise, sorry, but no. Why? User-made holds are often simply not aimed at everyone, and there's nothing wrong with that. There is a wide enough variety of holds that just about anyone should have enough to keep them busy, regardless of their taste.

Difficulty is not the only thing that's a matter of taste. I generally don't like heavily scripted holds, but if I want to be warned about that I have to read the thread. If I'm lucky, the hold description indicates that it is such a hold. Otherwise I only know that once I start playing it. (In which case I delete it and play something else. It's not to my taste, so I don't care how good or bad it is for a heavily scripted hold.) So consider yourself lucky that at least you only have to look at the difficulty rating for a warning.
09-29-2006 at 11:42 PM
View Profile Send Private Message to User Show all user's posts Quote Reply
NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged
icon Re: How do you rate holds? (0)  
I see that everyone beat me to my ideas, and I think I agree with most of you.

The only thing that I could possibly add is to point out that my "tagging" idea would be perfect to help people find holds to their desires.
09-30-2006 at 01:43 AM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
Mattcrampy
Level: Smitemaster
Avatar
Rank Points: 2388
Registered: 05-29-2003
IP: Logged
icon Re: How do you rate holds? (0)  
michthro wrote:
It does mean that I don't turn their work into something all about *me* by seeing it as just another opportunity to try and look clever and funny by way of lame sarcastic comments. (And failing miserably at both, not to mention failing to make an actual contribution.)

michthro: let's not get personal here. Nothing will get anywhere by making snide remarks about other people.

While I could certainly make comments up the wazoo about what I intended when Schik and I created the system (back when I was useful), two years on it's what people do with the system that is important. We'll never get everyone following a proscribed set of rules, but so long as everyone is being internally consistent it'll still work out as useful.

I'm not really sure what we're supposed to do about one person who changes their vote because someone else has posted something they don't agree with, other than register our disapproval as a community. We could have the voting records public and marked but that may lead to creators pressuring people to increase their votes, or more subtlely pestering people who voted it down about what they did wrong and how they can improve. I don't think it's fair on people to have to explain why they voted low to the creator, and it may lead to people voting higher just to avoid the scrutiny.

____________________________
What do you call an elephant at the North Pole?
Click here to view the secret text


[Last edited by Mattcrampy at 10-02-2006 07:17 AM]
10-02-2006 at 06:35 AM
View Profile Send Private Message to User Send Email to User Show all user's posts Quote Reply
Mattcrampy
Level: Smitemaster
Avatar
Rank Points: 2388
Registered: 05-29-2003
IP: Logged
icon Re: How do you rate holds? (0)  
eytanz wrote:
While I agree that it is true that the easier holds tend to be rated lower, I don't think this is entirely because of how people rate things, but to a large extent because people are putting more time and effort into building difficult holds, so there are just fewer good easy holds out there.

Double post!

Erik has said in the past that it's harder to build an easy hold that's interesting than a hard hold. I'm sure most of us would agree.

____________________________
What do you call an elephant at the North Pole?
Click here to view the secret text

10-02-2006 at 07:30 AM
View Profile Send Private Message to User Send Email to User Show all user's posts Quote Reply
Schik
Level: Legendary Smitemaster
Avatar
Rank Points: 5381
Registered: 02-04-2003
IP: Logged
icon Re: How do you rate holds? (0)  
Mattcrampy wrote:
While I could certainly make comments up the wazoo about what I intended when Schik and I created the system
My memory must be going... I don't remember the hold rating system being a Matt/Matt creation. However, all the talk about what "Overall" is supposed to mean sent me on a quest to find the original discussion of it, to see if anybody involved was more verbose about what it should mean. There's this topic, where I asked the public what they thought we should rate holds by. I also mentioned that there had been some private discussion already. This was about a month & a half before Mattcrampy took the reins from Malarame, though Cramp was certainly around at this point. Then there's the topic where I announced hold uploading was ready, about 4 months later.

I couldn't find any emails or anything with this private discussion I mentioned. If any of you who were involved in that discussion have any record of it, I'd love to get a copy to look over it again.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi
10-02-2006 at 08:25 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores Quote Reply
schep
Level: Smitemaster
Avatar
Rank Points: 864
Registered: 03-01-2005
IP: Logged
icon Re: How do you rate holds? (+1)  
I will keep this brief, since the thread is longer than I like 'em already.

First, I am among those who will rate a higher proportion of holds I conquer when all my ratings are visible and changeable from one place (whether in TCB or on a web page doesn't matter).

I think an overall "fun" rating works well enough, but don't really feel like defending it. "Tags" on holds might be an interesting thing to play with.

An interesting look on how another geeky community presents some rating statistics: [url http://www.ifcomp.org/comp05/results.html (OT plug: the '06 text games on that site are now available.)
10-03-2006 at 01:34 AM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
Mattcrampy
Level: Smitemaster
Avatar
Rank Points: 2388
Registered: 05-29-2003
IP: Logged
icon Re: How do you rate holds? (0)  
Schik wrote:
My memory must be going... I don't remember the hold rating system being a Matt/Matt creation.

You sure it's not my memory? I was sure I discussed it with you. I know I did do something with hooking it up on the front end.

Edit: It appears that Neil came up with the difficulty idea. I doubt I did very much at all.

____________________________
What do you call an elephant at the North Pole?
Click here to view the secret text


[Last edited by Mattcrampy at 10-04-2006 03:45 AM]
10-04-2006 at 02:51 AM
View Profile Send Private Message to User Send Email to User Show all user's posts Quote Reply
Oneiromancer
Level: Legendary Smitemaster
Avatar
Rank Points: 2936
Registered: 03-29-2003
IP: Logged
icon Re: How do you rate holds? (0)  
My memory on this issue was pretty spotty too...I definitely remember helping to test the original Holds board and hold upload things and making many suggestions, but I also remember that you (Cramp) did some design and had input too.

But then I do still happen to have some old PMs from that time (almost 3 years ago now) and in one of them I say to Schik:
Finally, I think rating difficulty out of 5 brains is tough, even with half-brains. You might want to go up to 10 just like for user rating. I think you should have these columns: author's difficulty, player's difficulty, and player's rating. These are all things worth sorting the holds by, and I think it could be really interesting to see how the opinions on how difficult the hold is change between the author and the players. Hopefully it wouldn't be too difficult to add another rating to the forum page, since it appears to be different from a poll.
So obviously some changes were made (there is no column for author's difficulty on the main Holds page, for example) but I guess I did have some input on the current system after all. Thanks for the nod, Mr. Cramp!

Game on,

____________________________
"He who is certain he knows the ending of things when he is only beginning them is either extremely wise or extremely foolish; no matter which is true, he is certainly an unhappy man, for he has put a knife in the heart of wonder." -- Tad Williams
10-04-2006 at 04:47 AM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
12
Page 3 of 3
New Topic New Poll Post Reply
Caravel Forum : Caravel Boards : General : How do you rate holds? (What's a perfect 10?)
Surf To:


Forum Rules:
Can I post a new topic? No
Can I reply? No
Can I read? Yes
HTML Enabled? No
UBBC Enabled? Yes
Words Filter Enable? No

Contact Us | CaravelGames.com

Powered by: tForum tForumHacks Edition b0.98.8
Originally created by Toan Huynh (Copyright © 2000)
Enhanced by the tForumHacks team and the Caravel team.