michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged

Banjooie wrote:

It's not your divine right that people should spend months building holds for you to criticise."

It's not your divine right that people should spend months building holds for you to rate it a 10.

That simply makes no sense at all. It's just an attempt at looking clever by lamely playing on words. You know, Banjooie, there's nothing clever about it. It's very easy to do. I, and many others, don't do it because it's cheap, low, annoying and serves no purpose, not because we're not "clever" enough.

Anyway, the point I'm making, in case you really didn't get it, is that you take good architects for granted. I don't, which doesn't mean that I'll rate all their holds a 10. It does mean that I don't turn their work into something all about *me* by seeing it as just another opportunity to try and look clever and funny by way of lame sarcastic comments. (And failing miserably at both, not to mention failing to make an actual contribution.)

It goes both ways, and you've yet to tell me why a high rating is inherently more justified than a low rating.

I never said that. Do you agree that holds should be rated on merit, and not, for instance, rated down because of a post made by someone other than the architect (or the architect, for that matter)?

Citrus
Level: Smitemaster

Rank Points: 1233
Registered: 01-08-2006
IP: Logged

Yes, I do. But I don't think we should vote higher because the guy put a lot of effort into it. That's unfair as well. Overall means the hold's overall, not the architect's overall.

____________________________
Call me Citrus.

NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged

michthro, I agree totally that we should encourage good architechs, I also agree that we should only give constructive criticism of hold, rather than just saying it sucks.

However, the hold rating is NOT for that. The hold rating is not for constructive criticism, nor for making Architech's feel good. The rating system is to help people find holds, with a difficulty or rating of their desire. I don't know how you managed to think otherwise.
There is a place for what you want, and that is the POST REPLY button, which is found almost everywhere. If you think that a hold shows promise, but the execution sucked, mark it honestly, rather than marking it what it should theoretically be if the execution was perfect, and then post a comment telling them why that is the case, which is real constructive criticism.

If everyone marked it like you do, hold ratings would mean absolutly NOTHING! is that what you want? To ruin the rating system?

Citrus
Level: Smitemaster

Rank Points: 1233
Registered: 01-08-2006
IP: Logged

You've got it wrong, michthro votes fine. He just wants the guy who voted a two to change it. That's where this all started I think.

____________________________
Call me Citrus.

Schik
Level: Legendary Smitemaster

Rank Points: 5444
Registered: 02-04-2003
IP: Logged

NiroZ wrote:
However, the hold rating is NOT for that. The hold rating is not for constructive criticism, nor for making Architech's feel good. The rating system is to help people find holds, with a difficulty or rating of their desire. I don't know how you managed to think otherwise.
There is a place for what you want, and that is the POST REPLY button, which is found almost everywhere. If you think that a hold shows promise, but the execution sucked, mark it honestly, rather than marking it what it should theoretically be if the execution was perfect, and then post a comment telling them why that is the case, which is real constructive criticism.

This is pretty much exactly what I've been saying.

If everyone marked it like you do, hold ratings would mean absolutly NOTHING! is that what you want? To ruin the rating system?

This, however, is simply bizarre. Maybe there was some communication problem at first, but michthro has made it clear many times since then that he doesn't want people to give extra bonus points for being a new architect, nor does he want people to replicate his vote. Don't try to turn this into a flamefest.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged

Well, yes, I basically agree with everyone on rating in general. This issue was complicated by a separate incident in the TUC thread, which is I suppose where the misunderstanding came in. In particular, NiroZ, I'm actually the one who's been making the point that people shouldn't simply rate a hold 2 without using the post reply button, because that's just saying:"this hold sucks". Rate it 2 and give some constructive criticism.

Here's what I disagree with:
Banjooie's outrageous statement that anyone who's never played DROD has the "right" to vote as they please. Compared to that it would be nothing if, for instance, I:
-rated every hold containing a wubba horde 1, no matter how good I know the hold is otherwise. That's not only unfair, but it really wouldn't help players find good holds if everyone (or a lot of players, at least) did that kind of thing.
-decided it would be appropriate to rate Tim's holds, and by association, everyone with a "t", "i", or "m" in their names, by getting swacked out of my mind and taking it from there.

The other problem I have is that if the fun-factor is taken too literally, difficult holds would get a lot of low ratings, regardless of whether it's difficult for the right or wrong reasons (according to players' opinions), simply because getting stuck is no fun. Difficult holds should be rated knowing that they're meant to be difficult. Players who don't enjoy difficult holds don't need the overall rating to tell them they won't enjoy the hold. To try and cover possible misunderstandings:
-I'm not a "difficulty snob". I feel exactly the same about holds meant to be easy/medium/a story/artistic/whatever.
-I'm not saying difficult holds should be rated higher. Im saying they shouldn't be rated down purely because they're difficult. If you feel it's difficult for the wrong reasons, rate it down.

Otherwise I agree wholeheartedly.

AlefBet
Level: Smitemaster
Rank Points: 979
Registered: 07-16-2003
IP: Logged

michthro wrote:
Do you agree that holds should be rated on merit, and not, for instance, rated down because of a post made by someone other than the architect (or the architect, for that matter)?

I don't think anyone here is defending the capricious type of vote that you're pointing out. I don't think it needs to be mentioned anymore because (1) we all agree this is a bad vote, (2) continuing to mention it will (I believe) serve as a red herring and divert the discussion without getting anywhere, and (3) whoever made this vote is either not reading this thread or entrenched and not going to change it no matter what is said (or already changed it and we didn't notice).

Here's what I disagree with:
Banjooie's outrageous statement that anyone who's never played DROD has the "right" to vote as they please. Compared to that it would be nothing if, for instance, I:
-rated every hold containing a wubba horde 1, no matter how good I know the hold is otherwise. That's not only unfair, but it really wouldn't help players find good holds if everyone (or a lot of players, at least) did that kind of thing.
-decided it would be appropriate to rate Tim's holds, and by association, everyone with a "t", "i", or "m" in their names, by getting swacked out of my mind and taking it from there.

If you really want to vote that way, I suppose I will defend your right to vote that way. Or rather, more to the point, even if I agree you don't have the right to vote that way, how do you propose we enforce more reasonable votes? Hopefully the number of people voting capriciously is a small enough percentage to not critically impact the overall average (which is one reason we need to encourage everyone with experience on the hold to vote on it).

The other problem I have is that if the fun-factor is taken too literally, difficult holds would get a lot of low ratings, regardless of whether it's difficult for the right or wrong reasons (according to players' opinions), simply because getting stuck is no fun. Difficult holds should be rated knowing that they're meant to be difficult. Players who don't enjoy difficult holds don't need the overall rating to tell them they won't enjoy the hold. To try and cover possible misunderstandings:
-I'm not a "difficulty snob". I feel exactly the same about holds meant to be easy/medium/a story/artistic/whatever.
-I'm not saying difficult holds should be rated higher. Im saying they shouldn't be rated down purely because they're difficult. If you feel it's difficult for the wrong reasons, rate it down.

Here I'm with you. Unfortunately, if you play a hold that isn't at your preferred level of difficulty, it can be rather hard to give it an evaluation based on how fun you think it would be to a person in the target audience. So you can either not vote at all or vote more subjectively. Either of these is fine with me. Hopefully most people who would vote the hold down because of its difficulty will choose to play something else in the first place so that those few who do play difficulty-inappropriate holds will not affect the overall quality score for the hold in question much.

Voting systems like this anticipate bizarre and illogical votes by averaging them over many more sensible votes. Hopefully we won't get stuck fixating on the one or two weird votes that invariably occur every so often.

____________________________
I was charged with conspiracy to commit jay-walking, and accessory to changing lanes without signaling after the fact .

++Adam H. Peterson

AtkinsSJ
Level: Master Delver

Rank Points: 107
Registered: 12-23-2005
IP: Logged

What I find awkward, being not that good at DROD, is that generally harder holds have higher ratings, and easier holds lower ones. I can either go for higher ratings, and get horribly stuck, or play the not-so-well-rated holds, and get a bit bored. Players who like difficult holds need to think about how fun it is for the desired audience as much as not-so-good players.

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

While I agree that it is true that the easier holds tend to be rated lower, I don't think this is entirely because of how people rate things, but to a large extent because people are putting more time and effort into building difficult holds, so there are just fewer good easy holds out there. This was discussed before (look at the thread for Beethro's Teacher, and the threads it spawned in this forum), and the situation is improving, but I think it's more a problem with how people build holds than with how they rate them.

____________________________
I got my avatar back! Yay!

Banjooie
Level: Smitemaster

Rank Points: 1645
Registered: 12-12-2004
IP: Logged

I still want to understand this doomsday scenario where all the new architects are getting downvoted off the site despite making obscenely good holds, when in this situation, /one guy/ didn't like a hold, so he rated it down.

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

Banjooie wrote:
I still want to understand this doomsday scenario where all the new architects are getting downvoted off the site despite making obscenely good holds, when in this situation, /one guy/ didn't like a hold, so he rated it down.

Banjooie - stop beating a dead horse, seriously. We've moved on past that point, most people discussing this have figured out what we agree on and what we disagree.

And in general, just because there's no huge catastrophe is no reason to avoid discussing what can be done better.

____________________________
I got my avatar back! Yay!

Citrus
Level: Smitemaster

Rank Points: 1233
Registered: 01-08-2006
IP: Logged

I think the scenario is more: Some guy was rude to change his vote to a 2 when people started to complain that it was maybe a tad unfair and that the voter should voice his opinion on whether or not it actually deserved the 4/2..

____________________________
Call me Citrus.

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

Ezlo wrote:
I think the scenario is more: Some guy was rude to change his vote to a 2 when people started to complain that it was maybe a tad unfair and that the voter should voice his opinion on whether or not it actually deserved the 4/2..

Look, there are several things going on in this discussion at once; one guy gave a low grade, someone complained, and then another guy gave a low grade. Some of the discussion (esp. by Michthro) was about the second guy who gave the low grade, other parts of the discussion (such as most of my posts) were about the first guy. This partially led to some misunderstandings, and partially the discussion evolved into an interesting debate on rating philosophy. But at this point, the specific series of events that led to the discussion starting in the first place are really not relevant anymore and it makes no sense to keep bringing them up.

____________________________
I got my avatar back! Yay!

---

[Last edited by eytanz at 09-29-2006 09:40 PM]

Citrus
Level: Smitemaster

Rank Points: 1233
Registered: 01-08-2006
IP: Logged

Ah, okay. As I was posting, your post came up, so I didn't see yours. But yours sort of makes mine irrelavent. Sorry.

____________________________
Call me Citrus.

michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged

AtkinsSJ wrote:
Players who like difficult holds need to think about how fun it is for the desired audience as much as not-so-good players.

You have a point when it comes to SmS holds, but that was discussed to death, and iirc sorted out, in the BT thread and others spawning from it. Otherwise, sorry, but no. Why? User-made holds are often simply not aimed at everyone, and there's nothing wrong with that. There is a wide enough variety of holds that just about anyone should have enough to keep them busy, regardless of their taste.

Difficulty is not the only thing that's a matter of taste. I generally don't like heavily scripted holds, but if I want to be warned about that I have to read the thread. If I'm lucky, the hold description indicates that it is such a hold. Otherwise I only know that once I start playing it. (In which case I delete it and play something else. It's not to my taste, so I don't care how good or bad it is for a heavily scripted hold.) So consider yourself lucky that at least you only have to look at the difficulty rating for a warning.

NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged

I see that everyone beat me to my ideas, and I think I agree with most of you.

The only thing that I could possibly add is to point out that my "tagging" idea would be perfect to help people find holds to their desires.

Mattcrampy
Level: Smitemaster

Rank Points: 2388
Registered: 05-29-2003
IP: Logged

michthro wrote:
It does mean that I don't turn their work into something all about *me* by seeing it as just another opportunity to try and look clever and funny by way of lame sarcastic comments. (And failing miserably at both, not to mention failing to make an actual contribution.)

michthro: let's not get personal here. Nothing will get anywhere by making snide remarks about other people.

While I could certainly make comments up the wazoo about what I intended when Schik and I created the system (back when I was useful), two years on it's what people do with the system that is important. We'll never get everyone following a proscribed set of rules, but so long as everyone is being internally consistent it'll still work out as useful.

I'm not really sure what we're supposed to do about one person who changes their vote because someone else has posted something they don't agree with, other than register our disapproval as a community. We could have the voting records public and marked but that may lead to creators pressuring people to increase their votes, or more subtlely pestering people who voted it down about what they did wrong and how they can improve. I don't think it's fair on people to have to explain why they voted low to the creator, and it may lead to people voting higher just to avoid the scrutiny.

____________________________
What do you call an elephant at the North Pole?

---

[Last edited by Mattcrampy at 10-02-2006 07:17 AM]

Mattcrampy
Level: Smitemaster

Rank Points: 2388
Registered: 05-29-2003
IP: Logged

eytanz wrote:
While I agree that it is true that the easier holds tend to be rated lower, I don't think this is entirely because of how people rate things, but to a large extent because people are putting more time and effort into building difficult holds, so there are just fewer good easy holds out there.

Double post!

Erik has said in the past that it's harder to build an easy hold that's interesting than a hard hold. I'm sure most of us would agree.

____________________________
What do you call an elephant at the North Pole?

Schik
Level: Legendary Smitemaster

Rank Points: 5444
Registered: 02-04-2003
IP: Logged

Mattcrampy wrote:
While I could certainly make comments up the wazoo about what I intended when Schik and I created the system

My memory must be going... I don't remember the hold rating system being a Matt/Matt creation. However, all the talk about what "Overall" is supposed to mean sent me on a quest to find the original discussion of it, to see if anybody involved was more verbose about what it should mean. There's this topic, where I asked the public what they thought we should rate holds by. I also mentioned that there had been some private discussion already. This was about a month & a half before Mattcrampy took the reins from Malarame, though Cramp was certainly around at this point. Then there's the topic where I announced hold uploading was ready, about 4 months later.

I couldn't find any emails or anything with this private discussion I mentioned. If any of you who were involved in that discussion have any record of it, I'd love to get a copy to look over it again.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

schep
Level: Smitemaster

Rank Points: 865
Registered: 03-01-2005
IP: Logged

I will keep this brief, since the thread is longer than I like 'em already.

First, I am among those who will rate a higher proportion of holds I conquer when all my ratings are visible and changeable from one place (whether in TCB or on a web page doesn't matter).

I think an overall "fun" rating works well enough, but don't really feel like defending it. "Tags" on holds might be an interesting thing to play with.

An interesting look on how another geeky community presents some rating statistics: [url http://www.ifcomp.org/comp05/results.html (OT plug: the '06 text games on that site are now available.)

Mattcrampy
Level: Smitemaster

Rank Points: 2388
Registered: 05-29-2003
IP: Logged

Schik wrote:
My memory must be going... I don't remember the hold rating system being a Matt/Matt creation.

You sure it's not my memory? I was sure I discussed it with you. I know I did do something with hooking it up on the front end.

Edit: It appears that Neil came up with the difficulty idea. I doubt I did very much at all.

____________________________
What do you call an elephant at the North Pole?

---

[Last edited by Mattcrampy at 10-04-2006 03:45 AM]

Oneiromancer
Level: Legendary Smitemaster

Rank Points: 2936
Registered: 03-29-2003
IP: Logged

My memory on this issue was pretty spotty too...I definitely remember helping to test the original Holds board and hold upload things and making many suggestions, but I also remember that you (Cramp) did some design and had input too.

But then I do still happen to have some old PMs from that time (almost 3 years ago now) and in one of them I say to Schik:

Finally, I think rating difficulty out of 5 brains is tough, even with half-brains. You might want to go up to 10 just like for user rating. I think you should have these columns: author's difficulty, player's difficulty, and player's rating. These are all things worth sorting the holds by, and I think it could be really interesting to see how the opinions on how difficult the hold is change between the author and the players. Hopefully it wouldn't be too difficult to add another rating to the forum page, since it appears to be different from a poll.

So obviously some changes were made (there is no column for author's difficulty on the main Holds page, for example) but I guess I did have some input on the current system after all. Thanks for the nod, Mr. Cramp!

Game on,

____________________________
"He who is certain he knows the ending of things when he is only beginning them is either extremely wise or extremely foolish; no matter which is true, he is certainly an unhappy man, for he has put a knife in the heart of wonder." -- Tad Williams