Announcement: Be excellent to each other.


Caravel Forum : Caravel Boards : The Site : Request: Show Hold Ratings with Bar Graph
New Topic New Poll Post Reply
Poster Message
CuriousShyRabbit
Level: Smitemaster
Rank Points: 3170
Registered: 10-14-2006
IP: Logged
icon Request: Show Hold Ratings with Bar Graph (+1)  
When I go to a thread on the Holds board, I see the average fun and difficulty rating for the hold. I also see the standard deviation of the fun ratings, which gives me an idea of their spread. I would like to see the spread of the fun ratings more visually, as a bar graph, just like the graphs that display the results of a poll.

The reason I make this request is that lately, average hold ratings have been sabotaged so they mean nothing at all. There's the problem with hitting "R" on the Change Location screen in game. And, I believe we also have an extrememly immature 1-bomber among us. About half the holds promoted this year, as well as some from last year have exactly one anomalous 1-rating. If we had the bar graph, the anomalous 1-ratings would stick out like sore thumbs, and architects and players alike would be able to see peoples' true appreciation of the holds.

While I'm not an architect myself, I feel this issue is especially important for architects. These days, it seems that very few people can find it in their hearts to leave comments on the Holds board (or in Architecture, but that's another story). The hold ratings are almost the only feedback architects are getting. Let's make sure they show what people really think.
07-22-2008 at 04:11 AM
View Profile Send Private Message to User Show all user's posts Quote Reply
zex20913
Level: Smitemaster
Avatar
Rank Points: 1723
Registered: 02-04-2003
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+1)  
According to a recent post in FR, mrimer says that the R key will be disassociated from the rating button in the next patch.

Whether we have a Uno-bomber (had to do it) remains another story, but one we'll be able to figure out soon enough.

____________________________
Click here to view the secret text

07-22-2008 at 01:10 PM
View Profile Send Private Message to User Send Email to User Show all user's posts Quote Reply
Syntax
Level: Smitemaster
Rank Points: 1218
Registered: 05-12-2005
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
zex20913 wrote:
Whether we have a Uno-bomber (had to do it) remains another story, but one we'll be able to figure out soon enough.
Well, it's possible to check who voted what (with admin powers) so it would be possible to find out... Whether that's time well spent is another matter. There'll always be this sort of thing going on.
07-22-2008 at 01:42 PM
View Profile Send Private Message to User Show all user's posts Quote Reply
Rheb
Level: Smitemaster
Avatar
Rank Points: 1211
Registered: 08-04-2006
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
CuriousShyRabbit wrote:
And, I believe we also have an extrememly immature 1-bomber among us. About half the holds promoted this year, as well as some from last year have exactly one anomalous 1-rating. If we had the bar graph, the anomalous 1-ratings would stick out like sore thumbs, and architects and players alike would be able to see peoples' true appreciation of the holds.
I'd vote for an admin to check this up, recently several of the top rated sank in average rating, and now have huge standard deviations.

The Goblin, Beethros Brain, Troshian Tower and Museum of Ooze are four examples, but I think there is more. :?

____________________________
Voligner is my very own DROD-like game. Please check it out!
07-22-2008 at 10:37 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores This architect's holds Quote Reply
Syntax
Level: Smitemaster
Rank Points: 1218
Registered: 05-12-2005
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
Rheb wrote:
I'd vote for an admin to check this up, recently several of the top rated sank in average rating, and now have huge standard deviations.
Agreed... unless we get more voters, any "unlikely" vote will skew things unfairly.
07-23-2008 at 12:01 AM
View Profile Send Private Message to User Show all user's posts Quote Reply
CuriousShyRabbit
Level: Smitemaster
Rank Points: 3170
Registered: 10-14-2006
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+2)  
As I said above, a large number of holds have been affected by these "erroneous" 1-ratings. In addition to the holds Rheb mentioned, they include Jutt's Salamander, calamarain's Pathways of Witherwood, calamarain's Symphony, jdyer's Floor 33, vittro's 39th Guard..., Chaco's Imperial Wandering, and that's just off the top of my head. It's pretty much everything recent on the Holds Board. As Syntax said, when newer holds are 1-bombed, it skews the ratings enormously.

Eventually, I hope all the "erroneous" ratings will be removed. Possibly this could be automated by removing all 1-ratings that are more than 2 standard deviations below the average. But it doesn't make sense to do all that work until everyone has the patch that prevents accidental 1-ratings. That's why, for now, I'd like to see the bar graphs, please. :)
07-23-2008 at 12:25 AM
View Profile Send Private Message to User Show all user's posts Quote Reply
Jatopian
Level: Smitemaster
Rank Points: 1842
Registered: 07-31-2005
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
CuriousShyRabbit wrote:
Possibly this could be automated by removing all 1-ratings that are more than 2 standard deviations below the average.
That's not really a good idea, as the standard deviation can vary widely before a hold has received enough ratings to have any kind of stability. Plus, hey, sometimes people really and truly feel that a popular hold deserves a 1, and it's their right to vote that way. I think Schik's stopgap should serve until the next patch, and then that should be the end of the technical solutions.

____________________________
DROD has some really great music.
Make your pressure plates 3.0 style!
DROD architecture idea generator
07-23-2008 at 12:37 PM
View Profile Send Private Message to User Show all user's posts This architect's holds Quote Reply
Rheb
Level: Smitemaster
Avatar
Rank Points: 1211
Registered: 08-04-2006
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+1)  
No, I agree with CuriousShyRabbit here, I don't see what the community could possibly gain at keeping all these random 1 votes, and if someone really hates a particular hold it isn't too hard to re vote it.

____________________________
Voligner is my very own DROD-like game. Please check it out!
07-23-2008 at 04:47 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores This architect's holds Quote Reply
Syntax
Level: Smitemaster
Rank Points: 1218
Registered: 05-12-2005
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
A potential solution would be to keep all the votes within a certain distance from the average. I seem to remember some maths formula using the standard deviation to only consider 95% of relevant votes. Approximating a bell curve or something...

That was a long time ago though :)
07-24-2008 at 06:43 AM
View Profile Send Private Message to User Show all user's posts Quote Reply
zex20913
Level: Smitemaster
Avatar
Rank Points: 1723
Registered: 02-04-2003
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
We could eliminate the outliers...but I think that leads to big-brother-ness far too often.

If somebody legitimately wants to vote 1 for a currently good hold or 10 for a currently bad one, why shouldn't they? Here's a situation that could exploit this as well: An architect makes a poor (but possible) hold, then gets 10 of his/her friends to rate it a 10. If we don't allow 1-votes, and the hold is deserving, the hold will be ranked highly even though it's potentially the worst hold in DROD.

Additionally, what do you do when the votes start trickling in? If there's an 8 and a 2, how do you deal with that?

There shouldn't be a system to automatically eliminate "different" votes. There should be (and now is, thanks to both Schik and mrimer) a system to automatically eliminate "Accidental" votes. I think that's sufficient.

Though I'm not opposed to the bar graph either. Perfect information FTW.

____________________________
Click here to view the secret text

07-24-2008 at 02:00 PM
View Profile Send Private Message to User Send Email to User Show all user's posts Quote Reply
Syntax
Level: Smitemaster
Rank Points: 1218
Registered: 05-12-2005
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
I wouldn't disallow a 1 vote.
I'd simply eliminate the 5% of votes which are furthest from the normal distribution.
07-24-2008 at 03:42 PM
View Profile Send Private Message to User Show all user's posts Quote Reply
zex20913
Level: Smitemaster
Avatar
Rank Points: 1723
Registered: 02-04-2003
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
You don't get a normal distribution until about 30 data points. I'm not sure if any of the holds have 30 votes.

Plus, I just don't like the idea one bit. The motto seems to be "conform or die".

____________________________
Click here to view the secret text

07-24-2008 at 05:11 PM
View Profile Send Private Message to User Send Email to User Show all user's posts Quote Reply
Moo
Level: Master Delver
Rank Points: 224
Registered: 10-14-2006
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
Does the forum know who voted what for what?
If so, when the 1-votes are cancelled, a forum mail could be generated automatically to whoever did the votes, to tell them what votes are affected.. Then if someone really did want to vote 1, they could do it again after the changes have been made. :cool
07-24-2008 at 10:42 PM
View Profile Send Private Message to User Show all user's posts Quote Reply
Oneiromancer
Level: Legendary Smitemaster
Avatar
Rank Points: 2936
Registered: 03-29-2003
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+1)  
The solution to single 1-votes affecting distributions is not to remove the 1-votes, but to increase the number of people voting, to really get the true distribution (as zex points out).

A graph isn't a bad idea, and it would be interesting from a nerd standpoint anyway, but it's not going to be shown on the holds table on the DROD website or through CNet, so it's still the average that is important.

____________________________
"He who is certain he knows the ending of things when he is only beginning them is either extremely wise or extremely foolish; no matter which is true, he is certainly an unhappy man, for he has put a knife in the heart of wonder." -- Tad Williams
07-24-2008 at 11:08 PM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
CuriousShyRabbit
Level: Smitemaster
Rank Points: 3170
Registered: 10-14-2006
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+1)  
Hmmm... I didn't mean to start an argument here. But yes, I'm a geek, and would really like to see the bar graphs, please. :)

However there are some points here I would like to address.

1. I'm 100% in favor of getting rid of the accidental votes of 0.5 brains, 1 fun. They cause hold ratings to be systematically lower. Their effects might be reduced by more honest votes, but they will never be "cancelled out" by more votes, because there isn't a way someone could "accidentally" rate a hold 10 brains, 10 fun. I'm a scientist, and when I know I've screwed up and introduced a systematic error (an error that always skews the result in the same direction) into my data, I throw that data out and repeat that part of the experiment.

1a. So then there's the question of how to get rid of the accidental votes of 0.5 brains, 1 fun. I think everyone would feel most comfortable if an admin (I wonder who that might be) did it by hand. They would pick through the ratings of the ~50 (?) affected holds, PM the people whose 0.5 brains 1 fun ratings look accidental, lovingly smooth everyone's feathers. But I would hate to see the accidental ratings left in there because time does not permit the "by hand" solution.

A one-time automated solution might be to throw out the 0.5 brains 1 fun ratings that are more than 2 standard deviations below the average. I realize that this solution would be unfair to some little twerp who honestly believes The Goblin or Troshian Tower or Imperial Wandering (or whatever) deserves a rating of 0.5 brains 1 fun. As Rheb said, that person would be inconvenienced by having to come back to the forum and recast their votes. The present situation (leaving the accidental low ratings in there) is also unfair, extremely unfair to the architects who made those holds. I don't think the effort it takes to cast a vote is in any way comparable to the effort it takes to make a hold.

2. I don't think we need to restrict people's votes in the future. I think any intentional vote should be allowed.

3. I do agree that more forum members should take the time to rate holds and comment on them. For a typical hold, after one month on the Holds board, more than 200 people will have downloaded it. But only 6-7 ratings and even fewer comments? That's not right. Come on, people! If you're going to partake of the many wonderful offerings on the holds board, you should at least give something back... a rating, a comment, or both. :)
07-25-2008 at 02:18 AM
View Profile Send Private Message to User Show all user's posts Quote Reply
silver
Level: Smitemaster
Rank Points: 915
Registered: 01-18-2005
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+1)  
Why not inconvenience the people who cast accidental votes, instead, and encourage them to recast their votes in ways which more validly reflect their opinions?

also - do you think the problem of accidental votes will magically go away when he unbinds "r" from "rate this hold" and moves the values to middle

a) not everyone will upgrade - the people least likely to upgrade are probably also most likely to accidentally hit 'r' and not care

b) if it's still a button, then people will still misclick and accidentally vote 5/5 - does that really help, either?

IMHO : the UI should be changed so neither a single keystroke nor a single button magically votes - the vote sliders should be on "undefined/no-vote" rather than "min" or "middle" - so if you click the button or hit 'r', then nothing happens unless you purposefully moved the slider off of "no vote".

but that only solves issue 'b'



____________________________
:yinyang
07-25-2008 at 02:25 AM
View Profile Send Private Message to User Show all user's posts This architect's holds Quote Reply
Tahnan
Level: Smitemaster
Avatar
Rank Points: 2459
Registered: 11-14-2005
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+2)  
silver wrote:
Why not inconvenience the people who cast accidental votes, instead, and encourage them to recast their votes in ways which more validly reflect their opinions?
Because the typical scenario--at least, every time I've accidentally rated a hold--runs as follows.

I download a hold, and it pops up on my screen. I hit "R" to restore, unthinkingly, and suddenly I've given the hold a 1/.5. And now: I can't really re-cast my vote. I can change my accidental rating to, say, 5/5, but I can't reflect my opinions because I don't yet have opinions; I haven't started the hold yet.

a) not everyone will upgrade - the people least likely to upgrade are probably also most likely to accidentally hit 'r' and not care

b) if it's still a button, then people will still misclick and accidentally vote 5/5 - does that really help, either?
For the former: am I right that only CaravelNet customers can rate in-game? If so, they're quite likely to upgrade, I think.

For the latter: I really, strongly suspect it's mistyping "R" and not misclicking that accounts for the vast majority of these votes. So yes, it really does help. Moreover, an accidental vote of 5/5 skews the results far less than an accidental vote of 1/.5, so even if an accidental vote goes through, it'll be better than it is now.

(That said, I agree that a default of no-vote would be better than 5/5. I also agree with CSR that the 1/.5 votes should be removed and if necessary recast.)
07-25-2008 at 08:44 AM
View Profile Send Private Message to User Show all user's posts High Scores This architect's holds Quote Reply
Tuttle
Level: Smitemaster
Avatar
Rank Points: 1545
Registered: 02-22-2003
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
(a) is a non-issue from now on because Schik posted in one of the other threads that the server now ignores 1/.5 votes submitted in-game. Non-upgraders who hit the button don't have any effect.

As for (b), I think it's a lot less of an issue -- I don't recall seeing anyone post about having accidentally mis-clicked the rating button, but several of us reported accidentally hitting R.
07-25-2008 at 04:00 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores Quote Reply
Schik
Level: Legendary Smitemaster
Avatar
Rank Points: 5383
Registered: 02-04-2003
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+5)  
I've implemented this. If you click on the normal yellow door graphic for hold rating, it will switch to a full graph of all votes, kinda like in normal polls. Click again and it will go back.


____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi
07-25-2008 at 07:13 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores Quote Reply
Dischorran
Level: Smitemaster
Avatar
Rank Points: 3407
Registered: 09-10-2005
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
Oh, very nice! Now I can obsessively check my hold's ratings just like I obsessively check my contest entries' ratings.

And now that this is up, it's notable that almost every hold has one or two 1 ratings.

____________________________
Click here to view the secret text

07-25-2008 at 07:25 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores This architect's holds Quote Reply
Schik
Level: Legendary Smitemaster
Avatar
Rank Points: 5383
Registered: 02-04-2003
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+4)  
Wow. I just found something very interesting... It was possible to rate a hold when not logged in. I had some protection in against that but it was broken and I guess I never tested it.

Most of the 1-votes were from the Guest account. I'll remove all of Guest's ratings.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi
07-25-2008 at 07:36 PM
View Profile Send Private Message to User Send Email to User Show all user's posts High Scores Quote Reply
CuriousShyRabbit
Level: Smitemaster
Rank Points: 3170
Registered: 10-14-2006
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
Schik wrote:
I've implemented this. If you click on the normal yellow door graphic for hold rating, it will switch to a full graph of all votes, kinda like in normal polls. Click again and it will go back.
Wow, these graphs look terrific! Thank you, Schik! I love graphs. I might spend a whole day just clicking on all of them...

And thank you also for removing Guest's ratings. That fixed a bunch of problems, including Salamander, Pathways of Witherwood, Floor 33, Desert Plateau, and others. But there are still quite a few suspicious-looking 1-ratings on the holds board. A random spot check shows that The Goblin, Imperial Wandering, Madness in the Spring, and Symphony are among those holds still affected. The graphs will allow me to construct a complete list of the 1-bombs.
07-25-2008 at 11:18 PM
View Profile Send Private Message to User Show all user's posts Quote Reply
noma
Level: Smitemaster
Avatar
Rank Points: 1594
Registered: 11-22-2006
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
Schik wrote:
...If you click on the normal yellow door graphic for hold rating...
I spent a while looking for the yellow "door", and then realized you meant yellow "bar."
07-25-2008 at 11:22 PM
View Profile Send Private Message to User Show all user's posts This architect's holds Quote Reply
zex20913
Level: Smitemaster
Avatar
Rank Points: 1723
Registered: 02-04-2003
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (+1)  
The door is a bar.

A poorly executed pun on "the door is ajar".

____________________________
Click here to view the secret text

07-26-2008 at 12:08 AM
View Profile Send Private Message to User Send Email to User Show all user's posts Quote Reply
rowrow
Level: Smiter
Rank Points: 432
Registered: 08-17-2004
IP: Logged
icon Re: Request: Show Hold Ratings with Bar Graph (0)  
noma wrote:
Schik wrote:
...If you click on the normal yellow door graphic for hold rating...
I spent a while looking for the yellow "door", and then realized you meant yellow "bar."


When's a door not a door?

Click here to view the secret text

:tomato

____________________________
B'hakhgra Du S'tra Moth'ness Ti!

[Last edited by rowrow at 07-27-2008 06:57 AM]
07-26-2008 at 03:46 AM
View Profile Send Private Message to User Send Email to User Show all user's posts This architect's holds Quote Reply
New Topic New Poll Post Reply
Caravel Forum : Caravel Boards : The Site : Request: Show Hold Ratings with Bar Graph
Surf To:


Forum Rules:
Can I post a new topic? No
Can I reply? No
Can I read? Yes
HTML Enabled? No
UBBC Enabled? Yes
Words Filter Enable? No

Contact Us | CaravelGames.com

Powered by: tForum tForumHacks Edition b0.98.8
Originally created by Toan Huynh (Copyright © 2000)
Enhanced by the tForumHacks team and the Caravel team.