mrimer
Level: Legendary Smitemaster

Rank Points: 5194
Registered: 02-04-2003
IP: Logged

wallu wrote:
My fear has some time been that some people flood crappy holds to holds board. I think that there must be some kind of system to prevent that kind of behaviour.

I'll add my thoughts on this and other issues mentioned here, and also some other stuff. I am not satisfied with having many low-quality (and/or "unfinished") holds published. Still, I don't think it would promote community growth in the long run to prevent people from publishing their holds whenever some users, even admins, think they are "bad" holds. I myself would much rather have the privilege to post a hold I make, even if it gets a low rating from players and/or admins, rather than being forbidden to post until I get a certain number of people who "like" my hold. Conversely, I also don't think some usual registered user, Mr. X, should have the power to vote another user's hold off the forum or relegate it to some lower status.

We are working to provide user holds of uniformly high-quality, and that service is available through CaravelNet. For non-CaravelNet holds, providing more organizational (i.e., weeding) tools to users should be helpful. On the forum, users already have the option to view and sort holds by rating, difficulty, and popularity. I'm planning on adding a similar feature to the in-game CaravelNet interface in the 3.0 engine. I'm not sure what more can be done to help the player find fun and high-quality holds while easily avoiding low-quality holds.

We are interested in giving users the power to "go forth and create!" even if the users in question don't do a professional job as some others do. That being said, Caravel does reserve the right to delete offensive or inappropriate holds published on the Caravel Forum. Also, with authors' agreement, edit holds (e.g. repair broken holds). Our goal is to keep things running smoothly, encouraging everyone to follow good priniciples and with a mimimum of policing.

____________________________
Gandalf? Yes... That's what they used to call me.
Gandalf the Grey. That was my name.
I am Gandalf the White.
And I come back to you now at the turn of the tide.

---

[Last edited by mrimer at 05-18-2008 05:52 AM]

ErikH2000
Level: Legendary Smitemaster

Rank Points: 2796
Registered: 02-04-2003
IP: Logged

Tim wrote:
Let me just say that I can imagine Danforth Strout saying: "Let's keep them. Look! Our current rooms are as large as the whole of North America, and the fact that you can't reach half of them is another convenience, because you don't need to do them!"

I think you're joking, Tim, but it's never been a motivation to boost our level count with shoddy holds. We've got over 8,000 rooms at the moment. I would be just as happy if we had half that. It's still a ridiculously large number.

-Erik


____________________________
The Godkiller - Chapter 1 available now on Steam. It's a DROD-like puzzle adventure game.
dev journals | twitch stream | youtube archive (NSFW)

Tim
Level: Smitemaster

Rank Points: 1979
Registered: 08-07-2004
IP: Logged

Just in case, I was joking.

Sorry for the inconvenience caused.

Edit: I have remove the joke.

---

[Last edited by Tim at 05-08-2006 11:41 PM]

Tuttle
Level: Smitemaster

Rank Points: 1548
Registered: 02-22-2003
IP: Logged

Oneiromancer wrote:
In general the problem is people submitting holds without going through the Architecture or Beta-testing at all. So first we have to decide how we are going to deal with them before worrying about how we deal with holds that actually do get tested.

<snip>

Perhaps another solution would be for there to be a way to directly upload a hold from the CaravelNet beta-testing forums to the Holds forum, and then for those that don't do that, to submit it directly to a "holding pen". If the hold was already on the Architecture forum, then the mod/admin can check to see if it was solveable pretty quickly and then confirm the upload. If not, then they can either try to do it themselves (meh) or ask for a set of demos or other proof that the hold is solveable (which is a pain).

I think that's hit the root of the problem, but the solution there may even be too manual. Why not implement something like "Any architect with less than x holds can't upload directly to the Holds board, but needs to have the hold in Architecture for y days first and then promote it from there."

x and y would need to be picked, but I suspect after much discussion that x would be 1, 2 or infinity and y would be in the order of days or weeks.

You'd still have a risk that someone would upload to Architecture, get people pointing out that it's unsolvable or badly broken and promote it anyway. I think that'd be unlikely though -- I'd guess the problem is more overconfidence in one's architecture skills than malicious intent.

AlefBet
Level: Smitemaster
Rank Points: 979
Registered: 07-16-2003
IP: Logged

I think that perhaps one small (and easy?) tweak that could be done is to put the Holds and Architecture forums next to each other on the list. The Holds forum should probably still be on top, just because players that never become Architects would probably care more about the Holds forum (and if a newbie comes to the forum, we'd probably rather showcase the holds in Holds than in Architecture). But if they were next to each other, the connection would probably be more natural. As it is, a new or lazy reader will get down to "holds" and see a list of holds people have made, and then go on to "hints and solutions." By the time they get to "bugs" their eyes glaze over and they don't ever actually see "architecture."

I'm just sayin'.

____________________________
I was charged with conspiracy to commit jay-walking, and accessory to changing lanes without signaling after the fact .

++Adam H. Peterson

Oneiromancer
Level: Legendary Smitemaster

Rank Points: 2936
Registered: 03-29-2003
IP: Logged

(Re: Tuttle's post.) I don't like that solution of mine anymore and now I prefer the latter one, wherein holds that don't pass muster simply aren't listed in-game or on the drod.net holds page. The way I would do it if I have to manage it somehow, is to do a survey of the non-approved holds once a week or so. If the holds have been completed, or were strongly tested on the Architecture or CaravelNet Beta Boards, then I flip a switch on the admin page and voila, it is now fully listed. Any hold that is found to not be completeable would not be listed until it is fixed. This doesn't have an arbitrary cutoff of how many holds you have released, it just relies on a simple "is it beatable or not?" question. And waiting 1-2 weeks for a hold to be listed in-game or on the drod.net holds page isn't a big deal; people can still download it the normal way. But there is at least a guarantee that holds listed in certain places are completeable, even if they don't have high fun ratings.

I agree with Adam too, I don't know why we haven't done it yet. But I need to talk to Schik so he can remind me if we have a really good reason for that or not. It shouldn't be that hard to do. /shrug

Game on,

____________________________
"He who is certain he knows the ending of things when he is only beginning them is either extremely wise or extremely foolish; no matter which is true, he is certainly an unhappy man, for he has put a knife in the heart of wonder." -- Tad Williams

---

[Last edited by Oneiromancer at 05-09-2006 12:12 AM]

Andy101
Level: Master Delver
Rank Points: 222
Registered: 07-22-2005
IP: Logged

My main worry with the hold situation as it is is the impression that it is giving to budding new DRODers. When I first started playing DROD one of the things that really attracted me to the game were the user made holds and there were some real gems out there. However I am worried that with the holds situation as it is that a new DROD player who comes along and is looking for some relatively easy holds to play has an all too high chance of coming across a shabby hold and getting a bad impression. As an experienced player I know that there are lots of good holds out there but they won't.

I'm not entirely sure what would be a good way to sort this out but I think that we should at least put together a list of good quality holds of varying difficulty to get people started. This list could be put in a prominent position on the website and forum. I know that at the moment it is possibly to sort holds by rating but if you do this the highly rated holds are more than likely to be hard holds. Even then this feature is only available on the website (and not particularly easy to find) and anyone who searchs the forums hold board for new holds to play can have a hard job finding a suitable one.

____________________________
"True perfection has to be imperfect"

---

[Last edited by Andy101 at 05-09-2006 12:11 AM]

miketo
Level: Delver

Rank Points: 61
Registered: 01-04-2006
IP: Logged

At the risk of injecting class warfare into the forums, what about adding a tiered classification for the builders, rather than the holds? You can have the "Apprentice" level and the "Master Builder" level. To be a Master Builder you need to post three holds that after a month have received a rating of 5 or better (or some other suitable, reasonably-valid criteria).

The chief benefit is that you can tell at a glance which holds have been built by individuals who met higher expectations for playability, fun factor, creativity, and solvability. This helps encourage ongoing participation in DROD by newcomers while simultaneously keeping the barrier low for new participants.

silver
Level: Smitemaster
Rank Points: 915
Registered: 01-18-2005
IP: Logged

did you read the argument that ensued when they announced only elite builders could submit SS holds? I don't think we need to add tiers. Plus, even a guy who has made a really good hold one time can make a really bad hold next time (it's possible - for an example, consider other artforms. Has there ever been a band for which every song on every album was of equal or increasing quality to all listeners?)

Rating by hold is good. I think people are overlooking the power Mike is putting into the 3.0 hold selection interface - the ability to sort holds by rating (or even see the rating after you've downloaded the hold ) will go a LONG way towards the perception issue. Especially if "rating, high to low" is the default sort for new installs


____________________________

miketo
Level: Delver

Rank Points: 61
Registered: 01-04-2006
IP: Logged

Silver:

Thank you for the reply. I wasn't clear when I wrote my post and you point out where I needed to put more detail into my thinking.

What the categories do is provide aggregate feedback over time over a number of holds. Sure, even the masters can produce a clunker once in a while. And apprentices can be one-hit wonders. Once you get those three good holds (and they don't have to be sequential), you're a Master Builder.

To continue the music analogy, lots of folks will listen only to certified hits. Others take great pleasure in listening exclusively to the indy scene. Being a Master Builder doesn't confer additional privileges, it's only an indicator of what quality holds you're likely to get.

It doesn't surprise me that arguments resulted over who could submit Smitemaster Selections. No one likes to think that the ladder will be pulled up before they even get to it. I specifically didn't say "only Master Builders can submit holds" or put restrictions on who can upload a hold to a particular category. Apprentices *should* be free to compete with Master Builders. You learn the most when you compete against people better than yourself.

But back to basics, the overall thread was about problems with holds being published as "final" before they had been beta tested or shown to have a solution. My suggestion doesn't address this; it only creates an environment where publishing a good hold has greater value than just publishing a first attempt at a hold. Greater minds than mine are needed to work out a suitable reviewing / rating system that isn't unduly burdensome.

--Mike

Stephen4Louise
Level: Smitemaster

Rank Points: 548
Registered: 04-06-2005
IP: Logged

miketo wrote:
Silver:
What the categories do is provide aggregate feedback over time over a number of holds. Sure, even the masters can produce a clunker once in a while. And apprentices can be one-hit wonders. Once you get those three good holds (and they don't have to be sequential), you're a Master Builder.
--Mike

In a tiered system like this, apprentices will be at a disadvantage. Players will tend towards holds built by Master Builders first, leaving us apprentices struggling to get people to play our holds.

In the other suggestion where an admin would check if a hold had been approved on the architecture board or in private beta, what is to stop the hold being modified after being tested. In my hold, I added a secret level which was only tested by myself. I decided to do this early on in the development of my hold as I wanted to make it worth playing for the testers too.

I believe that the current system is fine and can only be improved by more people voting. I admit that I vote a lot less since downloading holds in game became possible. Bad holds get low votes and don't get played, good holds get good votes and get played. Let's not make it harder for new architects to get those first few votes.

Steve.

PS. If you are inspired by these comments to play my well tested and 8.3 rated hold, you can find it here.

---

[Last edited by Stephen4Louise at 05-09-2006 10:57 AM : Added an unrequired plug for my hold.]

KevG
Level: Smiter

Rank Points: 333
Registered: 08-16-2004
IP: Logged

The real problem with rating architects is that all holds are not created equal. Tim has released 4 single level holds for a total of 68 rooms. Wesley has released 1 hold with 16 levels and 227 rooms. While Tim deserves to be considered a Master Builder does it really make sense to consider Wesley an Apprentice?

As far as the real question, the problem we're running into is that the forum is no longer at the center of DROD culture. When the Holds board was introduced it was pretty much a safe assumption that most, if not all, architects were familiar with the forum structure and the testing process. That simply isn't the case anymore.

Now we have people who aren't seeking out the forum until they already have a "finished" hold ready to share with people. They're going straight to the Holds board without looking around first. It's unlikely to occur to them that they should have other player test their hold first.

I really don't have an opinion on the ratings idea, but I do think it's time to make it harder to access the Holds board. The best solution, in my opinion, would be to disable direct uploading to the Holds board and only allow transfers from the Architecture board or a Beta board. Even if people were able to post to the Architecture board and immediately upload to Holds it would still serve the purpose of making them aware of the proper procedure.

Alternatively, simply making the Holds board harder to find would be a good idea. I disagree with Adam, I don't believe new players have any real need to access the Holds board. The sortable Holds Download page is where they should be getting new holds from. If this page were easier to find then the Holds board could be moved to the bottom of the boards list. Or, since it's possible to hide boards, the default option for new forum members could be for the Holds board to be hidden. They would have to gain at least some familiarity with the forum before uploading their hold.

Oneiromancer
Level: Legendary Smitemaster

Rank Points: 2936
Registered: 03-29-2003
IP: Logged

The problem with hiding the Holds board is that then new players won't be able to see or contribute to the discussions there, which isn't very useful. There are a lot of lurkers who play a lot of user-made holds before they ever post once, and we don't want to prevent them from being able to see those threads, in my opinion.

Game on,

____________________________
"He who is certain he knows the ending of things when he is only beginning them is either extremely wise or extremely foolish; no matter which is true, he is certainly an unhappy man, for he has put a knife in the heart of wonder." -- Tad Williams

NiroZ
Level: Smitemaster
Rank Points: 1302
Registered: 02-12-2006
IP: Logged

ok, heres my POV.

when a newbie enters the forum and goes to release a hold, he first submits it to architechture, which has to exist for 2 days( 1 day from the last update of the hold) before being trasfered to the hold board. This will hopefully stop people being really excited and submitting their hold on the spur of the moment.

Of course, problems with this could be that the person does not want to release publicly the beta-hold. thus they get caravelnet( thinking on this)

Once the hold hits the holds board, the 'select few' can test it, and if it somehow appears that a major bug slipped through, they can send it back to architechture. If the maker of the hold demonstrates that shows a sufficent understanding on how to make a good hold, the 'select few' can award him with a architects degree.

To get such a degree, one has to simply release one decent hold. This should be a good way to tell newbies that they are or aren't doing somthing right.

Ok, now slighty offtopic, but while thinking about this, i thought of another thing.

I just want to request that you can 'watch' holds on caravelnet, thus enableing you to store a list of holds you want to play, or a re playing, but want to keep it there so that they remeber to rate the hold when they finish it. (being alerted to when the author makes a new hold/beta hold would also be nice)

Mattcrampy
Level: Smitemaster

Rank Points: 2388
Registered: 05-29-2003
IP: Logged

Forgive me for being daft, but what exactly are we trying to accomplish here?

If we're trying to force people to go through training before they can build holds, that seems like an impractical goal just extrapolating from history. There have always been garbage level sets from people who think they're the next American McGee, but turn out to be, well, not. Those that take the ngative feedback as a sign and decide that they're going to have to improve will come to the Architecture board of their own accord, and the rest will become disheartened and give up.

If we're trying to make the holds on DROD.net be a quality set, I hesitate to ask where the not-so-good holds are going to end up, knowing that the answer is probably going to be 'nowhere'. Holds are not especially big, so DROD.net can certainly fit them in, and the community's here. And not anywhere else.

If we're trying to insulate people from bad holds, there are certainly steps that can be taken, and from the sounds of things they are being taken. If the whole idea is to reduce clutter, that's fair enough, and so some ideas along those lines would probably be better than stamps of approval. (I'd like the holds list to have, say, the 30 most popular holds on it and a full list option so that the bad holds don't cause too much clutter, personally.)

If it's something else, it might be useful to get that cleared up, because I think we're all arguing for different things.

____________________________
What do you call an elephant at the North Pole?

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

That's a good question, Matt. Here is what I think (I've said something similar above) -

For me, this isn't about hold quality. It's about hold integrity. It's about whether or not holds can provide the minimum experience they are supposed to - a set of rooms for Beethro to go through, from beginning to end. If a hold can do that, then it's a different question as to whether it does so well.

You are right that the only place for DROD holds is this community, but the holds board is not the sum of that community. There's the architecture board as well, where people can post partial holds, holds in progress, and holds that aren't really meant to be holds at all but something else like a small scripted sketch or something. The holds board is a place for holds that provide a certain type of gameplay - and winnability is an essential part of that gameplay, in my opinion.

I don't want a way to weed out bad holds. I want a way to weed out things that are not really holds - they might be almost holds, the author might have thought they are a hold, but they're really just a collection of rooms.

____________________________
I got my avatar back! Yay!

Schik
Level: Legendary Smitemaster

Rank Points: 5450
Registered: 02-04-2003
IP: Logged

I'm in the same boat as Matt here. I'm having a hard time figuring out what solution I like best, mostly because I'm not sure what problem we're trying to solve.

One big problem that we absolutely need to take more care to fix is unsolvable holds on the Holds board. A long time ago, Stefan fixed a bunch of holds for us, and just last night Mike Rimer fixed a few more. We'll try to either fix or delete unsolvable holds better in the future.

[Dangit eytan, you keep sneaking posts in while I'm trying to compose my post. ]

The other problem at least some people have is that there are holds that aren't very good on the Holds board. With every game I've ever seen that allows users to make levels, there are some bad levels. I'm curious what the actual problem is. Is the problem:
- You have too many holds listed as downloadable in-game that you don't want
- You feel like you have to play the bad holds because of high scores
- The fact that the holds exist makes you cringe
- Something else

To even begin fixing the problem, I need to understand what everyone perceives to be the problem.

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

KevG
Level: Smiter

Rank Points: 333
Registered: 08-16-2004
IP: Logged

Oneiromancer wrote:
The problem with hiding the Holds board is that then new players won't be able to see or contribute to the discussions there, which isn't very useful. There are a lot of lurkers who play a lot of user-made holds before they ever post once, and we don't want to prevent them from being able to see those threads, in my opinion.

I agree. But, the sortable Holds Download page has links to the forum discussion thread for each hold. The only thing you can't do there that you can do on the Holds board is upload holds. Making this page more visible, like with a link at the top of the forum, would eliminate the need to keep the Holds board prominent.

As far as MattC's question, I want to prevent people from uploading holds the instant they find the forum. We can't really force people to go through the proper testing procedure without a full blown review system. However, we can at least ensure that they are aware of the proper procedure by making them jump through a few hoops before uploading their hold. This won't eliminate problem holds *cough*Deep Hold*cough* but it will reduce the number that are submitted due to the architect not knowing any better.

gamer_extreme_101
Level: Smitemaster

Rank Points: 1939
Registered: 03-07-2004
IP: Logged

I agree with KevG. Why not make that page the main page and link to it on the top alongside the FAQ and Profile links, move the Holds forum down to the bottom and the architecture forum up, add a link to uploading holds on there and a link to the Architecture forum, and change the little "Discuss" icon on holds.php to simple text that reads "Discuss/Rate". Worst case, we can consider that a short-term solution to see how many more people end up using the Architecture forum to post their works rather than upload them directly.

____________________________
--That guy with a million different aliases since he doesn't like this name anymore.

michthro
Level: Smitemaestro
Rank Points: 679
Registered: 05-01-2005
IP: Logged

Matt x2: The short answer is that we want to prevent new architects from uploading straight to the Holds board instead of first posting in Archtecture.

At first I wanted some kind of quality control system, mainly for the reasons given by Andy101, but I've been converted to the point of view that that's just too complicated. Anyway, the biggest problem is solvability. At the moment, the quality of holds available isn't bad actually - "garbage" etc. are rather strong words. I'm more worried about what may still happen.

So, how to get budding architects to go to Architecture: How about going ahead and doing what KevG and Patrick said? The only potential problem is holds being more difficult to get hold of, but KevG pointed out why that's not a problem. So, try it and see. If it doesn't work, take it from there.

I also suggest changing the descriptions of the Architecture and Holds boards slightly, as they are misleading. For Architecture we have: "For the submission and discussion of new holds as works in progress..." For the Holds board: "Well, we had to put completed new holds somewhere..." I don't think people who upload straight to CaravelNet consider their work to still be in progress, and I can see how a new architect may be misled into thinking that Holds is the place to go.

Schik
Level: Legendary Smitemaster

Rank Points: 5450
Registered: 02-04-2003
IP: Logged

Well, you guys were so good at rewriting the CaravelNet expiration reminder emails, why don't you take a crack at rewriting the descriptions for the Architecture and Holds boards?

____________________________
The greatness of a nation and its moral progress can be judged by the way it treats its animals.
--Mahatma Gandhi

Tuttle
Level: Smitemaster

Rank Points: 1548
Registered: 02-22-2003
IP: Logged

Quick thoughts:

Holds:

Got a finely polished architectural masterpiece? MoMA won't exhibit it for you? Upload it in here so everyone can marvel at your work. If your hold hasn't been put through its paces by a few other players yet, you should probably go here or here first.

("here" links to architecture and beta stuff)

Architecture:

First-time builder looking for feedback and testing? Master architect ready to tease the public with tidbits from your latest creation? In either case, upload your hold here and enjoy the attention.

Tscott
Level: Smiter

Rank Points: 382
Registered: 02-10-2003
IP: Logged

wallu wrote:

ErikH2000 wrote:
We've got over 8,000 rooms at the moment. I would be just as happy if we had half that. It's still a ridiculously large number.

I had a feeling that the number of rooms is getting to 10,000. So, I counted them yesterday and got 9,924.

...and 1888 topics in the Hints and Solutions forum. That's a little over 19% - almost 1 out of every 5 rooms released has been hard enough to stump at least one person to the point where they needed to ask for help. Wow!


I did like the suggestion someone had of moving the Architect forum so it's next to the holds forum. That just make sense and would make it more noticeable to newcomers on the forum. Also requiring someone to register to the forums to upload a hold would help. I think there was at least one hold that was uploaded by a non-forum member and the fact that they could do that surprised me. That really seems like a feature that could be abused.

I am very wary of any quality control beyond "can all the rooms be solved and then can the player exit the hold". Just to use one fairly well known example, I know several people have enjoyed the "War" holds and several people haven't, and I wonder how far those holds would've gotten if there was some sort of review process.

____________________________
And I can recall our caravel: a little wicker beetle shell with four fine maste and lateen sails,
its bearings on Cair Paravel. O my love, O it was a funny little thing to be the ones to've seen.
-Joanna Newsom "Bridges and Balloons"

Pinnacle
Level: Smitemaster

Rank Points: 1137
Registered: 06-10-2004
IP: Logged

Why not link to the Holds board only through the Architecture board?


____________________________
Once (adv.): Enough.
Twice (adv.): Once too often.
~Ambrose Bierce, The Devil's Dictionary

---

[Last edited by Pinnacle at 05-17-2006 06:31 PM]

Syntax
Level: Smitemaster
Rank Points: 1218
Registered: 05-12-2005
IP: Logged

Excellent idea!

Also, rename Architecture to Holds so that it becomes the first port of call.

AlefBet
Level: Smitemaster
Rank Points: 979
Registered: 07-16-2003
IP: Logged

Syntax wrote:
Also, rename Architecture to Holds so that it becomes the first port of call.

I still think that for people who aren't developing new holds, the current Holds board is where they'll most likely want to go, and where we'd most like them to go. Forcing them to go through the board for holds under development to get to the board for mature and finished holds seems really backward. And it would most likely cause people to want to leave their holds on Architecture indefinitely, rather than face the prospect of "promotion" to a virtually invisible board.

____________________________
I was charged with conspiracy to commit jay-walking, and accessory to changing lanes without signaling after the fact .

++Adam H. Peterson

eytanz
Level: Smitemaster

Rank Points: 2708
Registered: 02-05-2003
IP: Logged

Yeah, I agree with Adam on this. Hiding the "holds" board isn't a solution but just a deferral of the problem at best, and it poses as much of a disadvantage, if not a bigger one, to legitimate uses of the holds forum as it does to the ones we wish to prevent.

____________________________
I got my avatar back! Yay!