The Vengeful Polyglot

Why a Rating System isn’t Right for the Dungeon Finder

Posted on: August 25, 2011

While the Dungeon Finder has been lauded by many as a great success and a quality of life improvement for World of Warcraft’s playerbase, it has not been without well-deserved criticism. In particular, the social ramifications of cross-server grouping I briefly addressed in my previous Dungeon Finder post have been pushed to the forefront.

Having followed the ups and downs of this issue on the forums fairly closely since the system’s implementation, I’ve seen one proposed solution presented time and time again: that a ratings system should be imposed so that “good” players can somehow be given priority over “bad” ones while using the Dungeon Finder, thus hopefully creating a more pleasant social experience while running randoms. While Blizzard has announced that it will not implement such a system, players’ enthusiasm for the proposal is unabated.

Personally, I believe Blizzard should stand firm in its position. I don’t think a rating system would be beneficial to the Dungeon Finder; rather, I think any change to this effect would only worsen the already toxic social environment found in some random groups. It is, however, an issue which warrants closer inspection.


What’s broken about the Dungeon Finder?

Since I know I probably beat this issue to death before, I’ll attempt to be brief: in part because of its cross-server nature, widespread use of the Dungeon Finder has more or less killed the “social contract” that tenuously existed between players on the same server. That is to say that prior to the random Dungeon Finder, poor behavior could lower a player’s reputation, which encouraged player accountability. Due to the increasing anonymization of the game provided by features like the Dungeon Finder, however, a player’s reputation no longer holds much if any import. A player who behaves selfishly or unfairly in a dungeon can leave party without punishment, and never run into those groupmates again.

I think it’s particularly the lack of punishment which raises the hackles of players who believe they have been treated unfairly while in a random dungeon. This is understandable, and I’ve definitely been there myself. No one likes to have their play tainted by someone who is insistent on being a jerk. It only adds insult to injury that the rude player almost always gets away with acting inappropriately. This feeling of impotence, of not being able to “do” anything to the person who slighted them, is what players are likely seeking to avoid when they propose changing the system.


What’s the proposed solution?

The most common suggestion I’ve seen on both the official forums and personal blogs is that the Dungeon Finder should include a ratings system, theoretically so that unhelpful, rude, or otherwise “bad” players can be isolated and penalized in some way for their poor performance or behavior. This is usually intended to provide two benefits for players who conduct themselves appropriately: they would gain some explicit benefit (generally faster queue times than lower-rated players), as well as the implicit benefit of not having to deal with players who tend toward being uncooperative.

There is no standard idea about how exactly the system should be constructed or which benefits should be conferred to players who behave appropriately. Each proposal is different, though I’ve found they can broadly be put into two categories: those which let players rate others directly, and those which rate automatically based on measurable game events. Those which allow players to rate others typically provide for a survey at the end of the dungeon, allowing each player to rate the others based on their performance. Those which automatically rate generally add or deduct points based on concrete game events like dungeon completion, leaving a dungeon early, or voting to kick another player.

As with the specific rating systems described, these proposals also vary based on their direction and intended effect. Some aim to have “good,” or highly rated, players given priority over “bad” ones in queue: if you’re a good DPS who queues at the same time as a bad DPS, your wait time would be shorter. This does not proscribe group composition, however, as some seek to; some suggest the segregation of these groups entirely, so good players are only grouped with other good players (presumably those rated above a certain threshold), and bad players only with other bad players. Still others want rating to have limited, rather than global, usage, so that rating someone up or down impacts only how likely you are to group with that person in the future.


Why is a rating system a bad idea?

Put simply, if you ask ten players what makes a player “bad” you’ll get ten different responses.  It could be poor attitude, the inability to play their role properly, or any number of other reasons one player considers another unacceptable. Most agree that a player who is both rude and unskilled is “bad” and someone who is pleasant and skilled is “good,” but border cases are less well defined. Some would rather play with a novice player who is kind than someone who plays very well but is intolerably arrogant, whereas others hold the exact opposite opinion.

Given that there is no way for players to make consistent, objective judgments about which players are “bad,” a system based on player-made ratings would never be able to operate as described in the ideal. Since standards are inconsistent, ratings would be as well. Players can say, “jerks should be grouped with jerks,” but if the entire playerbase doesn’t agree on what makes someone a jerk, this cannot realistically be accomplished.

Another obvious issue with player-controlled ratings, aside from differing standards, is abuse. It is irrational to claim that the playerbase is entirely honest or considerate; empirically it certainly hasn’t been my experience. There are some mean-spirited players who would downvote every player they came across, regardless of performance. I would expect there to be checks on this type of behavior (such as those placed on the Vote Kick feature) which stop a serial downvoter from influencing ratings after a certain point, but abuse is still an issue many of the proposals don’t address.

Additionally, novice players are put at a disadvantage as there is little differentiation between intentional bad play and that based on inexperience. Those who are trying something new for the first time could find themselves harshly downgraded and discouraged from the very activity which might ultimately assist them in becoming better. We shouldn’t allow players to be punished because they don’t know any better; we already discourage newbies enough as is, given the unhelpful environment generally found in randoms.

Those systems which eschew direct player involvement have their own problems. First, they often mimic features already in game. Leaving a dungeon early already comes with its own penalties, as does voting to kick another player; players should not suffer multiple penalties for the same behavior. Second, they are unable to account for intent. The inability to distinguish between well-meant and malicious situations is a far bigger problem.

To take an example from a specific proposal I found, assume players gain rating from completing dungeons and lose it from voting to kick other players or leaving dungeons early. Being kicked, however, does not impact a player’s rating. A situation for potential abuse readily is apparent, which is demonstrable through a hypothetical, but likely not uncommon, situation: a player queues as a tank who does not actually intend to tank; the rest of the group, understandably, votes to kick him after he intentionally wipes the group.

Under this proposal, the group members who acted appropriately are penalized, while the malicious faux-tank is not, and can immediately queue again to do the same thing to another group. Clearly, the other members should not have been penalized, although the action of kicking should normally downgrade a player’s rating (since it’s seen as negative). The system described simply doesn’t account for times when players deserve to be kicked.

Any system adopted should not allow a situation like that in the example to occur as posited, but since the system can only account for discrete in-game events without consideration of  intent or special circumstances, the result discussed is likely. By biasing toward being fair to the kicked player, they have unintentionally opened up the potential abuse of players who rightfully kick another. Just as player-based ratings can be abused on a whim, so too can “objective” systems be abused due to their inability to account for the multiplicity of intents behind the same discrete situation.

The final nail in the coffin for me is that such a system, as I’ve partially touched on before, is completely unnecessary given existing features. If there is a player you don’t wish to group with in the future, instead of making a global rating system for everyone (or even one that tweaks the probability of whether you see that player in a random dungeon again), just add that player to your ignore list. This prevents you from ever grouping with them again, and it works on a cross-server basis.

Given that this feature is already in-game, the reason for these proposals must be that players want to avoid ever having to group with a bad player; they never want to see a bad player, not even the one time it would take to recognize and ignore them. I don’t however think this is cause for a implementing another system which is largely redundant; the burden of having to group with an undesired player once prior to removing them from your future instances is just not unreasonable.

What’s the verdict?

While I expect proposals for this type of rating system will continue to be championed, there is no realistic or fair way for such a system to be implemented, and furthermore, no need for such a system at all. Having considered many of these suggestions on an actual as well as hypothetical basis, I firmly stand by Blizzard’s intention to not implement a Dungeon Finder player rating system; I can’t see any situation where these not only fail to improve the situation, but in fact make it even worse than its current incarnation. While I agree that there should be consequences for bad behavior in random dungeons, I strongly believe that a rating system is not the proper way to accomplish that.


3 Responses to "Why a Rating System isn’t Right for the Dungeon Finder"

That was a very academicly written assesment of the situation. Agree with you 100%, thanks for your post.

I’d write more…but you’ve really covered everything I can think of.

Thank you! My writing tends to be… a bit more formal that I’d really like it, but it’s hopefully not ambiguous at least!

Your basic sum up (correctly) is that there isn’t an objective or truly fair way to classify players as good or bad. And who would want to be stuck with the bad guys!

Thinking about it as a programmer, you maybe could do a decent job making some kind of index based on the stats, gear, and achievements the game does record. I can manually inspect a player and in 2 minutes tell if they are a decent technical player. Clures like wearing the wrong stats, no enchants or gems (come on, green gems are like 1g and add a hell of a lot more than empty sockets), or gear that doesn’t suite the spec are dead give aways for losers. The game could also analytically judge dps/healing/threat output, inappropriate pull %’s etc.

What it can never judge is attitude. I’ve had more groups ruined by players that take offense to small things than anything else. But I guess it could record stats for bailing in the middle. But it’s hard to judge because sometimes I’ve been in groups where I try and try for a while, but then it becomes clear that even if we will eventually succeed, it will be too painful to be worth it. So bailing becomes the smart thing to do.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Blog by a programmer cum linguist cum writer cum total geek. One who pretentiously uses "cum" in place of any other logical connectives. Direct questions to the Ask Lauren page!

My Flickr Photostream

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 22 other followers

%d bloggers like this: