The nature and challenges of community moderation
My friend Sacha is putting together a SXSW panel - Living in the Matrix: Communicating past Agent ‘Smith’ based on a geek conversation we have on community moderation. I’m sharing the ideas below and if you like it, or want to hear more about it, please vote for the panel so that I might go troll SXSW :) Thanks!
So…
The legendary Linus’ Law states that “Given enough eyeballs, all bugs are shallow”. It states that with enough community scrutiny, bad content/contribution will be noticed and taken down. From my work on the sociology of Wikipedia and experience in Yahoo!, I would add that there are three key elements to having a successful community (which a lot of designers/start-ups seem to be conveniently unaware of).
They are
(1) the Composition of your community
(2) the Tools to empower them (i.e. the system of moderation)
(3) the Timing between noticing and taking down bad content
Let me elaborate.
(1) Composition of your community
For Linux and Wikipedia, the communities are composed of many extremely smart, technical, virtually connected geeks that are experts in online communications and are very comfortable with setting up policies and norms for their communities to follow. They are also good at developing hands on tools to execute the policies.
For example, Wikipedia community established the three core norms: Neutral Point of View (NPOV), Be Bold, and Assume Good faith. Its developer communities built a special app, VandalFighter, to help editors monitor realtime edits among the many other useful tools,
Vandal Fighter is software that watches Wikipedia edits in real-time and has many built-in filters (multiple regular expressions can also be used) to allow users to specify exactly what types of information they would like to see. Examples: watching custom article watchlists, watching edits by non logged in (IP) editors, ignoring edits by people marked as “ok” (or, the reverse as well), and watching comments with the word “revert” in them are just a few of the almost limitless ways in which Vandal Fighter can be configured to produce custom output.

VandalFighter screenshot (source)
Basically when you have a bunch of really smart people using your community product, the plus side is they are also the key people that will help you develop and refine community policies, and even help you develop tools for moderation. The downside is the challenge for the product people to remain open and transparent, and be liberal enough to accommodate opinions and suggestions. In that regard, Flickr’s done a pretty good job in encouraging communities to tell them what they want and actually responding to those needs by both (1) talking to the users; and (2) implementing new features or explain why not if it’s not possible.
The problem I see is that people assume that their community will also function like the highly technical, systemic crowds above, which mostly is NOT true. If you have a fashion community website, people may flag a button to state if they like or hate the content, but don’t expect them to fill in a form to explain why – people are usually pretty reluctant to work so hard when they are just browsing a site. If they keep on seeing bad content, they will leave and will not try to fix it for you. Think about the design – how can you gather data about the quality of the content without needing your users to do a lot work?
(2) The moderation system
The key to a robust moderation system is that – you can’t rely on old school customer care services alone if you know that your community is going to grow in scale. Traditional customer care is expensive, inconsistent, and slow in turn around. What it means is that your community is soon going to know they know better than the people sitting behind the hot desk, on both how the product works and how to game the product. You will need to work out a moderation system that incorporates community realtime feedback, such as the Slashdot meta moderation system, where trusted users will be empowered with more karma points to help promote or suppress content. I encourage you to read through the documentation as I know most world’s robust systems share similar principles as the Slashdot mechamism, and here’s my favourite bits,
Who
It’s probably the most difficult part of the process: Who is allowed to moderate. On one hand, many people say “Everyone”, but I’ve chosen to avoid that path because the potential for abuse is so great. Instead, I’ve set up a few simple rules for determining who is eligible to moderate.
- Logged In User If the system can’t keep track, it won’t work, so you gotta log in. Sorry if you’re paranoid, but this system demands a certain level of accountability.
- Regular Slashdot Readers The scripts track average accesses from each logged in user. It then selects eligible users who read an average number of times. The homepage doesn’t count either. It then picks users from the middle of the pack- no obsessive compulsive reloaders, and nobody who just happened to read an article this week.
- Long Time Readers The system throws out the newest few thousand accounts. This prevents people from creating new accounts to simply get moderator access, but more importantly, means that newbies will have to be part of the community for a few weeks before they gain access to the controls to a system they don’t understand.
- Willing to Serve If you don’t want to moderate, just visit your user preferences, and set yourself as ‘Unwilling’
- Positive Contributors Slashdot tracks your “Karma” (see the FAQ). If you have non negative Karma, this means you have posted more good comments than bad, and are eligible to moderate. This weeds out spam accounts.
So the end result is a pool of eligible users that represent (hopefully) average, positive slashdot contributors. Occasionally (well, every 30 minutes actually) the system checks the number of comments that have been posted, and gives a proportionate amount of eligible users “Tokens”. When any user acquires a certain number of tokens, they become a moderator. This means that you’ll need to be eligible for many of these slices in order to actually gain access. It all works to make sure that everyone takes turns, and nobody can abuse the system, and that only “regular” readers become moderators (as opposed to some random newbie ;)
Basically when you start to develop a new system with communities of users, remember to add the above top five variables trackable into your backhand – even though you may have only a handful of kitties using your site, if in the fortunate even of your community taking off, these data WILL come in handy.
(3) Notice and taking down time
Cannot emphasise this enough – remember when you were a kid at school, there were teachers that pupils respected as opposed to those who were bullied by kids or just completely be ignored? What made the difference?
The answer is the consistency of treatment and the responsiveness in delivering rewards and punishment. Forgive my Pavlovian tone here, but online communities rely even stronger on fairness, consistent treatment, and timely response since people cannot see one another in person. Lots of emotional information is missed out, hence collective community behaviors are shaped by the system, not just by the people. If you let a few trolls take over your sites, your community suffer. In the normal life cycle of events, communities will moan and seek help, and if nothing happens, they may stay and tolerate the bad content, which would leave a sour taste and encourage people to act in a uncivilized manner – trolling the trolls. Other light-minded members will leave because it’s not fun anymore. So – when you design your community – make sure you have factor in the notice and take-down time by incorporating first two points by defining the nature of your community and tools for empowerment first. These are not nice-to-have – these are necessarily conditions for any online communities to succeed.
Okay I’ll be a bit hardsell here and repeat myself – If you’d like to hear more on community management, please vote for my friend’s Sacha SXSW panel
Living in the Matrix: Communicating past Agent ‘Smith’
http://panelpicker.sxsw.com/ideas/view/4414
I am probably missing a lot of the core details as well, will be filling this in from your comments and links and hopefully this will build into some sort of useful documentation. ;)









