I’ve written about reputation systems before. While they’re certainly not my primary area of research, I’ve been mulling over a few things lately.
First, there are lots of other people working on reputation systems. That link goes to a bibliography of 123 papers on reputation systems, and the last one was written in 2004. Google Scholar brings up 134,000 hits. Since I have done no formal research on this topic, I have read none of these. I’m just saying.
When people talk about reputation systems, they’re referring to the mechanisms that allow participants in a system to make public judgements about each other based on experience. For example, eBay’s feedback system. Here’s a screenshot of my eBay feedback:
Pretty nice, huh? Well, let me tell you that if there was a list of feedback that I’ve given to people, it would be equally positive. And I have dealt with a lot of crappy sellers on eBay, people with ridiculous requirements, people who didn’t read my shipping instructions and sent stuff off to my ex-roommate who doesn’t like me, etc. Right now I’m dealing with a red velvet damask comforter cover that is stuck somewhere between Arkansas and Brooklyn. I can’t leave negative feedback for any of these people, because they would go ahead and leave negative feedback right back. The impact of negative feedback is too high for both buyers and sellers, and the risk to buyers is too great for them to be honest.
Let’s look at another wonderful site, Friendster. When Friendster was conceived, by the boy wonder and strategic genius Jonathan Abrams, he thought of the “comments” section as a way to leave feedback on your friends. Like, I might “review” my friend and how he was such a great guy that any girl would be lucky to date him, which might make a potential date think positively about him. This has two flaws. First, nobody’s going to honestly dissect the flaws of their friends in a public forum unless they really don’t want to be friends anymore. And second, you can’t comment about the people that you really don’t like and have useful dirt on (cheated on his last four boyfriends, stole a dollar out of the tip jar last time we had coffee, never shuts up about her ex), because you’re probably not Friendster friends anyway, and they can erase any negative comments. [Obviously, all of this is made totally moot by the fact that nobody uses comments for anything except THANKS FOR THE ADD, happy birthday, have you seen this Mentos/Diet Coke video, let me compliment you so you'll be forced to compliment me back, etc. etc. type stuff anymore).]
That doesn’t mean there aren’t ways to make reputation judgements on Friendster. There are a zillion ways. (Doesn’t it feel retro, like 2003, using Friendster for an example instead of MySpace or Facebook or Bebo?) You can make snotty judgements on the person’s crappy taste in music or movies. You can read between the lines on their weak-ass comment section (“Fred is a great guy who I never see much but is always cool when I do!!! Luv ya Fred!!!) or their four friends. Or their 450 friends and their passel of Hot Chicks they’ve carefully collected. But what it does mean is that the reputation system that’s built into Friendster (or MySpace, or Facebook) doesn’t work.
We have a wide variety of social norms and social practices built up around avoiding being honest about our friends. There is pretty much no way you’re going to get people with social ties that matter to them to be honest about each other. The faux pas of getting caught talking smack behind a friend’s back is bad enough; imagine doing that in public. And consider how pissed off people get about being dropped as LiveJournal friends, or erasing a comment. It’s easier for some people to create unbelievably elaborate default filters on LJ than to defriend their ex-girlfriend, because it’s such a final manuever. It’s the equivalent of saying you’re dead to me.
Back to reputation systems. The second major thing I want to point out is that reputation is contextual (the next paragraph of ideas stolen from Jimmy Wales). Say I am an Amazon reviewer. All my reviews of mid-90s emo bands are totally spot on. I really know my Cap’n Jazz from my Get Up Kids. But when it comes to true crime novels, I have a big axe to grind and I give everything one star and rant about how true crime is violence porn and is destroying Western society. The emo folks will probably find my reviews very helpful, and I’ll have a great reputation based on that. But the true crime folks will probably find me an irritating bombastic pest, and will consistently mod down my reviews.
It’s very hard to boil “reputation” down to a + or – system, a number of stars, or one line of comments. In face-to-face interaction, reputation is an amazingly malleable beast that could never be easily quantified. Sure, you trust your brother with your life, but do you trust his driving? Is your best college buddy’s drinking problem changing how you feel about them? What about someone you sit next to at work? Someone who wrote a bad review of your band? Human judgement is unbelievably capricious and irrational and based on emotion and gut feelings and all kinds of other things that can’t be trusted.
So, what does this mean for websites? Well, first, I think it’s safe to say that reputation systems are important. Recommendation systems, the people-who-liked-this-also-liked-that, are great, although they’re also prone to mistakes and they don’t really deal very well with obscure preferences (Last.fm recommends Radiohead and the Beatles to everyone. Thanks! I hadn’t heard of those two!). But in reading web content, I would much rather read content by smart people my age or in my field or at least people without horribly sexist views or who speak fluent l33t. Is there a way to filter that kind of content up, given all the intrinsic faults in reputation systems? And also, we have to consider that there are probably people who’d rather read negative reviews of true crime novels, or people who look upon the equality of women with suspicion. What I consider reputable may certainly not be universal.
More thoughts to come.