Sunday Thoughts: Reviewing Reviews
Head banging.Published: 16:30, 05/09/2010 by Staff.
12When you post something on the internet, you’re openly inviting discussion and critique. We’re lucky here at TheSixthAxis because we’ve managed to create and nurture an active community and a readership that’s happy to voice their thoughts openly and without – wherever possible – fear of any sort of backlash or abuse – which is great we love reading your comments and most of the time they really make our day, but the one aspect that still confuses us on a regular basis is how people react to reviews.
Our review policy is linked to at the foot of every page, but it still appears that some of our readers don’t really understand what our reviews are all about, quickly flicking to the numerical score at the foot of the text and deciding that the game is, for example, ‘average’ because it’s a 7/10 score. 7/10, as you’ll see from our review policy, is ‘quite good’ – inFamous and Heavy Rain are, in my personal opinion, quite good. That’s just my opinion, yours may differ and that’s fine, that’s human nature. By that’s the review score we went with and despite the comments at the time, I still stick with both those scores.
Average, though, by definition, is very much 5/10.
Games that we’ve ranked as ‘average’ recently, then, include Bladestorm, Haze and Enemy Territory: Quake Wars, and, as of this weekend, Mafia II. Metacritic says we score games, on average, 1.8/100 points higher than the industry average, a point which reader Jacklum raised earlier today in the comments of the Metroid review. This, in my opinion, is completely irrelevant and as our editor Peter said with regards to his Mafia II review, we don’t get caught up in what other reviewers are saying about games in any way – it doesn’t matter.
In fact, knowing what others were scoring a game in advance of a title we were reviewing might even be counterproductive. We try, as much as possible, to ‘go in blank’.
What’s perhaps more alarming is the way the industry outside of TheSixthAxis (and a couple of other publications) has already set a ’7′ as the ‘average’. One reader of the site said that “actually a 7 is a average score, 5 would be the middle of the score-scope, so to say, but the way games are scored, 7 is the average. I dont say that’s the best way to do it, i also would prefer if 5 would mean average, but once a standard is settled, its hard to change.” It’s not hard, at all, it’s easy – don’t treat a 5/10 as anything but average.
And, for goodness’ sake, don’t think a ’10′ is perfect.
The way I see it it’s others that have skewed the score upwards. I don’t know why or when this happened and any attempt to try to work it out is well beyond the scope of this blog – besides, I’m not privy to the editorial policies of any other site apart from ours. But I do know what when we review a game we try to ensure that, if nothing else, that middle point is always at the forefront of our minds. And it’s the text that takes us days to produce, not the number at the bottom that, as much as a single integer can do, is meant to summise everything that’s above it yet provokes so much confusion.
We’ve blogged about review scores here on TSA before, but it’s worth re-discussing perhaps, especially as it can be so disparaging going over old ground in the comments for every new review. Essentially, we want you to read the text, weigh up the pros and cons and then, if you desire, glance at the score at the botom but always take that score in the context it’s intended. All reviews are subjective to some degree, that’s personal opinion for you, but we do balance out the good and bad as objectively as anyone can do.
And, as we’ve said before, look at as many other reviews as you can do before making up your mind.
Disclaimer: All comments are the opinion and responsibility of the individual author and not TheSixthAxis. You must read and agree to our terms before reading and commenting on this site. User comments are not always moderated by TheSixthAxis.
« Previous 1 2 3 Next »
« Previous 1 2 3 Next »
Most Comments This Week
- PS3 Firmware 4.10 Brings, Wait For It, A Decent Browser! [101]
- OnLive’s Evolution [72]
- CoD: Black Ops Has The Best Ending Ever [67]
- PS Vita TV Ad Campaign Revealed [65]
- PS Vita’s Load Times Analysed [56]
- What We Played #39 [49]
- PS Vita: Relic or Revolution? [47]
- PSN Store Update: 08/02/12 [45]
- Vita PSN Prices Announced [44]
- Is The PS Vita’s Marketing Really On Target? [43]
TSAtv: Original Video Content
Batman book review
Kris Dancing, again
Joe Danger interview
Sonic Generations
05/09/2010 at 17:10
Member since: Jul 2009
TSA doesn’t have the capacity to review as many games as the likes of IGN or Gamespot. I’m sure the guys bring us the reviews to games they think are important to us a community whilst simultaneously reaching to the wider industry. This is why the average score on TSA games is high-ish. It has nothing to do with the individual merits of any given title and whether it deserves a 5 out of 10.
I agree with the general consensus where the industry at large is far too generous with its handing out of points. The moment TSA adhere to this is the moment I lose respect.
05/09/2010 at 17:18
Member since: Aug 2009
I agree with the general consensus where the industry at large is far too generous with its handing out of points. The moment TSA adhere to this is the moment I lose respect.
Good comment, one of … in not *the* main reason I visit this site every day and is always my first port of call gaming-wise is because I get the feeling it’s made and produced by guys (and gals) just like us … written by gamers, for gamers, it’s something I feel is important and Raen kind of hit upon it last weekend in his Sunday thoughts piece.
I (for one) wouldn’t have it any other way.
05/09/2010 at 17:23
Member since: Aug 2009
In my opinion, a lot of corporate gaming sites are overscoring games these days (*cough* IGN *cough* Infamous *cough*) and I think that’s part of the problem:
The critic goes into a review with LOW standards
They score the game high
The public buys the game
The public doesn’t like it
The public blames the critic (8/10? Bullshit! I’m only gonna buy games that get 10/10 now!)
Now the public has HIGH standards
Games that score between 5 and 7/10 are overlooked despite having their merits.
Oh, and I think money comes in somewhere.
Repeat to fade.
I think the critics need to raise their standards so that the public can truly differentiate the good from the bad when reading their reviews. I’m not one of those ’10/10 = perfect game’ types, but we really need to consider what merits such a high score. Things like innovation, immersion, technological breakthroughs.
That’s what I like about The Sixth Axis: they don’t fall victim to hype. And as such their review of Infamous is the only one on the internet that I agree with :)
05/09/2010 at 17:31
Member since: May 2010
Derek Akora is a fairly average medium, sometimes he’s mean too.
07/09/2010 at 00:20
Member since: Aug 2008
Genius!
(But it’s spelt “Acorah” I think.)
05/09/2010 at 17:41
Member since: Forever
When I review (and I encourage all our staff to do the same) I do it as honestly as I can and with as little influence from external sources as possible.
Ask nofi or Raen how many times I put a review in our publishing queue and then send them emails where I’m panicking a little bit because I literally have no idea how well the game has done elsewhere and I worry that I’ve scored it too low or too high.
This week’s Mafia II review is a perfect case in point. I hadn’t read a single word of a single review before I put my review and score in the queue and I was really worried that I might have been too harsh. I spent hours re-reading and re-assessing my review so that I was totally sure of myself (and I am).
Just before it went live on the site I looked around at some other reviews and saw that it was really polarising opinion so I read a few of the big names that scored it highly and I was shocked at how little they knew about the game they were rating. One outlet didn’t know what dates the game took place between. Several places rating between 6 and 9 out of ten had chosen to completely ignore the tired narrative (some even complimenting it, as if they’d never seen The Godfather, Once Upon a Time in America, Goodfellas or The Sopranos) and the copious repitition. In short, reading those poorly constructed reviews from major outlets (I can forgive the naivity in the small amateur blogs) made me sure of my own review and score.
A review is different from an opinion. A review has to have a solid, researched base and be hinged on objective positives and negatives. Sure, opinion plays a part too – in how much you’re willing to forgive or enjoy – but it’s only a small part in what goes in to writing a review. Anyone who comments on a website can have an opinion and voice it but very few of those people (and some who do it professionally) can actually write a good review.
05/09/2010 at 18:33
Member since: May 2010
the ego has landed…..joke chill out.
05/09/2010 at 18:38
Member since: Aug 2009
I thought cb explained himself and TSA’s reviewing policy very well … what part of what he said prompted your post?
05/09/2010 at 18:44
Member since: May 2010
as i said it was a joke, chill out. he gave a glowing review of his own reviewing ability (which i agree with) i thought that with me saying i was joking that it was obviously a joke but some people need things spelling out for them so hear you go…. j o k e
05/09/2010 at 18:48
Member since: Aug 2009
Thankyou, while you didn’t necessarily answer my question at least I now know how to spell joke. :)
05/09/2010 at 18:52
Member since: May 2010
re-read my reply as i sort of fully answered your question.
05/09/2010 at 19:45
Member since: Mar 2009
I can confirm that I read a bunch of reviews of Crackdown 2 to put to rest CB’s fears of being alone in his opinion (not that it would have changed it). He’s such a drama queen ;)
05/09/2010 at 21:04
Member since: Nov 2008
A T T I T U D E.
1 point to me for spelling ;)
05/09/2010 at 17:47
Member since: Aug 2010
I agree about reading, but numbers do help.
Scores for things like value and replayability are important to me.
05/09/2010 at 18:29
Member since: Sep 2008
You’re right there, scores in individual areas help you decide whether to buy a game because you know what you are after, for example I go for serious plot most of the time. At the end of the day what does an overall score mean? Is it a ‘buy this’ factor, or is it ‘this is my idea of the perfect game’?
Personally, the narrative does it for me, and TSA do a great job of the narrative.
05/09/2010 at 18:44
Member since: Mar 2009
The way i see it review scores are just a guideline and if it is a game i was after i will buy the game anyway and then judge it for myself. Take crackdown 2,it gets hammered by people for bringing nothing new to the game from the first one but I loved the first game and new what I was after and that’s fine, I love the second too.No game has the perfect score but any game can be perfect for you its all down to personal taste.
05/09/2010 at 19:10
Member since: Dec 2009
I have disagreed with a few of the scores given on TSA but i’ve found the reviews themselves thorough and informative. Reviews are best taken as a guide and if you read a handful of them you won’t be long averaging out the score as far as your own interest is concerned.
Also it must be a bit tricky for any reviewer figuring a scoring scale/average across genres and platforms.
I hate to see a game i’m enjoying get a low/average score because – well because i’m enjoying it! :)
Anyway reviews are rarely pivotal in my purchase decisions although i like to read them and they can provide detailed info about a game i’m unsure about and if i’m unsure about a game i’ll read several reviews anyway.
Regardless of my differing opinion on some scores i like TSA just fine as it is -open to discussion and everyone’s opinion- and would never have joined in the first place if it had been overtly enthusiastic with it’s scores.
05/09/2010 at 19:20
Member since: Jun 2009
We are just used to the number being a indicator these days and i doubt it will ever change. Like it or not we always go on the score, if i had £35/40 to blow on a game and there where 2 games rated either end of the scale, i know what i would go for
05/09/2010 at 19:22
Member since: Jun 2009
Also we all have varied tastes so AAA title for you may not be my cuppa tea
05/09/2010 at 22:07
Member since: Oct 2009
shouldn’t 2 or 3 of the team review a game then its a more fair score than just one persons opinion.
05/09/2010 at 23:32
Member since: Jun 2009
I second that.
05/09/2010 at 23:57
Member since: Jul 2009
I think time constraints prevent that (at a guess).
06/09/2010 at 00:05
Member since: Forever
not just that, it’s technically impossible for us to get a single disk around the country in time for the game’s launch with enough time for everyone to play it. we’re not all sat in some cushy office like some outlets…
06/09/2010 at 00:27
Member since: Jul 2009
Gutting. If only there was a device that enabled you to copy the game to the HDD then pass it onto the next staff member. :-p
06/09/2010 at 02:28
Member since: Aug 2008
I thought all the main staff lived in a council flat together. Well that’s what the staff page says anyway.
Could there not be a way to allow staff to vote out of 10 for a previously reviwed game.
e.g. you review a game and give it 8, which would be displayed as it is. Then if CB, buys it, he can vote what he thinks the game deserves, then show the average of the staff votes below the reviweres score
06/09/2010 at 03:45
Member since: Oct 2009
but the Mafia 2 review was 7 days after the game was released.
06/09/2010 at 12:37
Member since: Jun 2009
With all due respect nofi, some of the reviews on this site are written weeks, if not months, after the game has come out!
05/09/2010 at 22:15
Member since: Sep 2009
I give this thread a 7/10, in thread reviewing that’s about average
http://instantrimshot.com/
I think the reason a 7 is given to what are basically average games on some of the big sites (ign in particular) is because if they only give the game a 5, sales will be lower for the game, the publisher won’t be happy and will start throwing their toys out of the pram and not send over review code to IGN, who lose readership because “you don’t even have a review of supersoldierblasterman 2″ (oh yes, clichés!). So they hand out an inflated review, publisher happy, ign happy, advertisers happy, bank managers happy, gamer… lied to.
Happily, TSA is rather excellent.