Braben: Game Reviewers Can’t Win

Industry veteran David Braben, the man most recently responsible for Kinectimals, has spoken out about game reviewers.  In a blog over on Develop, he questions the difference in taste between a writer and their audience, calling into question the old adage: “it wasn’t aimed at me, and so it must be bad.”

Naturally, here at TheSixthAxis we like to be as objective as possible with our reviews: developers and publishers have spent considerable time, money and effort bringing the game to market and you guys, the buying public, will be the ones spending your money.  Or not.

– ARTICLE CONTINUES BELOW –

“As developers,” says Braben, “we all like to bemoan the odd game review from time to time – because games are close to our hearts, and increasingly our wallets both in terms of sales and Metacritic-based incentives.  Most reviews are targeted at what are often called ‘core’ gamers; people like us that follow games avidly, and are very experienced at playing them. ”

“A problem starts to occur when the audiences’ tastes differ significantly from the reviewer’s – or developer’s – own tastes. This is becoming more of an issue as our industry matures to include a great many people outside this group – particularly so if the group targeted is not just this ‘core’.”

Braben mentions some key Nintendo games: Animal Crossing, Nintendogs, and asks why the reviews are “noticeably lower” than their quality might suggest, because the reviewer didn’t think the game was for them.  “This attitude makes sense if the audience for the review is effectively the same as the reviewer themselves, but for a review on TV, on a website for kids and adults, or in the mainstream media, it does not.”

“Just as it is difficult to develop games for a different audience, it is difficult to review them too,” he says.

He then says that the overall effect for this difference in taste is “about 10 per cent” but says he’s “delighted” by the mature response by the games press over Kinectimals, which we scored a solid 7/10 – reviewing the game for the audience intended: kids.

Braben then mentions Achievements and Trophies, and says that they’ve been a “curse” for a “small minority of hurried or irresponsible reviewers” as they’ve been caught out scoring a game when only having played it for a limited period of time. “Most reviewers are excellent at what they do, and it is a very hard job with, frankly, little glory,” says Braben.

He’s right.  If a game’s massive, like Gran Turismo 5, it’s difficult to assign a score, something we still have yet to do – it’s not always possible to 100% a game even after weeks.

“As an industry, there is something we could do to recognise this – effectively a Metacritic for reviewers.  The best reviewers give spot-on reviews pretty soon after a game is released. They do not wait to see what others say, but nevertheless consistently come very close to the final average score. There could be a prize for the best each year.”

It’s an interesting blog, and one certainly worth reading.

– PAGE CONTINUES BELOW –

34 Comments

  1. Whilst he has a point I don’t think it’s a widespread problem. I would never agree to review Rockstar game or a fighting game because I just dont like them and don’t know enough about other titles in the genre. OPSM have also said on their podcasts they would never give a sports game to a non-sports game fan.

    • I think Braben’s point might be skewed slightly by the reviews for Kinectimals, which would have been written with derision from some quarters, not really with the intended audience in mind.

      Perhaps.

    • I agree with Tuffcub, though as time has passed, I have found myself adapting to more and more types of videogame, including sports and racing which always used to slip under the radar.

  2. Surely even if they don’t like the genre themselves they should be experienced enough to know whether it really is worthy of a certain score keeping their audience in mind.
    Saying that I’ve never reviewed anything so I take my hat off to a reviewer. I guess in some cases you’re being quite brave if you don’t know what other people have scored games before you.

    • Yes, this is why the argument that “reviews are just an opinion” is so stupid. If you’re not able to be mostly objective (assuming it’s impossible to be totally objective) then you’re not writing a review, you’re writing an opinion and anyone can do that. Real reviewing is difficult, shouting opinions is really easy.

      • The review you guys write can only ever be a mix of fact and opinion (although what measure of each is down to the individual). The factual parts could be the quality of sound or the incredible graphics, etc. However, how those elements come together (or not, as the case may be) is something of an opinion and one that we all appreciate. It’s the bit where you are responsible enough to review genres you’re comfortable and happy to assess (god knows you should never ask me to review a football game unless you want your front page littered with profanities).

        I suppose, thinking about it, all reviews around the world should start with the likes of “if you don’t like {insert genre} games then the score of this game needs to have 50% of its points deducted” or something similar but the media rely on readers having at least some semblance of sense about them.

        When all of that is in place (and usually taken as a given) it’s down to you guys to be objective as possible with the title at hand whilst simultaneously looking to the genre and/or current generation to see how it rates amongst its peers although this is more personal preference too.

        Either way, TSA do a bang-up job and I’m happy fondling you all some more.

  3. I read a bit of the blog post, and I see where he is coming from. When I began writing user reviews a long time ago, the scores were always in favour of games I liked, not necessarily games which were good. However, now that I write for the benefit of others, I put myself in their shoes when it comes to the reception of a game.

    One of the best examples of writer preference would be Jeff Gerstmann’s review of DC Universe Online which scored a 2-star rating. Being a heavy MMO player, he instantly picked apart the quest system and discarded the action-sensitive combat. He also mentioned how ridiculous it was for a user-created hero to stand toe-to-toe with some of DC’s biggest icons. The content of the review didn’t exactly echo the score, and sounded as if it were aimed at only the hardcore MMO players. With that said, it’s still a brilliant review.

    • A general review and one specific to the genre, thats the way i think it should be,
      a review by a fan who can be objective, and then a second “more detailed” review where by the reviewer can tear a game apart and look at its insides so to give the hardcore fans something, but i believe this should be seperate to the first

      • An interesting way of putting it, but one which I personally couldn’t see working. The most accurate way would probably be a four-way Famitsu type of review in which they get four writers assigned to one game and then take the average score.

  4. I have huge respect for David Braben and I agree with most of what he’s saying here. The only issue I would have is with this: “They do not wait to see what others say, but nevertheless consistently come very close to the final average score”
    Do you want “fitting in with the average” to be awarded or do you want reviewers stepping outside of the norm and reviewing games for their intended audience (as we always try to do)? Because in the wider world of reviews (not here) it’s generally only the top third of a scale that’s used. So scores are artificially high and, as Braben points out himself, negatively weighted against certain platforms and genres. So awarding the norm seems counter-productive to the rest of his argument.
    Certainly an interesting debate to have though.

    • I don’t mean to bring up the same case again, but this is sort of what happened when I was reviewing DC Universe Online. Scores were scattered from the low 40s to the high 80s with the average being 70 or so, and for a brief period I felt the urge to lower my score of 9, as if there were something in the game I had missed completely. Nearly 30 days down the line and if anyone asked whether my opinion on the game has changed, I would comfortably say no.

    • That bit surprised me too. Should the reviewers who fit the average be rewarded just because they’re in line with everyone else? Is that how reviews are to be judged?

      I don’t think so.

  5. He does have a point, what my daughter finds to be the best games ever and a 10/10 would have the piss took out of them the majority of people.

    Hannah Montana The Movie: The Game was a pile off poo, but everything that made it a pile of poo didn’t matter to here.

    As far as his last point goes, there is a Metacritic for reviewers (well, reviewing sites anyway) though its called Metacritic which allows you to view a sites profile and shows you how much they vary from the industry average.

    • I’ve noticed that an increasing number of games are failing to get reviews at all (or not enough to reach the 4 required to get a metacritic score). It is predominantley for games that cater to those that know less about games and as a result probably need to know more about whether a game is any good.

      Hannah Montana is a prime example. It was great for its target audience but out of a 11 Hannah Montana games only 2 have enough reviews to feature on metacritic.

      Gran Turismo 5 has 80 and Halo Reach has 99, at some point more and more reviews stop being useful. These are also 2 games where reviews were unlikely to effect sales. Yes reviews serve purposes beyond informing game buyers but the quantity of reviews is massive waiting toward the “fashionable” games.

      One review has alwasys stood out to me which was in OPM a while back for EyePet. The reviewer got their kids in to try it out and used what they saw from their kids playing it and from what they said in order to complete their review. The review was based on how well it worked for its target audience. Unfortunatley I haven’t seen this since.

  6. Reviewing is very subjective I think, because ultimately it’s based on the individual’s experience of the game. Which, depending on the individual is always going to be different. Which, like the act of translation, is up to interpretation. There is no right or wrong review, there is just that particular reviewers interpretation of how the game does. Of course reviewers should try to be objective, so it shoudln’t matter whether you have knowledge of previous games in the series or etc, because the quality of the game should speak for itself, should it not? It’s nice to have background information or being a fan of that series or genre, but that runs the risk of predudice. For example, I could say that Resident Evil 5 wasn’t as good as Resident Evil 4 because it has a shorter story mode. But that isn’t relative of Resident Evil 5 is as a game on its own. It may be shorter than other games, but it could still do things just as well. But then the risk of having a non fan review is that they may never have a positive experience, thus leading to an instantly negative review. My point is, yeah the process of reviewing is flawed somewhat, but that’s the beauty of it. Reviewers are just guidelines for peoples’ purchase. They are not a final verdict.

  7. He evil?

  8. hmm ok.. now go and make Elite IV :)

    • I am firmly behind this sentiment.
      In fact, I’ll offer David Braben a deal: Give me Elite IV or some HD remakes on the download stores and I’ll buy a trophy for whatever reviewer you like most ;)

      • To use an annoying internet meme : +1

  9. One careful grenade in this comments section and we’d kill most of TSA staff. Wow. Sorry, Dave. Collateral damage. :-p

    • Ah, tits. More innocent people. I’ll behave.

      • I’d survive. I always survive. Like some kind of gamesplaying super-cockroach. that wears pants.

      • I heard pants were optional in your house.

    • Well his comments are aimed at us rather than you.

    • Why a Grenade Mike? Just send me and i will guarntee you that they would all be dead in minutes. My rates are very affordable and i always require payment up front.:P This message will self desrtuct in 20 minutes. PS. stop stealing my teabags.>:P Or use a “laser” to hold them ransom for 1 million pounds. *laughs evily*:P

  10. I’m all for a “metacritic of reviewers.” I think this is a fantastic idea. I invest so much time in a review as a sign of respect to the developer; creative people who have invested a lot more than the hours I’ve spent playing their game for review.

    Some reviewers are consistently brilliant in their objective and informed analysis. Others are complete hacks and it physically upsets me that their opinions make up a metacritic aggregate for any game.

    • Metacritic of reviewing sites, is on each site’s profile page.

    • And where abouts do I find that? (I just had a look. Not easy to find. Or I’m dumb. Very well could be b) here)

    • The Gamesmaster Killzone 3 review annoys the hell out of me. To summarise the review: Better graphics, not much else has changed. Score – 77%. If it’s basically Killzone 2 with beter graphics, why does it score 77% when Gamesmaster scored Killzone 2 as 91%?

      Im not saying 77% is wrong, but the score doesnt make sense with the review.

      I’ve yet to read a TSA review that I disagree with. I’ve had to stop for coffee breaks for a couple of them, but that’s a good thing :)

Comments are now closed for this post.