XI
you are not logged in
Opinion

Metacritic: Analysing the Analysis

What aggregate scores can tell us about the current state of gaming.

Metacritic.

That word has probably sparked a multitude of thoughts in your mind: you likely think it’s either a redundant aggregate of quantified qualitative statements, or the most comprehensive database of review scores from a selection of the biggest and best sites on the net. Regardless of your opinion, what it does do is provide a useful tool in determining the median reception of a title amongst a selection of sources the company deems representative of the “gaming press”.

But what can we glean from the service’s scores when taken in batches? What do the top 20 games from across all of the three home consoles say about the current landscape of interactive entertainment? Are there any patterns that begin to form, any discoveries we might make from the clinical precision of numbers on spreadsheets?

At the time of writing – 4th September 2011 – the following are the top 20 games in the Metacritic archives…

So what do these numbers mean in a wider context than simply the best games by platform?

In terms of high scoring exclusive titles by system, Nintendo’s mini-monolith comes out on top, featuring eleven games, PlayStation 3 trails behind with five and the Xbox 360 in third with four. These figures mean one of two things, either that the Wii is a haven for creativity far beyond Sony and Microsoft’s offerings, or that the company has had to rely on their own titles to provide their critically acclaimed output without the backing of third parties.

I’d argue it’s the latter any day of the week and the statistics would back up the point. Grand Theft Auto IV, BioShock, Mass Effect 2 are all conspicuously absent from the Wii when they appear on both of the other consoles. One might also extrapolate that if the Wii had matched the 360 and PS3 in terms of power from the very beginning, Nintendo would be leading the charge in terms of quality and widespread appeal amongst core gamers. What does this therefore say about the WiiU? Could this metric be a solid argument for Nintendo’s new console?

The numbers also confirm the long-held argument that new IP is being marginalised by sequels, though here they shed a little more light on why. The Wii once again comes first for top 20 rated titles not based on an existing IP with five entries, the PS3 is second again with four and the 360 third with three. These numbers are a little fudged by my definition of “new property”; DJ Hero I counted as a new IP but Red Dead Redemption, Call Of Duty 4: Modern Warfare and Super Mario Galaxy I didn’t, for instance, though arguably they are totally unlike their predecessors.


Titles like Super Mario Galaxy help Nintendo lead in our analysis.
What’s interesting to note however is that these numbers are not based on sales, they are based on reception from reviews from the press at large. The belief that sequels are damaging the industry by being stale becomes a myth: sequels and spin-offs have been more well received than original IP consistently this generation. Again this leads to further questions, is this simply because sequels are the only projects receiving significant funding to compete with the massive budgets of a Halo or Elder Scrolls? Evidence for this comes once again from the spreadsheet above, a large portion of the new IPs topping the chart were downloadable, a space that requires less investment of money from developers and publishers.

IPs divided by party also give a clear indication as to which console manufacturer is creating or securing the highest rated exclusives. Once again Nintendo wins the day with Super Mario Galaxy and its sequel taking spot one and two amongst their releases, Naughty Dog’s Uncharted 2 being the second best title on Sony’s hardware and Gears Of War and Halo 3 coming in at 8th and 10th amongst the American company’s portfolio.

Now by this point one might feel that Microsoft’s console isn’t doing too well in terms of games quality by platform but in actuality the reverse is true. We see by adding the total Metascores of the top 20 together that the Xbox 360 is by far the most critically well received line-up, collectively scoring 1880, the PS3 nabbing second with 1870 and the Wii massively falling behind with 1813, lacking a number of mid-nineties scoring titles. From here then it’s easy to see that Microsoft may not be the best at securing exclusives or creating a huge catalogue of quality first party titles, but it is home to the largest quantity of high profile third party titles.

Another confirmation we can pull from the numbers is that Japanese companies have lost their collective grip on the Western audience. The Xbox 360 features eighteen titles in its top 20 made in Europe or the United States, the PlayStation 3 sixteen and the Wii has just twelve, likely due to the high number of first party titles that feature so prominently.

It’s useful to bear in mind that Metacritic is indeed largely an aggregate of Western-centric sites and is therefore a perfect service to discover the opinions of gamer “taste-makers” in Europe and North America. What might shock though is just how few titles on the list are made in Europe. With the exception of Grand Theft Auto IV which is overall the most critically acclaimed game of this generation of hardware, it’s only the LittleBigPlanet and DJ Hero franchises, plus downloadable Art Of Balance, that find a place in the Top 20 across the three platforms. Perhaps the question on everyone’s lips right now should not be “why are Japanese games not grabbing Western gamers?” but “why aren’t European games grabbing Western gamers?”

Of course these results need to be taken with a pinch of salt. The critic scores that appear on this list are chosen by Metacritic, which may well have an effect on overall scores. Still, the numbers speak for themselves: Microsoft’s console may not be a haven to new ideas but it’s doing the best critically at time of writing and the opposite can be said of the Wii, with Sony’s effort consistently being a middle ground. Does this say something about the medium as a whole? I think so. Quality may not necessarily depend on radical new ideas as it once did but the sequels that are produced are of such a high calibre that this may not necessarily matter any more.

Read more: #
21 Comments
  1. TheShepanator
    Member
    Since: Nov 2009

    I personally take whatever metacritic says with a pinch of salt, but its a pretty good indication on how good a game is. I would say I enjoyed uncharted 2 more than GTAiv though.

    Comment posted on 20/09/2011 at 18:19.
  2. teflon
    Community Team
    Since: May 2009

    Slight flaw to the top 20 list being that there’s games outside the top 20 that have the same score as the 20th entry.
    So, Killzone 2 is on 91, just as Rock Band 2 and 3 are. Gears of War 3 is level with the 20th on the 360 too. Also, it’s great to see that Batman: Arkham Asylum is sitting pretty on 91 and 92 for the respective platforms, /also/ level with the 20th.

    However, a good analysis, and quite a good question you pose at the end, with regards to EU developers. I think that with the UK, it could be down to it being such a racing game dominated community. The only racing game to break into any top 20 being Forza.

    Not that other racers aren’t up there. There’s plenty that are in the upper 80s, such as Dirt 3 or F1 2010.

    Comment posted on 20/09/2011 at 18:20.
    • Darth Newdar
      Member
      Since: Oct 2010

      I personally am slightly suspicious of reviewers’ scores of racing games. The Great Eight seems to be everybody favourite’s score – a nice middle ground, I suppose, of good but not perfection. 9s and 10s are rarely awarded to racing games, for some reason.

      Comment posted on 20/09/2011 at 19:01.
      • teflon
        Community Team
        Since: May 2009

        If there’s one racing game on PS3 that should have broken out of that zone, it’s Wipeout HD. Except it was all ported tracks and vehicles from the PSP games, otherwise it might have managed it. Even so, it managed to pull in quite a few 9s.

        Each racing game I own this generation is flawed in some regard. GT5 isn’t a game, F1 2010 wasn’t quite the finished article, Dirt 3 wasn’t rally enough, Burnout Paradise had annoyances at launch that soon disappeared.

        I can’t think of any really perfect racers, and whilst games like Uncharted and Gears can come back with sequels that have stronger plots, better set pieces etc. etc. All that racing games can really do is more racing, which can feel a bit “seen it”.

        Comment posted on 20/09/2011 at 22:56.
  3. 3shirts
    Member
    Since: Aug 2008

    I have a few sites (mainly this one) whose reviews I tent to trust more. I will read those and maybe then have a glance at metacritic to see the general consensus. If there are any outliers I might see what it was that made them such.
    To me, it is a useful tool but I wouldn’t base anything on just numbers.

    Comment posted on 20/09/2011 at 18:24.
  4. Foxhound_Solid
    Is a smart cookie.
    Since: Dec 2009

    Great read, enjoyed that. Metacritic is just an amalgamation of individuals opinions. Nothing more or less. The operative word is opinion.

    Comment posted on 20/09/2011 at 18:54.
  5. Soild_Nat
    Member
    Since: Apr 2009

    I consider myself to be a metacritic of sorts. I have played games for that long now that I pick and choose what I am going to buy based on what I have liked in the past and if something is similar to it.

    I only ever read reviews on TSA these days simply to see what features and overall feel the game has and the final score never persuades me to change my mind, if I think I will like something then I am usually not wrong,

    Comment posted on 20/09/2011 at 19:10.
    • 3shirts
      Member
      Since: Aug 2008

      Doesn’t that mean you miss out on clever original IP like Mirrors Edge, Flower or Heavy Rain?

      Comment posted on 21/09/2011 at 08:31.
      • Soild_Nat
        Member
        Since: Apr 2009

        No it doesn’t as I have played all of those games and loved them. I was drawn to Mirrors Edge because of the unique art style and game play. Flower because it was so different and from a well respected studio, after missing out on Flow I wanted to see what all the fuss was about. Heavy Rain drew me in because of the story and unique take on interacting with a game, I even replayed it when the Move patch came out.

        Comment posted on 22/09/2011 at 11:40.
  6. Darth Newdar
    Member
    Since: Oct 2010

    Nice analysis. I think it is widely acknowledged that Nintendo’s first-party output is of the very highest quality. Your point about sequels is absolutely right – the video game industry is unique in that the sequel is almost always an improvement over the original IP (the exact opposite to film).

    It’s worth remembering that Metacritic is all measured on reviewers’ scores – not their words. I would argue this leads to some slight funny average Metascores, in places. For instance, I wonder if downloadable games often end up with a higher metascore, because reviewers have much lower expectations when they sit down to play the game. Reviewers expect less of them, and are therefore more impressed when they are of a high quality (such as Braid, Pac-Man, Peggle, and Limbo). So those games get 9s, while a game where the reviewers expected more of the game – such as Gran Turismo 5 – gets slightly lower scores.

    Comment posted on 20/09/2011 at 19:14.
    • a inferior race
      I'm special
      Since: Jul 2009

      Some reviewers do take price into account. As I would expect a £40 game to have more flashy 3D graphics than a game for £10.

      Comment posted on 20/09/2011 at 21:53.
  7. TSBonyman
    Member
    Since: Dec 2009

    Interesting stuff Watch- i mean xeroxeroxero :)
    I’m surprised to see The Orange Box hasn’t figured so high in the PS3 camp as it has for 360, were there xbox achievements?

    Comment posted on 20/09/2011 at 19:59.
    • david24
      Member
      Since: Feb 2011

      its in the low 80′s as it was apparently a ‘horrible port’. i brought it for portal and have played through most of half life two and am yet to find any glitch’s or problems. it is very low res but still looks quite good although i haven’t sen it on xbox

      Comment posted on 20/09/2011 at 20:51.
      • bacon_nuts
        Member
        Since: Mar 2011

        The main problem people have with it is the audio, there are blocks in the game where there is an annoying static and crackling upsetting the audio of the game. Other than that in my experience the game works fine. Except TF2 is awful on PS3, it’s just empty in comparison to PC/MAC. This stops it from being higher.. I think.

        Comment posted on 20/09/2011 at 23:50.
      • ProjectJAY
        Member
        Since: Aug 2009

        The PS3 version of The Orange Box got some criticism for much longer load times and more dips in framerate (plus connection issues in TF2) compared to the other versions. Played identically, though, so it probably didn’t deserve as much flak as it got.

        Comment posted on 20/09/2011 at 23:51.
    • TSBonyman
      Member
      Since: Dec 2009

      I absolutely loved The Orange Box (still do, although i’ve never bothered with TF). I had one occurrence of nasty slowdown in HL2 Episode one but i didn’t notice any major flaws with the other games. It’s a pity if people were put off by the negative reports but i’m sure many people who played Portal2 will have sought out The Orange Box since then. If you haven’t you’re missing out. :)

      Comment posted on 21/09/2011 at 01:59.
  8. tonycawley
    Pint! Pint!
    Since: Feb 2009

    Very good analysis and very interesting reading, what I believe is missing from it though, crucially, is a cross-reference of metascores and total sales, or even opening week/month sales. We could then have started to get an understanding of the link between review scores and game sales.

    Comment posted on 20/09/2011 at 20:30.
  9. david24
    Member
    Since: Feb 2011

    I’ll always check the metascore of a game I’m thinking of buying and use them to get a wide range of reviews. i find its a great tool to find quality games that i might of missed. tsa’s reviews are great and easily the best written but the scores seem to always be a bit cautious and tend to only be 8, 5 and 3 with the odd 10 or 7 through in.

    Comment posted on 20/09/2011 at 20:48.
  10. Scythegpd
    Member
    Since: Jul 2010

    I don’t really follow reviews. Why?
    1 – I’ve yet to find a place that has a comprehensive set of review guidelines, thus there is far too inconsistency with reviews, some games being penalised for not being groundbreaking and massively different from their predecessors whilst others being overlooked despite being the same thing. Or gameplay issues built into a massive thing with some games whilst being ignored in others. People say reviews are personal opinions but my view is that a person who claims to review professionally should not have any personal opinions or preferences involved, a “professional” should treat a review as an objective comparison against it’s peers. Professional reviews should be an objective assessment, not a personal opinion piece. Without a strict and published set of review criteria and policies no review is trustworthy as it’s too easy to apply subjectivity when reviews should be objective.
    2 – I’ve seen too many occasions on metacritic where they just seem to overlook certain review sites in some instances but include them in others. It’s bizarre that some games have review from 60+ sites but others only have reviews from 40+ sites despite all the same 60+ still reviewing the game. Or putting bizarre scores on review sites that don’t actually provide a final score (be it a number or a letter)

    I tend to more trust my own judgement. I’ve played many poorly reviewed games and thought they were brilliant and on the other end of the scale highly reviewed games and thought they were rubbish.

    Comment posted on 20/09/2011 at 21:24.
    • nofi
      One for all.
      Since: Forever

      “I tend to more trust my own judgement. I’ve played many poorly reviewed games and thought they were brilliant and on the other end of the scale highly reviewed games and thought they were rubbish.”

      I think that’s key. Take bits from a few sources, play a demo if there’s one and make an informed decision based on everything you know, including the dev and the publisher too.

      Comment posted on 20/09/2011 at 22:29.

Leave a Reply

You must be logged in to post a comment.

Latest Comments