Sony Computer Entertainment America’s President and CEO Shawn Layden has spoken to IGN about the botched Driveclub launch and gave a rather flimsy excuse.
In the development cycle, we try to do all things. In the development cycle, we try to test against every possibility. We have a [Quality Assurance] team, we have a QA plan. You do a beta test, you scope against that. But now, in a connected world, you can’t effectively test in your house or in your beta group what it means to have 50,000, 100,000, 200,000 users hit your service. And the guys [at the studio] are struggling with that. It’s throwing up things they had not anticipated.
You can’t test for 50,000 players in a beta. Shawn, let me draw your attention to the stats from the Destiny beta.
As you can see Destiny had over 850,000 players logged on simultaneously, quite a few more than then the 50,000 that you say you cannot test for, and 400% more than your maximum suggestion of 200,000 players. I’m pretty sure Evolution could have had 50,000 players connected to the Driveclub servers if they had created an open beta rather than locking it behind closed doors.
Shawn also said Evolution are hard at work on fixing the game but “it is going slowly.” A new patch went live today which includes a load of fixes plus support for the first expansion pack which will be free.
Source: IGN
beeje13
I can’t believe there arent tools for developers to use that SIMULATES 50000 or however many connections at once.
double-o-dave
Hold on a second guys, I may be reading this wrong, but he isn’t saying you can’t test 50,000 people in a beta.
He’s just stating they couldn’t effectively test what stress 50,000 people would have in their beta group as they may have only had 1,000 people in the group.
I’m by all means not defending the DriveClub Cluster F*ck but I don’t think he’s telling lies, I think people are interpreting this wrong.
bunimomike
His mention of a beta test is a bit vague. If it’s “open” then they could have tested for far larger numbers. If he’s saying they couldn’t do that in-house then he’s right. He just also comes across as staggeringly short-sighted about what they should have done next. Silly buggers.
Amphlett
In that case it would be a simple process of making an open beta where members of the Playstation playing public can download and play the game.
TSBonyman
Clearly, all they needed to do was consult with the IT guy who carries the entire internet around in a small box.
Forrest_01
Who, Jen from the IT Crowd?? :D
Amphlett
I hope a Sony engineer doesn’t have the internet in a small box because I downloaded and put the internet on to a DVD for my Dad at the weekend.
3shirts
Only if it’s been completely de-magnetised
Avenger
I don’t think it’s the numbers that are so alarming to me, but the part where he says ‘you can’t effectively test […] in your beta group what it means for X amount of players to hit your service’. Correct me if I’m wrong, but yes you can actually find out what it means for so many players to hit the service. That is what beta testing is all about. Very unusual stuff.
Lyts1985
See, now maybe I’m just being naive but to me it seems this is exactly what they should’ve used the PS+ edition for. If they’d solely released that 2 weeks early then no paying customers would’ve been stung.
I guess from a business point of view that could’ve crippled sales what with all the subsequent problems, but at least it would’ve been tested and nobody would’ve felt screwed out of £50 they’d never have spent on a broken product…
3shirts
No, a large scale public beta is the answer. The PS+ version would have been expected to be a ‘final’ product so, as you say, the problems would have tanked sales. A beta is not final so when all these problems came to light, people (well, most people) would have understood that bugs are expected.
Lyts1985
Yeah true, but these days a public beta seems to be akin to a demo version which, considering the cut down content, isn’t too far off the PS+ version.
Either way, it needed some sort of mass test for load and bugs. Neither of which happened here. This announcement just rubs salt in the wound…
Spence1115
*raises hand* Hi, professional tester here. Yes, yes you can. You can simulate as many users as you want. Is it free? No. Is it really expensive? No. Is it pretty easy? Yes.
Or, an open beta. That will also work for load testing, but ideally you do it before hand.
xIONx
Someone point them to the Evolve alpha infographic which demonstrates that not only is this guy on drugs but the figures quoted can be smashed over a weekend alpha utilising multiplatform architectures.
Like the Destiny alpha it shows how a UAT should be carried out.
I got my refund from Asda after a long conversation with a customer service lady and the butch lesbian security guard who refuted my right to a refund as it was outside the 28 day period. Driveclub. Club. Club doesn’t work. Money please thank you.
DICEclub that would sum it up.
Like Gutter ball’s “he pulled a Munson” I think the software equivalent is “they launched like Driveclub”
I think the space shuttle Challenger launch was more successful. (sorry)
xIONx
I cannot edit. The movie was Kingpin. Meh.
double-o-dave
I didn’t manage to get into a single game of Evolve throughout the Alpha.
Kennykazey
This game is the biggest flop of the year. I’d be surprised if it gets a sequel.
Maybe it’s time to do another Motorstorm instead. In the vein of the first two. That’d be something on the new hardware… And mainly offline too!
DeadlyChipmunk
How he thought that statement would go down well when people are still wondering when the main game will be 100% complete or the PS+ one will actually turn up beggers belief!
TSBonyman
On a positive note .. Photo mode results are looking good.. Oh and get that GAF thread title.. :P
http://www.neogaf.com/forum/showthread.php?t=935041
double-o-dave
Yeah, I’ve took some pretty cool pics already. Great feature. It’s worthy of a TSA DC photo forum page