Game Bug testing

Discuss and discover all the great games of yesteryear!

Moderators: mknott, NickThorpe, lcarlson, Darran@Retro Gamer, MMohammed

Post Reply
James A
Posts: 3486
Joined: Sun Dec 11, 2005 7:34 am
Location: Wrexham
Contact:

Game Bug testing

Post by James A » Wed Feb 01, 2012 8:23 pm

Has anyone ever done this as a job or known anyone who has. I got wondering about this after getting Duke Nukem Forever. Ive had to start the game now four times, each time i come back to it theres a bug freezing the game game that stops the charecter moving. So i delete all the game saves restart and the same thing happens again which is making the game unplayable almost.

Different sort of thing but with COD Black Ops i managed to get through three of the last four levels without firing a shot as the AI had all stuck to the spot not to mention alll the online bugs. Theres also plenty of other examples but these two really stick out as obvious things.

So how much effort do game companies put into bug testing and trying to find these faults before releasing a game. Some seem almost flawless whilst others appear as if not a care in the world has been put on them.
Image

User avatar
SIR'86
Posts: 80
Joined: Mon Mar 01, 2010 6:38 am

Re: Game Bug testing

Post by SIR'86 » Wed Feb 08, 2012 10:50 am

I've done games testing, and have managed a test team at one point in my career.

Companies put enormous resources into testing. No games company deliberately wants to release buggy games.

However games are extremely complex, and bugs will happen. You can write a 10 line program which has over 100,000 routes through the code - you couldn't possibly test all of them in a reasonable time frame, so you have no hope to test every code path in modern games which have millions of lines of code.

Some game types (e.g. RPGs or multiplayer games) are even harder to test, as they're non-linear and have many random interactions.

Another issue is that your test team will have for example 40 people playing the game for thousands of hours, whilst the released game will be 'tested' by millions of people playing for millions of hours, and so it's likely they'll find bugs you've missed. It's even worse on PC games where the hardware specs aren't fixed like consoles.

As a game gets closer to release only the major bugs such as crashes are fixed, as each fix carries the risk of introducing more unknown bugs, so minor bugs may be left in.

In fact most games ship with known 'waived' bugs in the bug database. Some of these cannot be fixed without causing more serious bugs, others are too hard to replicate, and only trigger rarely, or have only happened once. Others are too minor to fix due to the risk of introducing more bugs.

The ability to patch games has made it less essential for dev teams to catch every serious issue for release, but that doesn't excuse those times when games have been released in an unplayable state out-of-the-box.

User avatar
greenberet79
Posts: 3378
Joined: Thu Jan 21, 2010 10:37 am
Location: Liverpool

Re: Game Bug testing

Post by greenberet79 » Wed Feb 08, 2012 11:14 am

That's really interesting, I never knew anything about testing.

Could it be that the next generation of games would be so complex and therefore so potentially ridden with bugs that it would actually be pointless making them so complex in the first place??
"Very soon our luck will change, we'll do what we need to do to win matches, and we'll fly."

User avatar
ToxieDogg
Posts: 8356
Joined: Thu Dec 22, 2005 8:54 am
Location: Vice City, a.k.a. 'Liverpool'

Re: Game Bug testing

Post by ToxieDogg » Wed Feb 08, 2012 11:28 am

James A wrote:Has anyone ever done this as a job or known anyone who has. I got wondering about this after getting Duke Nukem Forever. Ive had to start the game now four times, each time i come back to it theres a bug freezing the game game that stops the charecter moving. So i delete all the game saves restart and the same thing happens again which is making the game unplayable almost.
Which version is that? I've played through the game start to finish on Normal, and almost to the end again on Very Hard and not had that happen to me ever. :?

As for bugs, I don't mind them too much so long as they're not game breaking. It's only the ones like Far Cry 2 originally had that corrupted your game saves and prevented anyone from finishing the campaign before it was patched that I think are really unforgiveable.
Sig pic temporarily removed...The Magnificent 7 will ride again 8)
Lost Dragon wrote:The 1st rule about Feedback is..

You do not give Feedback!

User avatar
random_dave
Posts: 680
Joined: Wed Mar 05, 2008 7:30 am
Location: Sheffield

Re: Game Bug testing

Post by random_dave » Wed Feb 08, 2012 11:58 am

I started in 'the industry' with testing, and as a warning i'll say that it's incredibly sould destroying.

first off, games that are released with a high percentage of faults are usually nothing to do with the QA team failing, most of the time it boils down to increased pressure on release dates from publishers, especially in the current financial climate, and shipping a game that you can mostly get from start to finish is preferable to spending another few hundred thousand pounds on development to make it right.

Crunch time is the worst, just because you have done all your tests and regressions for the day, and it's already 4 hours into overtime, if the producers decide that they may need you later as people are still working you will have to keep ad-hoc testing until they say you can go home. This is even worse if they supply a version of the game that day which is practically untestable, as you then have to find some way to professionally pass the next 6 hours till the promise they'll have something that works, only for you to get to the 8th hour and them tell you that they won't (good job there's nothing better you could have been doing at home in that time, if they'd sent you home straight away, you know, like eating and sleeping)

Then there's essentially 2 types of QA, publisher QA and developer QA
Developer QA positions are getting increasingly rare, but are generally the most interesting and tolerable. You will be working with the latest builds, and often now doing 'smoke' or 'burn' tests to get as much coverage on the games features in the shortest time possible to make sure that it's stable enough to be worth spending the time delivering it to the publishers QA and them spending a large amount of time testing it more thoroughly. if you're lucky you may even get to do some minor design/audio/production tasks too, and actually have a hand in the game. You can often be expected to work with a designer to help balance the difficulty and accessibility of the game too.

Publisher QA I have never done, but generally get a pretty bad impression of it from people i get to speak to about it. These days this is where the majority of the testing will get done, and teams are assembled on a short term contract basis with as little chance as they can get away with of keeping you under their employ beyond that. They seem to have less issues with long hours, due to better shift patterns, but this comes at the price of the environment being a lot less laid back, and I've heard of many restrictions on not just lunch but short breaks for the loo and refreshments too. You will often be testing a version of the game that is many days or versions out of date, and there is very little way to have an input on anything that constructs the game. You may however get posted to work away in a development studio to booster the development qa, and although you still wont get to help with any of the other jobs on the team the other benefits of developer qa will then start to apply, although you will need to be prepared to live in a hotel for a few months

I'm going to stop typing now, there's no doubt a lot more i could say, but i'll give it a rest until something comes up ;)
MSN: dlanki [at] Hotmail.com
360: lanki of badger
psn/twitter/playfire: random_dave

User avatar
Megamixer
Posts: 15081
Joined: Fri Jan 23, 2009 6:18 am
Location: Staffs, UK

Re: Game Bug testing

Post by Megamixer » Wed Feb 08, 2012 12:30 pm

I think Betheseda need more bug testers :wink:

Like random_dave says above, it's all down to publishers who I imagine are worse than ever these days where deadlines are concerned. I personally wouldn't want to even consider doing bug testing as job because I know I'd end up hating games as a result of working with them all the time. Maybe not an inevitability but a horrible possibility all the same.

The biggest bug that made me gnash my teeth was the save data corruption glitch in Soul Calibur III. Strange that such a nasty and apparent glitch made it past testing and even worse that it wasn't patched for the Platinum re-release.

Also, ports made for last gen platforms when the main emphasis is on the current gen. These efforts tend to be full of bugs and generally crap because it seems that nobody cares enough about a crusty old console to make sure that the software is decent. Just Cause on PS2, I'm looking at you.
Retro is a state of mind, and cares not for your puny concepts like dates and calendars.

James A
Posts: 3486
Joined: Sun Dec 11, 2005 7:34 am
Location: Wrexham
Contact:

Re: Game Bug testing

Post by James A » Wed Feb 08, 2012 1:42 pm

ToxieDogg wrote:
James A wrote:Has anyone ever done this as a job or known anyone who has. I got wondering about this after getting Duke Nukem Forever. Ive had to start the game now four times, each time i come back to it theres a bug freezing the game game that stops the charecter moving. So i delete all the game saves restart and the same thing happens again which is making the game unplayable almost.
Which version is that? I've played through the game start to finish on Normal, and almost to the end again on Very Hard and not had that happen to me ever. :?

As for bugs, I don't mind them too much so long as they're not game breaking. It's only the ones like Far Cry 2 originally had that corrupted your game saves and prevented anyone from finishing the campaign before it was patched that I think are really unforgiveable.
360 version. Not sure whats going on with it. Going to restart it again if i get chance tonight but if it happens again then its gettin traded in.
Image

Post Reply

Who is online

Users browsing this forum: No registered users and 6 guests