173 replies to this topic

[KuruptU4Fun]

Barnaby Jones, on 16 April 2013 - 11:18 AM, said:

Its quite simple... Given the time they took to design what is effectively stat tracking (ELO) and the length of time they have been collecting actual data to populate their formula, to only figure out this type of issue now speaks very clearly to the lack of any reasonable amount of internal testing prior to pushing it out on the servers. I dont have to have developer level knowledge of the system in order to recognize a major bug should have been found and fixed far earlier than now, and given the length of time between the death of 8mans to the discovery of the bug, the only explaination is that they didnt test ELO properly on their own internal test bed, if at all.

You also realize I assume that almost every ELO ranking system created is based off of the one that Xbox started using when they created LIVE. And that one is still far from perfect.

[Barnaby Jones]

"PGI, You have literally 0 idea how QA is run."

fixed that for you.

[Barnaby Jones]

KuruptU4Fun, on 16 April 2013 - 11:47 AM, said:

You also realize I assume that almost every ELO ranking system created is based off of the one that Xbox started using when they created LIVE. And that one is still far from perfect.

I am not saying all must be perfect. I am saying that the repetition of errors, the frequency of bugs, and the lack of speed in correcting them tells me that PGI has understaffed QA, if they have any at all.

ForestGnome, on 16 April 2013 - 11:49 AM, said:

HAH, so you cry about not having a QA, then suggest they run scripts instead? Is that what i'm hearing?

You're whole argument is turning into BAWWWW BAWWWWW I WANNA COMPLAIN ABOUT SOMETHING!

No i am pointing out that scripts would have spotted this inflation issue months before ELO went live.

This bug being discovered now says they didn't even attempt to run scripts at all. Scripts along with live data is what should have been done. PGI appears to have half assed their testing.

[M4NTiC0R3X]

a.) they're not ''screwing up''
b.) it's a beta..
c.)

[Barnaby Jones]

ForestGnome, on 16 April 2013 - 12:21 PM, said:

1. most people on QA do not have the know-how or the tools to create a script required for that.
2. Maybe you should write a script for them to use if it is that big of an issue for you? I'm sure if it is as serious as you say, they'd love you for making one for them. You might even get a forum banner.
3. If you actually READ anything the devs post you would find out that ACTUALLY DO run automated test scripts and that is EXACTLY WHY so many of those graphic bugs were missed last patch (disco lasers anyone?)
So seriously, stop crying already. You've already been proven a hypocrite, and now you have shown your true ignorance.

1. so you agree with my point that PGI's QA is either inept, inadequately staffed, or half assing their job.
2. ahhh, the NO U response... it really proves your point.
3. Doesn't this run counter to your point 1? And once again, if they do run scripts AND test beds to discover the issues, but those issues were missed in the last patch because they DIDN'T run the scripts AND test them on test beds BEFORE putting it out on the live servers, doesn't that prove my argument yet again that they are half assing it?

Save your personal attacks, they just make you look irrational.

Edited by Barnaby Jones, 16 April 2013 - 12:32 PM.

[Loler skates]

Barnaby Jones, on 16 April 2013 - 12:29 PM, said:

1. so you agree with my point that PGI's QA is either inept, inadequately staffed, and half assing their job.
2. ahhh, the NO U response... it really proves your point.
3. Doesn't this run counter to your point 1? And once again, if they do run scripts to discover the issues, but those issues were missed in the last patch because they DIDN'T run the scripts BEFORE putting it out on the live servers, doesn't that prove my argument yet again that they are half assing it?



Save your personal attacks, they just make you look irrational.

Having worked QA, I can assure you that for most testing automation is actually a really bad way to go about it.

Automated testing will not pick up problems unless the testing suite is incredibly advanced and tests rather exhaustively.

Testing by hand is far more likely to pick up problems. The issue I have is that many of the problems *first implementation* of artemis for instance should have been caught before release. Evidently there isn't enough hand testing of new features going on.

Automated testing is really only as useful as the problems the person writing it can conceive of tests for given the nature of bugs being unexpected/ outside of expectations automated tests will usually miss them.

To much of what PGI does is by the seat of their pants, instead of taking a good hard look at things and building stuff from a systems perspective.

[Barnaby Jones]

Loler skates, on 16 April 2013 - 12:34 PM, said:

Having worked QA, I can assure you that for most testing automation is actually a really bad way to go about it.

Automated testing will not pick up problems unless the testing suite is incredibly advanced and tests rather exhaustively.

Testing by hand is far more likely to pick up problems. The issue I have is that many of the problems *first implementation* of artemis for instance should have been caught before release. Evidently there isn't enough hand testing of new features going on.

Automated testing is really only as useful as the problems the person writing it can conceive of tests for given the nature of bugs being unexpected/ outside of expectations automated tests will usually miss them.

To much of what PGI does is by the seat of their pants, instead of taking a good hard look at things and building stuff from a systems perspective.

Precisely.

I am not advocating nothing but scripts, nor am I unaware of the added time and costs of hands on.

I am stating that all of the ups and downs we have seen with respect to bugs and balance makes me question the capabilities (hell sometimes even the existence) of a QA dept inside of MWO.

PGIs focus seems to be too much on MC, and not enough on a quality product, and THAT is a fundamentally losing position the longer they continue to have it.

[Matthew Craig]

http://mwomercs.com/...ost__p__2247592

[Barnaby Jones]

of in other words, the amount of testing they view as reasonable before pushing something to live is way off. If the options are implementing **** in a buggy state, or taking longer to implement stuff bug free, they need to choose the latter.

[Barnaby Jones]

Matthew Craig, on 16 April 2013 - 12:46 PM, said:

http://mwomercs.com/...ost__p__2247592

As mentioned in a prior post we are bringing online tools internally to help provide much wider test cases for FPS and we'll be working shortly to improve performance where we see it has degraded from previous builds.

The addition of more resources for QA to do their job is a positive step, but forgive me if I take a negative wait and see approach before I declare PGI has gotten their **** together( and return to my previous position of defender of PGI, rather than my current attacker of PGI position.)

[Barnaby Jones]

ForestGnome, on 16 April 2013 - 12:49 PM, said:

1. No, i'm saying you seem to assume QA are gods.
2. Seriously, if you don't know anything about code or scripting, you should be telling people to go code or script.
3. Again, you seem to think QA are gods. Omnipscient beasts capable of spotting a bug before it even happens. Beings so far beyond humans that the moment they even discover a bug, it instantly vanishes into a quantum void.

you are arguing from an emotional standpoint, not a logical one. address the logic in my statements rather than trying to diagnose my psyche.

Quote

Or you just need to play a different game.

So my desire to see this game be successful and my anger when I see roadblocks to that success should lead me to just give up and go play a different game? sounds like a childish "I'm taking my ball and going home" reaction to me.

[Satarosun]

Ranek Blackstone, on 15 April 2013 - 04:47 PM, said:

The art department isn't involved in the bug hunting.

[Loler skates]

ForestGnome, on 16 April 2013 - 01:17 PM, said:

But you obviously don't care about this game enough to read what the DEV's say on the matter, and you said it yourself that this game needs omniscient QA to be successful, and that's just never going to happen. So, go play another game.

THat dev post acts like these bugs just appeared two weeks ago.

Hud bugs have been prevalent for what? 5 months+?

I think they've had more than enough time to nail them down.

[Barnaby Jones]

ForestGnome, on 16 April 2013 - 01:17 PM, said:

But you obviously don't care about this game enough to read what the DEV's say on the matter, and you said it yourself that this game needs omniscient QA to be successful, and that's just never going to happen. So, go play another game.

you seem to make alot of assumptions.

I do read what the devs say.