Jump to content

Ability to Properly Test Game Non-Existent


23 replies to this topic

#1 nungunz

    Member

  • PipPipPipPipPipPipPip
  • Urban Commando
  • 612 posts

Posted 03 November 2012 - 10:30 AM

To whom it may concern,

As a Legendard Founder, closed beta tester and now an open beta tester, I really think it is a bad decision to stop wiping the game. Yes you went to 'Open Beta', but it's in name only. Without resets you've essential launched the game. The problem I see with this is that a lot of
functionality has yet to be implemented and a lot of tweaks still have to be made.

I appreciate the initial non-reset mentality, especially in order to test out how free-to-play will react with the trial mech mechanics and economy. It's very useful data to be had and needs to be done. But why not just disable Founder's mechs for a week or two and have *everyone* go through the trial phase. It would also give a better indication of the whole PUG vs. Premade griping that is going on the forums when everyone is relegated to the same trial mechs. Consider it a 'control group' if you're aware of how the research and development process goes.

Another case in point would be the proposed new double heat sink functionality with 0.14 heat per second as opposed to 0.20 heat per second. I would very much like to test this out as my initial calculations for my mechs and a even some of the PPC and Large Laser builds (Awesomes, K2 Catapult, Large Laser Atlas or Centurion) show that they either make heat dissipation worse so single heat sinks are better or that the gain is so marginal it isn't worth the C-Bill investment for upgrading, paying for extra sinks, and the associated repair costs. In either case, my conclusion is that the 0.14 heat sinks are not worth buying for *any* mech.

HOWEVER! I want to test this out.

But without resets to the game and to the amount of MC (and without injecting MC for the free testers) I really don't want to spend any actual money and C-Bills just to test something out. I would assume that most others (especially free players) feel the exact same way. Because of this you will probably get very biased and skewed data!

I really, really want to be able to test my math and hypothesis out(essentially a simulation) in-game (data for verification/validation), but have no incentive to do so. The current way the game runs actually discourages proper game testing.


The DHS example above is just one trade-study on the phase of the 'beta' we're in (it's not a true beta at all, you've essential launched that game by removing resets). Any ther new maps, additional features, and modifications to the mechanics, etc won't be properly tested with accurate and valid data to back up the research and balancing process unless wipes (and MC injections) still occur. I really hope that everyone at PGI takes this in stride and realizes that
essentially releasing the game only hurts the development process of this game. I would very much like to see MWO to survive, but in order to do that, we (as testers) need to actually be able to test the game and be able to give the developers at PGI useful (and unbiased) data and opinions.

We can't do this with how the game currently functions. I urge the developers at PGI to realize that the way they released to 'open beta' was a very, very big mistake. Make an announcement to admit this mistake; and let us actually get back to a productive testing environment.

As an example to my above DHS anecdote, here is what I've tested on my current mechs. It's a limited pool of chassis (but are some very common ones) and I would like to test out the PPC Catapult-K2 (rather than guass builds) as well as various PPC and Large Laser Awesome builds. But I have no reason or incentive to do so....in fact I am completely discouraged because in order to test I would have to spend a large amount of my personal money (which I already have done) or spend *a lot* of hours playing the game to earn the C-Bills in order to test the effect of 0.14 heat sinks vs. 0.20 heat sinks.

I will reiterate:

*This is not a good testing environment*

Please, please, please return the game to a proper and useful testing environment! It helps you and it helps us.

#2 Abrahms

    Member

  • PipPipPipPipPipPipPipPip
  • 1,478 posts

Posted 03 November 2012 - 10:31 AM

You thought PGI would do what is logical here?

#3 Warrax the Chaos Warrior

    Member

  • PipPipPipPipPipPipPip
  • Bad Company
  • Bad Company
  • 925 posts
  • LocationMyrror

Posted 03 November 2012 - 10:41 AM

I agree that the "testing" we are doing right now isn't generating much in the way of useful data, but not for exactly (or only) the reasons that you list. The bad netcode has made so many weapons not work anywhere near like they are supposed to, and the lack of collisions has made all battles play out in a completely unrealistic way. Those two issues more than anything else have broken the game to the point that any data gathered is completely skewed.

As far as that relates to the economy being a part of why we aren't in a good testing environment right now, I think that these last couple of patches might qualify as enough of a "major catastrophe" to justify another wipe before release. That would be rough on free players, but some sort of an adjustment to the initial free player experience might make up for it.

#4 Odins Fist

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 3,111 posts
  • LocationThe North

Posted 03 November 2012 - 10:48 AM

View Postnungunz, on 03 November 2012 - 10:30 AM, said:

To whom it may concern,

As a Legendard Founder, closed beta tester and now an open beta tester, I really think it is a bad decision to stop wiping the game. Yes you went to 'Open Beta', but it's in name only. Without resets you've essential launched the game. The problem I see with this is that a lot of
functionality has yet to be implemented and a lot of tweaks still have to be made.

I appreciate the initial non-reset mentality, especially in order to test out how free-to-play will react with the trial mech mechanics and economy. It's very useful data to be had and needs to be done. But why not just disable Founder's mechs for a week or two and have *everyone* go through the trial phase. It would also give a better indication of the whole PUG vs. Premade griping that is going on the forums when everyone is relegated to the same trial mechs. Consider it a 'control group' if you're aware of how the research and development process goes.

Another case in point would be the proposed new double heat sink functionality with 0.14 heat per second as opposed to 0.20 heat per second. I would very much like to test this out as my initial calculations for my mechs and a even some of the PPC and Large Laser builds (Awesomes, K2 Catapult, Large Laser Atlas or Centurion) show that they either make heat dissipation worse so single heat sinks are better or that the gain is so marginal it isn't worth the C-Bill investment for upgrading, paying for extra sinks, and the associated repair costs. In either case, my conclusion is that the 0.14 heat sinks are not worth buying for *any* mech.

HOWEVER! I want to test this out.

But without resets to the game and to the amount of MC (and without injecting MC for the free testers) I really don't want to spend any actual money and C-Bills just to test something out. I would assume that most others (especially free players) feel the exact same way. Because of this you will probably get very biased and skewed data!

I really, really want to be able to test my math and hypothesis out(essentially a simulation) in-game (data for verification/validation), but have no incentive to do so. The current way the game runs actually discourages proper game testing.


The DHS example above is just one trade-study on the phase of the 'beta' we're in (it's not a true beta at all, you've essential launched that game by removing resets). Any ther new maps, additional features, and modifications to the mechanics, etc won't be properly tested with accurate and valid data to back up the research and balancing process unless wipes (and MC injections) still occur. I really hope that everyone at PGI takes this in stride and realizes that
essentially releasing the game only hurts the development process of this game. I would very much like to see MWO to survive, but in order to do that, we (as testers) need to actually be able to test the game and be able to give the developers at PGI useful (and unbiased) data and opinions.

We can't do this with how the game currently functions. I urge the developers at PGI to realize that the way they released to 'open beta' was a very, very big mistake. Make an announcement to admit this mistake; and let us actually get back to a productive testing environment.

As an example to my above DHS anecdote, here is what I've tested on my current mechs. It's a limited pool of chassis (but are some very common ones) and I would like to test out the PPC Catapult-K2 (rather than guass builds) as well as various PPC and Large Laser Awesome builds. But I have no reason or incentive to do so....in fact I am completely discouraged because in order to test I would have to spend a large amount of my personal money (which I already have done) or spend *a lot* of hours playing the game to earn the C-Bills in order to test the effect of 0.14 heat sinks vs. 0.20 heat sinks.

I will reiterate:

*This is not a good testing environment*

Please, please, please return the game to a proper and useful testing environment! It helps you and it helps us.

.
WIPE everything weekly... "BUT", the Founders get "NO" Founders Mech, and they must run "ONLY" uncutomizable TRIAL Mechs, and they get "NO" C-Billl boost, as this is not a good way to test the economy... Also Grouping will be disabled during the rest of BETA..
.
No, you wouldn't go for that, there would be buckets of tears being collected, from people with already overdeveloped sense of pilot skills, and self worth..
.
I don't see anyone giving up what they "HAVE" wihout an argument.....

Edited by Odins Fist, 03 November 2012 - 10:49 AM.


#5 nungunz

    Member

  • PipPipPipPipPipPipPip
  • Urban Commando
  • 612 posts

Posted 03 November 2012 - 10:49 AM

View PostWarrax the Chaos Warrior, on 03 November 2012 - 10:41 AM, said:

I agree that the "testing" we are doing right now isn't generating much in the way of useful data, but not for exactly (or only) the reasons that you list. The bad netcode has made so many weapons not work anywhere near like they are supposed to, and the lack of collisions has made all battles play out in a completely unrealistic way. Those two issues more than anything else have broken the game to the point that any data gathered is completely skewed.

As far as that relates to the economy being a part of why we aren't in a good testing environment right now, I think that these last couple of patches might qualify as enough of a "major catastrophe" to justify another wipe before release. That would be rough on free players, but some sort of an adjustment to the initial free player experience might make up for it.


Hands down I agree to this as well.

'Open Beta' is really a misnomer as at this point we really aren't testing anything....mostly because we can't reliably test anything with how the current game is running.

What we have is an unfinished game that has been launched.

PGI (if anyone is listening), someone really needs to get a hold of the reigns and bring the game back into a testing environment (isn't that what we're supposed to be doing?), not some rick-shod launch.

#6 Frosted

    Member

  • PipPipPipPipPipPipPip
  • The Seeker
  • 515 posts

Posted 03 November 2012 - 10:52 AM

Or they could just release 2 clients. A test client where you can test incoming changes/content and a "release client".

#7 nungunz

    Member

  • PipPipPipPipPipPipPip
  • Urban Commando
  • 612 posts

Posted 03 November 2012 - 10:54 AM

View PostOdins Fist, on 03 November 2012 - 10:48 AM, said:

.
WIPE everything weekly... "BUT", the Founders get "NO" Founders Mech, and they must run "ONLY" uncutomizable TRIAL Mechs, and they get "NO" C-Billl boost, as this is not a good way to test the economy... Also Grouping will be disabled during the rest of BETA..


I would totally be fine with this! This kind of testing needs to be done. Though, I would prefer the option of running against other Founders and be able to use the founders mech.

Grouping shouldn't be disabled, honestly, as this is how the game is going to run on actual 'release' (heh). It is worth a try though (for a week or two), just to get more data.

However, PGI needs to clearly state their testing goals in each iteration and make it clear what we're doing. There has been no direction so far.

Quote

No, you wouldn't go for that, there would be buckets of tears being collected, from people with already overdeveloped sense of pilot skills, and self worth..
.
I don't see anyone giving up what they "HAVE" wihout an argument.....


Hence the 1-2 week test blocks. And as for what people 'Have'. Yes, the Founders will ***** and moan, but the communication should be clear that we are here to TEST, not play the game. PGI unfortunately has created an environment where people feel entitled to their 'stuff'.

I doubt there will be a way to completely come back from this, but for the love of all that's holy some organization to the test process at all will help us a ton. And help the game become successful.

#8 Bogus

    Member

  • PipPipPipPipPipPip
  • Bridesmaid
  • 487 posts

Posted 03 November 2012 - 10:55 AM

Here's an idea, do what Eve-Online has done since they launched and have a separate test server for true testing. Periodic wipes but items are freely available. Also put in place rules that if you're on the test server you need to be testing, or at least not interfering with people who are.

Aside from letting people keep their shiny this would also create a place for true testing to happen, like seeing how hitboxes work and futzing around with lag. Even with wipes you just can't do that on the main servers.

#9 nungunz

    Member

  • PipPipPipPipPipPipPip
  • Urban Commando
  • 612 posts

Posted 03 November 2012 - 10:55 AM

View PostFrosted, on 03 November 2012 - 10:52 AM, said:

Or they could just release 2 clients. A test client where you can test incoming changes/content and a "release client".



View PostBogus, on 03 November 2012 - 10:55 AM, said:

Here's an idea, do what Eve-Online has done since they launched and have a separate test server for true testing. Periodic wipes but items are freely available. Also put in place rules that if you're on the test server you need to be testing, or at least not interfering with people who are.

Aside from letting people keep their shiny this would also create a place for true testing to happen, like seeing how hitboxes work and futzing around with lag. Even with wipes you just can't do that on the main servers.



Both of the above would be great! And/Or at least put users and rotations between the clients (1-3 week rotations) so one server isn't biased because people only want to jump on one particular server.

Edited by nungunz, 03 November 2012 - 10:56 AM.


#10 Fenix0742

    Member

  • PipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 265 posts
  • LocationUSA

Posted 03 November 2012 - 10:59 AM

You're not a beta tester though. You're a paying customer. The devs have made this abundantly clear with their focus on economy, dismissal of imbalance in weapons, and refusal to publish complete patch notes. So quit complaining and buy more premium time. That way you can afford switching between singles and dubs.

#11 William Radick

    Member

  • PipPipPip
  • Moderate Giver
  • Moderate Giver
  • 64 posts
  • LocationH-Town Texas

Posted 03 November 2012 - 11:01 AM

View PostBogus, on 03 November 2012 - 10:55 AM, said:

Here's an idea, do what Eve-Online has done since they launched and have a separate test server for true testing. Periodic wipes but items are freely available. Also put in place rules that if you're on the test server you need to be testing, or at least not interfering with people who are.

Aside from letting people keep their shiny this would also create a place for true testing to happen, like seeing how hitboxes work and futzing around with lag. Even with wipes you just can't do that on the main servers.



This.

#12 nungunz

    Member

  • PipPipPipPipPipPipPip
  • Urban Commando
  • 612 posts

Posted 03 November 2012 - 11:03 AM

Anyone mind if I link this thread in the suggestions forum? Feel free to post your thoughts there and keep bumping it.

I've sent a few emails back and forth with one of the GameMasters through the support email address and gotten favorable feedback.

#13 Stimbles

    Member

  • PipPipPipPipPip
  • Knight Errant
  • Knight Errant
  • 165 posts
  • LocationEquestria

Posted 03 November 2012 - 11:05 AM

I know I'm not willing to grind for and waste millions of C-Bills in a live environment to test double 1.4sinks, so I won't be using them at all from now on.

#14 Odins Fist

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 3,111 posts
  • LocationThe North

Posted 03 November 2012 - 11:08 AM

View Postnungunz, on 03 November 2012 - 10:54 AM, said:


I would totally be fine with this! This kind of testing needs to be done. Though, I would prefer the option of running against other Founders and be able to use the founders mech.

Grouping shouldn't be disabled, honestly, as this is how the game is going to run on actual 'release' (heh). It is worth a try though (for a week or two), just to get more data.

However, PGI needs to clearly state their testing goals in each iteration and make it clear what we're doing. There has been no direction so far.



Hence the 1-2 week test blocks. And as for what people 'Have'. Yes, the Founders will ***** and moan, but the communication should be clear that we are here to TEST, not play the game. PGI unfortunately has created an environment where people feel entitled to their 'stuff'.

I doubt there will be a way to completely come back from this, but for the love of all that's holy some organization to the test process at all will help us a ton. And help the game become successful.

.
SEE I said nobody would give up what they have (called it)... No Compromise, either the whole idea, or none of it...
The economy cannot be tested correctly with part of the community earning a C-Bill "BOOST", so that goes away...
No Grouping, so Newcomers can actually want to test for more than 1 hour, then leave MWO forever...
.
Founders must play Trial Mechs (no customization), just like the rest of the newbs for a month...
After one month (WEAPONS) testing goes into effect, and everything goes back to normal for Founders, and the Trial Mech users go back to being stomped by premades, and people with "SUPERIOR" customized Mechs, also grouping comes back, but no group bigger than 3 for a month....
.
Everyone with a Founders tag and Founders Mechs that are "customizable", and have a C-Bill Boost would not agree on this, even "YOU" didn't.... I called it....
.
"I don't see anyone giving up what they "HAVE" wihout an argument....."

Edited by Odins Fist, 03 November 2012 - 11:30 AM.


#15 King Hrothgar

    Member

  • PipPip
  • 46 posts

Posted 03 November 2012 - 11:23 AM

Indeed testing is dead. The whole point in a beta is so you can test stability, the economy and find ways to break the game. We can do none of these currently. I'm certainly not dumping my MC into ensuring that various mechs and fittings are working the way they should. And actual c-bill rewards for the matches are so terribly anemic that it takes days just to upgrade your existing mech, nevermind buy a new one. We simply cannot test the game properly under current conditions. About the only thing that has been verified is that every stock mech is completely broken as is (we knew that already) and the payouts for successful matches are at starvation levels for anyone running an owned heavy/assualt and that's with the premium bonus.

#16 nungunz

    Member

  • PipPipPipPipPipPipPip
  • Urban Commando
  • 612 posts

Posted 03 November 2012 - 11:32 AM

Quote

Everyone with a Founders tag and Founders Mechs that are "customizable", and have a C-Bill Boost would not agree on this, even "YOU" didn't.... I called it....


Did you read what I wrote, or just what you wanted to see? I said I agree with you and that we should get rid of Founders Mechs or C-Bill Bonuses during test.

BUT also have the option of only dropping against other Founders as well as customized mechs.

It's not a compromise, it's testing two things at once. Please thoroughly read a post before you go to town on it!

View PostOdins Fist, on 03 November 2012 - 11:08 AM, said:

.
SEE I said nobody would give up what they have (called it)... No Compromise, either the whole idea, or none of it...
The economy cannot be tested correctly with part of the community earning a C-Bill "BOOST", so that goes away...
No Grouping, so Newcomers can actually want to test for more than 1 hour, then leave MWO forever...
.
Founders must play Trial Mechs (no customization), just like the rest of the newbs for a month...
After one month (WEAPONS) testing goes into effect, and everything goes back to normal for Founders, and the Trial Mech users go back to being stomped by premades, and people with "SUPERIOR" customized Mechs, also groupoing comes back, but no group bigger than 3 for a month....
.
Everyone with a Founders tag and Founders Mechs that are "customizable", and have a C-Bill Boost would not agree on this, even "YOU" didn't.... I called it....
.
"I don't see anyone giving up what they "HAVE" wihout an argument....."


As I said before, this would be the optimal way to do things. Unfortunately the way PGI has implemented the system to this point means that entitlement has run rampant. Yes I will admit that.

Doing a complete 180 isn't possible because (as you stated) people won't accept it. Doing 100% what you suggest has a high chance of killing the progression of the game (just as much as the current system).

What needs to be focused is what can be done to salvage the system of testing as we have no easy way of doing a complete restart given that the Founders will react just as negatively as the free players.

Implementing things like VOIP in-game to help with the issue of being able to communicate will help (as you said), but it doesn't fix the testing environment/methodology.

Eliminating grouping will tick off everyone not just Founders. What if I want to play in a LAN or with 2-3 friends and group with them (whether they're founders or not). What if I want to group with a new player to help show them the ropes? Grouping will be central to the game in the long-run so it can't just be eliminated. Limited (which will happen in the next patch), but not eliminated.

Hence why I'm trying to show what's wrong with the current system and trying to figure out ways to salvage the system as a compromise (because that's the only option that won't segregate paying and free users into two separate communities).

View PostKing Hrothgar, on 03 November 2012 - 11:23 AM, said:

Indeed testing is dead. The whole point in a beta is so you can test stability, the economy and find ways to break the game. We can do none of these currently. I'm certainly not dumping my MC into ensuring that various mechs and fittings are working the way they should. And actual c-bill rewards for the matches are so terribly anemic that it takes days just to upgrade your existing mech, nevermind buy a new one. We simply cannot test the game properly under current conditions.


This, this right here!

Edited by nungunz, 03 November 2012 - 11:31 AM.


#17 Parnage Winters

    Member

  • PipPipPipPipPipPip
  • The Nimble
  • The Nimble
  • 414 posts

Posted 03 November 2012 - 01:01 PM

O.o

By several other posters own statements and your own you've all found several bugs, things you'd like to have improved and general thoughts and opinions on changes to weapon systems and mech parts. Which is essentially exactly what a Beta that's open is supposed to do.

What it seems you want(correct me if I've mis-read) is a sort of play around with everything style open beta which has it's uses but I don't feel you've given any good reason to do such besides essentially "I want it." You have plenty of things to report on and discuss the proof of such is by the numerous threads poping up on topics from DHS, to LRMS, to Gauss to trial mechs and interface.

I think you are being somewhat disingenuous to think that you have nothing to test and nothing to give feedback on. Feedback that yes they do listen to and do take interest in hearing what you think. You do need to understand however that the build you are playing is not the latest build of the game, you have other groups testing out the features you think are being untested admittedly not as numerous due to it being internal but it is happening.

#18 Warrax the Chaos Warrior

    Member

  • PipPipPipPipPipPipPip
  • Bad Company
  • Bad Company
  • 925 posts
  • LocationMyrror

Posted 03 November 2012 - 02:47 PM

View PostParnage, on 03 November 2012 - 01:01 PM, said:

O.o

By several other posters own statements and your own you've all found several bugs, things you'd like to have improved and general thoughts and opinions on changes to weapon systems and mech parts. Which is essentially exactly what a Beta that's open is supposed to do.

What it seems you want(correct me if I've mis-read) is a sort of play around with everything style open beta which has it's uses but I don't feel you've given any good reason to do such besides essentially "I want it." You have plenty of things to report on and discuss the proof of such is by the numerous threads poping up on topics from DHS, to LRMS, to Gauss to trial mechs and interface.

I think you are being somewhat disingenuous to think that you have nothing to test and nothing to give feedback on. Feedback that yes they do listen to and do take interest in hearing what you think. You do need to understand however that the build you are playing is not the latest build of the game, you have other groups testing out the features you think are being untested admittedly not as numerous due to it being internal but it is happening.


The only bug we can really report right now is that hitboxes and netcode are screeeewwwwweeedd uuuppp; that makes any discussion concerning the balance of DHS and energy weapons completely useless. The fact that all of our MC, c-bill and cash expenditures are 'for realsies' now means that very few people are willing to take the risk to test anything. I think we are going to need another reset at or before release.

#19 FelixBlucher

    Member

  • PipPipPipPipPip
  • 159 posts

Posted 03 November 2012 - 05:00 PM

i would suggest simply removing the cost of materials for now, until we get all, or most of the items into the game. Then we can properly test them without feeling that we are losing valuable money/time. I know the thought corsses my mind when I consider purchasing the different items. thinking about if I really want to spend my "hard earned" Cbills on something that may not work right.
In such a situation, we are not really testing the various weapons and systems. This is really how we should have been testing everything. Then we would be able to see the trends of who gets what weapon the most. if one particular weapon is used more than any other ones, then it is unbalanced. Of course this can be tested to some degree now, but not with the ease of mind that players get from knowing that they can experiment freely. without such an environment, I fear that true balance will not come easily.

If we hadn't had to re-earn our Cbills during Closed Beta, I am sure that players would not have gotten as tired of the wipes as they did.

#20 Draco Argentum

    Member

  • PipPipPipPipPipPipPipPip
  • 1,222 posts

Posted 03 November 2012 - 06:15 PM

This stopped being a serious beta back in closed beta when they put the live style economy in and stopped giving welfare warriors free MC. Now that even paying customers don't get resets on real money purchases this is a launched game in all but name.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users