14 replies to this topic

[Roland]

So I've noticed a ton of folks saying things like, "Why should it take 4 weeks to make new changes? Just make the change, and let us try it!"

I know the answer, and it's that going into the "live" game with a bunch of untested stuff can have a huge negative impact on the overall impression that folks get. Certainly, with things like moving to DX11, or changes to various aspects of netcode, putting untested stuff out in the wild carries a lot of risk. We've seen past patches where lots of folks in the community were hit with bugs that left the game essentially unplayable for them. Thus, I think PGI wants to be cautious with big changes.

However, there are many of us who are willing to play guinnea pigs, and won't just freak the hell out over bugs, even huge ones.

As such, I think that it may be useful to implement some sort of "test server", similar to what CCP runs in EvE. Make a parallel server, perhaps even separated from the "real" accounts, where the devs can throw in new fixes. There would be no guarantees of stability at all. A certain fix might totally take down the test servers for a while until rolled back.

But at the same time, this type of solution might allow PGI to leverage the section of the player base who isn't that worried about running into bugs. And it would allow the "real" game to develop at a fairly slow, stable pace, while getting even more testing of new code, thus allowing it to become stablized even more quickly.

Certainly, implementation of such a set of test servers is not a trivial undertaking, but I think that perhaps it could help speed up testing of new features, while also protecting the main game from accidental introduction of new bugs.

[Bhael Fire]

Yeah...I joined up during closed beta for a reason; I WANT TO HELP MAKE THIS GAME AWESOME.

They should definitely consider this as an option for a few select players.

[Dirk Le Daring]

Great idea. PGI, you can count me in if you ever decide to at the least, try it out. EDIT: Realistically, I do not think I have the skillset, and would be wasting time and resources. Damn strikethrough, cannot get rid of it now.

Edited by Dirk Le Daring, 23 January 2013 - 04:49 PM.

[Roland]

Bhael Fire, on 22 January 2013 - 08:48 PM, said:

Yeah...I joined up during closed beta for a reason; I WANT TO HELP MAKE THIS GAME AWESOME.

They should definitely consider this as an option for a few select players.

No, I am not suggesting some kind of "cool kids only" club. You misunderstand me.

I'm suggesting that they just set up separate test servers like in EvE.. anyone can join them. You jump through some hoops to configure the client to connect to the test server instead of the live server, and then you are playing in test land.

Thus, you're just giving the option to anyone who wants to risk playing a TOTALLY broken, and unbalanced mess. Because, frankly, that's what it'll be at many points. That's the point. It's a test server, where you could have all kinds of crazy bugs and exploits... You will no doubt see extremely unbalanced weapons, or game crashing bugs,etc.

But by having test servers that folks can voluntarily connect to, you give the developers freedom to put much more lightly tested code out into the wild for testing by actual users, while protecting them from folks raging with posts like, "OMG, I WANT A REFUND, THIS CRAP IS BROKEN"

[Roland]

Bump for day two, just cause I made this pretty late last night. I think a good number of folks would support this type of testing environment.

[Fut]

Roland, on 22 January 2013 - 08:44 PM, said:

However, there are many of us who are willing to play guinnea pigs, and won't just freak the hell out over bugs, even huge ones.

Honestly, I thought this was the entire point of Beta. PGI has thousands of people who will not only work for free (finding bugs, balancing issues...etc) but some of them will actually pay them while doing it (MC Purchases..etc)!

The development of this game would go so much faster if they actually used us as beta TESTERS, as opposed to players who happen to be enjoying the game while it's still being made.

As it is right now there's some major aspects of the game that aren't slated to come out for about a year. An entire year! That's crazy. All because the Devs would rather test locally for months.

Roland, on 22 January 2013 - 08:53 PM, said:

I'm suggesting that they just set up separate test servers like in EvE.. anyone can join them. You jump through some hoops to configure the client to connect to the test server instead of the live server, and then you are playing in test land.

Thus, you're just giving the option to anyone who wants to risk playing a TOTALLY broken, and unbalanced mess. Because, frankly, that's what it'll be at many points. That's the point. It's a test server, where you could have all kinds of crazy bugs and exploits... You will no doubt see extremely unbalanced weapons, or game crashing bugs,etc.

But by having test servers that folks can voluntarily connect to, you give the developers freedom to put much more lightly tested code out into the wild for testing by actual users, while protecting them from folks raging with posts like, "OMG, I WANT A REFUND, THIS CRAP IS BROKEN"

The idea for an actual TEST SERVER is amazing though. It wouldn't hurt their playerbase/income at all, and it'd allow them to use testers for testing.. Win-Win.

Win
Win
Win.

[HRR Insanity]

They already posted that they have these servers... they're just internal only.

[Roland]

Yes, obviously they have test servers.
This suggestion focuses specifically on making an open alpha test server, akin to EvE's.

[glycerin]

I think this is a great idea. Make it moderated by a PGI employee with about 30-35 community people to playtest changes. This could help quality as well.

Another idea:
1. Every week PGI sets a couple times to do tests. (example: Wednesday and Thursday at 3 PST)
2. The people that can make it at that time slot from the community put their name in a hat.
3. A preset number of people are selected (like a lottery) with an appropriate number of backup players in reserve for the test time. (Make a prerequisite for who may apply, maybe based on match playtime to avoid multiple entries/signups)
4. Play testers are briefed about which issues they are testing and required to fill out reports regarding their playtime.
5. Play testers that miss their time or don't report correctly are forever stricken from future play testing.

Sounds like a lot of benefits for a little management time.

[sycocys]

I'm really for either idea - totally down for them just unleashing hell on the main server - but I can understand why they wouldn't when you have that portion of the community that is in disbelief that this is a beta game running on a new set of production standards, it'd be chaos and the rage/QQ threads would be out of control.

If there were a way that I could hop onto an alternate test server to help them test future implementations I'd totally do it, not sure how they'd do it with so much of the game being client-side, but heck i'd be just as down to test out server-side only additions.

[Roland]

sycocys, on 23 January 2013 - 08:35 AM, said:

I'm really for either idea - totally down for them just unleashing hell on the main server - but I can understand why they wouldn't when you have that portion of the community that is in disbelief that this is a beta game running on a new set of production standards, it'd be chaos and the rage/QQ threads would be out of control.

If there were a way that I could hop onto an alternate test server to help them test future implementations I'd totally do it, not sure how they'd do it with so much of the game being client-side, but heck i'd be just as down to test out server-side only additions.

The main reason why they can't just turn new junk loose on the live servers, is that the game is essentially "released" for all intents and purposes. That is, people are paying REAL MONEY for crap in this game. As a result, people expect some degree of stability. PGI, rightfully, does not want to just stick in something like DX11, when it includes the risk of causing many of their paying customers to be entirely unable to play the game at all for a week at a time.

But having a separate test server, accessible to all, allows them to gain the benefit of having a huge set of testers, while not having to deal with the fallout that could come from making players unable to play a game they are paying for.

And again, this is not something which should only be accessible to a select few. The whole point of this is to make it widely accessible to ANYONE who is willing to risk playing with alpha-quality software.

For certain types of changes, like moving to DX11, this would offer an infinitely faster method of identifying game crashing bugs, because it's going to end up hinging a lot of specific hardware configurations. Those types of bugs just aren't going to be fully driven out by small internal test teams.

I think having open alpha servers, side by side with the real live servers, will enable PGI to dramatically speed up their development time by more quickly identifying game breaking bugs, which will enable them to more quickly get new features out into the stable live servers for the overall community to enjoy.

[Bhael Fire]

Roland, on 22 January 2013 - 08:53 PM, said:

No, I am not suggesting some kind of "cool kids only" club. You misunderstand me.

Not really a "cool kids club"...but definitely more tightly controlled for quality. I was thinking more along the lines of what glycerin suggests here:

glycerin, on 23 January 2013 - 08:34 AM, said:

1. Every week PGI sets a couple times to do tests. (example: Wednesday and Thursday at 3 PST)
2. The people that can make it at that time slot from the community put their name in a hat.
3. A preset number of people are selected (like a lottery) with an appropriate number of backup players in reserve for the test time. (Make a prerequisite for who may apply, maybe based on match playtime to avoid multiple entries/signups)
4. Play testers are briefed about which issues they are testing and required to fill out reports regarding their playtime.
5. Play testers that miss their time or don't report correctly are forever stricken from future play testing.

[Karl Split]

I'd happily sign another NDA and help out in alpha servers, given the endless crying by the forum trolls about everything i can see why PGI wont release anything to us currently unless they are pretty sure it will float.

If we were on the titanic and PGI started releasing lifeboats your average MWO forum poster would complain about the color and refuse to get in ffs

[Roland]

Honestly, I'm really proposing a much more open test environment. As long as you clearly state that the test servers are just that... a test environment where nothing is guaranteed, and no aspect of the game seen in the test servers is guaranteed to make it into the live environment, then that's really all you need to do.

Trying to more tightly constrain the user base of the test servers eliminates some of the purpose I'm trying to work towards here... Because the point is to expose a much wider base of users to those new features.

As Insanity pointed out...they already have test servers that are used by a small population... those are the internal PGI test servers.

The whole point of this idea is that you open up that test environment to a much broader population, and thus more quickly identify bugs that are related to specific users' system configurations, and more quickly identify balance issues that really only pop up when things are turned loose into the general population where people are more driven by winning than they are by just testing stuff in a sterile manner.

[glycerin]

more thoughts about this?