Garth Erlam, on 14 May 2012 - 12:22 PM, said:
You think we're trying not to break the game? You've never seen a testing team at work have you?
The first thing the testers did was make that mach 1 Jenner, then came the Small Laser Armada, followed by the SRM Renaissance. We have competitions for the best K:D, the best W:L ratio. Our best player uses a Hunchback -
s/he used to use a Catapult.
We do 2v1's, 10v10's, 3v4's, everything. We stack teams on purpose, we try to see if our best player can take on two, three times his/her number in opponents - with each 'Mech.
We give kudos and (not real) 'prizes' for discovering the best cheats, the best abuses.
And I dunno about your friends, but mine like ruining my stuff more than telling me how pretty it is
I do have to jump in on Outlaw's defense here. I don't think that's what he meant. I realize you guys have full time testers (and have done dev testing on some AAA games before) and I positively realize you guys are trying to break it and find the most powered stuff possible.
The concern he was really trying to express (which I think both Outlaw and I both agree will be helped vastly by a BV/Tonnage check) is that of team work, in particular experienced MechWarrior teams. Because it sounds like there is a ton of testing to see what is viable in duals and small scale controlled fights, but really, those are great for fun little duals and absolutely not a gauge of how battles in past games have played out.
To summarize what he's trying to say, we routinely would take in MW4 would take on a Daishi with a Wolfhound and win; a lot of the time this was our test of new pilots, in fact. Our co-founder's record was untouched at winning these fights, and mine was close. That's 35 vs 100 tons; I positively believe that any 'mech can easily beat any 'mech in a one on one fight in MWO, but the same
has been true in the past. The problem starts when you, say, take 4 Daishi vs 4 Wolfhounds: suddenly you'd have a situation where not one Daishi would die. 8 vs 8 and they wouldn't even get damaged seriously.
My point is that focus fire is probably the biggest, most important thing in all of MechWarrior, and lights/mediums rely on speed and being able to out maneuver heavier 'mechs to fall into their blind spots. They won't have that chance if there are no real blind spots (due to units being clustered up), and they pack the firepower that, combined, can decimate the lighter 'mech almost instantly before it can get close enough to expose weaknesses.
Really that's what I also believe will impact the friends & family beta more than lone pilots doing exceptional things, and also will skew the tonnage system dramatically. Honestly I think it would be a huge, huge benefit if you could do something similar to what Valve has done with it's recent games and invite competitive teams and players into an NDA-backed pre-closed beta, in order to see how organized groups with lots of past experience perform. Really any of the notable units from NBT that can be found in these forums would be a great help.
Again, I think the BV/Tonnage match making thing will help tons to preventing unfair fights; I'm more explaining what I think Outlaw was meaning to say, and that I also really believe it would be really helpful to get any of the more experienced preexisting teams on the MWO forums into some matches shortly before the actual semi-public beta starts (so they can be matched exclusively against each other, without resource restrictions) to see how they react. I think it's very likely that even if you have individual pilots that exceed what these teams bring to the table in testing, you'll see a
huge shift in tactics that might radically impact game balance overall. It's definitely worth considering.
Garth Erlam, on 14 May 2012 - 12:22 PM, said:
You think we're trying not to break the game? You've never seen a testing team at work have you?
As someone who has seen many top-tier, dev-housed test teams at work including on some really major MP games there is some truth to what he says; mostly for the reasons explained.
Professional testers often have a few weaknesses, depending on how the crew is setup. I'll skip the common issues with bug testing over play balancing, because that's the major issue here; and the biggest problem is that testers often will not often step up and take command over a situation, because in a professional environment it is taboo'ed and often outright discouraged by test leads. Often by the time testing is late enough when real ad-hoc play balancing can be done (as the majority is bug hunting prior), it's too late to seriously correct major balance problems that would go on to be exploited by the community afterwords.
The problem with friends & family betas is often similar (not wanting to boss friends or family of other co-workers around), with the added bonus that they may not be familiar or experienced in the genre/game series at work. These kinds of testers (as well as Kleenex testers, who are very useful for different reasons) can be extremely valuable for different kinds of feedback and finding all kinds of game balance issues, but again, often miss the mark with team situations.
That's why the trend of bringing in pro players and teams at various companies has started, and in the past, giving alphas of games to clans of professional map monkeys also was undertaken outside of regular testing. All kinds of things can be found when you have a group mentality locking in on "play to win at all costs" versus individuals or leads/devs can often see through virtue of being "in the box."
Anyway, this is pretty wordy and it's possible a lot of this stuff has already been addressed. I suspect that PGI is doing it's testing internally exclusively, which is
very much preferable to involving outside tester groups due to the highly incestuous way that the industry promotes the people in charge of these projects (a rant so long it could literally fill a book). I do hope they are considering the idea of using veteran teams to help do final play balancing before going live however, as I'm convinced the results could be really surprising to everyone involved through the fault of no one.
Edited by Victor Morson, 15 May 2012 - 04:10 AM.