41 replies to this topic

[Chazer]

Shumabot, on 09 April 2013 - 07:25 AM, said:

Except they don't then "implement a fix or tweak". Thread reopened.

You are right, they didn't make any tweak to missle damage when it was found that it was doing excessive damage.

[3rdworld]

Chazer, on 09 April 2013 - 07:56 AM, said:

You are right, they didn't make any tweak to missle damage when it was found that it was doing excessive damage.

A bug, that caused missiles to do too much damage is considered a balancing tweak?

Oh and they nerfed them to the ground. That isn't balancing.

[jeffsw6]

3rdworld, on 09 April 2013 - 08:05 AM, said:

Oh and they nerfed them to the ground. That isn't balancing.

This is true. I just loaded up an LRM boat, legged an enemy Commando without ECM, and fired three salvos of 3xLRM15 at him from 190m range. They didn't do 0 damage to him, but he did not die. Teammates finally cut him down with lasers.

It is ridiculous that a legged, already-wounded, and basically stationary Commando can absorb 135 LRMs without getting killed.

[Reptilizer]

Chazer, on 09 April 2013 - 07:56 AM, said:

You are right, they didn't make any tweak to missle damage when it was found that it was doing excessive damage.

Do not know if that example is really backing your point:

As far as i remember the problem behind LRMageddon was found and analysed by a single dedicated and informed player. Then the devs "looked into it" and nerfed the whole thing into the ground with changing three parameters of the weapon at once.
Then they declared it to be a temporary hotfix and asked the player base for the "feel of that". Lots of feedback was given. The "temporary hotfix" is still in place. Nobody in their right mind uses LRMs.
Really not sure if you wanted THAT procedure as an example of how well balancing mechanisms are implemented into this game...

Edited by Reptilizer, 09 April 2013 - 08:31 AM.

[Syllogy]

Reptilizer, on 09 April 2013 - 05:55 AM, said:

Disclaimer:
This thread has not been designed to instigate any forum fights. This thread has the sole purpose to discuss the viability of changing ingame parameters according to forum trends as stated in the ask the devs #35:

"M0rpHeu5: Do you check the suggestions and the game balance topics?
A: Regularly. We tend to look for trends, rather than specific suggestions. When we see many threads about something being too powerful, we examine the root causes and implement a fix or tweak."

Bad way to single out stuff that needs fixing is bad for several obvious reasons including forum trolls. I would much prefer a balanced approach BASED on in game data. Discuss

To all volunteer moderators:
So please, as long as people are interested in discussing this somewhat unorthodox way of balancing in-game features, you might consider to leave this thread open since it does not violate any forum rules.
Or am i missing something here???

No, why would they? They just collect telemetry from dozens of data-points just to show players are uberl33t they are.

....really? The Devs take into account 2 outputs of data: Telemetry collected from in-game data, and forum responses to the issue.

Telemetry cannot account for live-experience, but it can account for actual figures (how many are used, what damage is actually done, what mechs are more popular, and just about every other imaginable factor)

Forum Responses cannot account for actual figures, but can reflect live-experience. In short, the more one-sided the playerbase sees a topic, the more out-of-balance that mechanic or weapon is.

Edited by Syllogy, 09 April 2013 - 08:36 AM.

[tenderloving]

jeffsw6, on 09 April 2013 - 07:02 AM, said:

Forum feedback isn't useless, but it shouldn't be taken as truth, either. There was nothing but LRM boat QQing for weeks even before the splash damage issue. During that time, other weapons were broken; and the LRM QQ drowned it all out. If something is over-powered, it will cause QQ. Under-powered? Much less.

Here is the root problem: there are statistics now that are even visible to PLAYERS, and PGI is completely ignoring that data. They have a big picture view of that data across all players, and can probably produce data on specific mechs, maps, and weapons over a large number of players and drops. They're not doing this.

They can also tell how many hours are played with various weapons equipped. That is a good indicator of whether or not a weapon is useful. If everyone is using Medium Lasers to some extent, they probably don't suck. Almost no-one using MG, Flamer, Small Pulse Laser -- an indicator that these weapons are not very useful. SPL is a filler and there are many other energy-weapon choices with similar ton/slot costs; otherwise there would be massive out-cry about it.

They can also collect data on what weapons produce the most killing blows. Right now, that would be the high point-damage weapons: AC20, Gauss Rifle, PPC, ERPPC. This is not necessarily an indication that the weapons themselves are over-powered, but that concentrated point damage deserves some thought. I think they are problematic, but many players do not agree. It's a complex issue that is hard to understand without a combination of data, player feedback (QQ), and if they decide something should be changed, TESTING.

Currently they are not doing any of these things well. This is a bad sign.

It is also plainly obvious that they are either not listening to the community staff folks that they hire to aggregate player feedback and communicate it to them, or those staff are not doing a good job. I tend to suspect the problem is on the developer end at this point, since some of these issues are obvious with simple arithmetic or trivial queries against the database of player stats.

Dang this was an amazing post. You summarized my exact feelings about what has been going on. If it would fit in a signature I would make it my sig.

[IceSerpent]

Reptilizer, on 09 April 2013 - 05:55 AM, said:

"M0rpHeu5: Do you check the suggestions and the game balance topics?
A: Regularly. We tend to look for trends, rather than specific suggestions. When we see many threads about something being too powerful, we examine the root causes and implement a fix or tweak."

Bad way to single out stuff that needs fixing is bad for several obvious reasons including forum trolls. I would much prefer a balanced approach BASED on in game data. Discuss

While I certainly agree that using forum trends is a bad way to balance things, in this particular game balancing via satistical analysis of in-game data is not a very good idea either.
It's done this way in other games (mostly MMOs) because a toon can have wide variety of skills (i.e. CC, healing, reactive skills, buffs, etc,), and creating a good mathematical model for that kind of variety is a non-trivial task - i.e. how would you compare a class that has a stun and a heal to a class that has a buff and an additional damaging skill?
When you have mostly "damage skills" (the only thing that even comes close to CC in an MW game is flamer and other stuff like heals doesn't exist), you can and should balance it in excel (or on a napkin). This gives an additional benefit of easily detecting weapon damage bugs - when your model estimates that you should kill an Atlas in 10 shots, and in reality it only takes 7 shots, you know that something is not quite kosher.
Unfortunately, PGI decided to go for "let's change this number and see what happens" approach for some weird reason.

[Deathlike]

tenderloving, on 09 April 2013 - 08:34 AM, said:

Dang this was an amazing post. You summarized my exact feelings about what has been going on. If it would fit in a signature I would make it my sig.

You could always link to the specific post, by copy+pasting the link from the forum post # (there's a link embedded in that).

Edited by Deathlike, 09 April 2013 - 08:46 AM.

[ElLocoMarko]

If I didn't have a job, I'd be selling myself to PGI as a statistics analyst (which I am sooo not qualified for).

With just a few post-patch data snapshots would quickly reveal things that QC missed like LRMpocalypse and over the top SRM damage. Look for chassis with very high or low kill/death ratios. Then these things could be fixed quickly before anyone gets too entrenched. I'm not saying we automatically bring everything to a middle point. I just want them to be aware of the outliers much earlier.

Just collect data and find the hot and cold spots. See what weapons people are running and which ones are not.
What weapons are doing their declared damage, which ones are over, which ones are less. Is damage dealt = damage received (before ammo explosions) Hopefully the "headless" testing that was described recently is being monitored in this fashion.

[Deathlike]

The reality is that I think there's an odd... personality when it comes to PGI and balance.

1) When there's math to be involved.. we talk about "usage" and "application"... the key important aspects about how a weapon is used in real time play. So when you see the current rage on the UPness or OPness of a weapon, it's because it is blatantly obvious that they haven't used it enough to get the proper grasp.. or context on how effective a weapon is based on the numbers.

2) A lot regarding min-maxing builds is about the #s... but yes you still have to take them into a match to see how it feels. We already have numbers on flamers and how bad it was that it causes the user to heat up more than the target.

If we take a proper context of say a player's gameplay or mech selection, you can get a decent idea of how they are in live gameplay. Think baseball when it comes to the metrics. If you track a decent ELO player that uses a different mech... don't be surprised a bit if their ELO tanks.. in order to adjust to the grinding they are doing in a new mech. The current ELO system only considers the weight class and not a particular chassis.. which is not that useful. If I grinded with a 3L (with it mastered), I'd probably do much better than a Jenner-D that I have yet to grind. However, ELO is factored in the same way... except that the game uses my 3L ELO and says "hey, you can do just as well in a Jenner-D", which I don't believe will be the case for a while anyways.

You can also figure out trends.. assuming PGI has tracked me from the start, I have used flamers in the past. When stats have gone live, these stats are not there... and for good reason. If you were to track an average ELO player vs a high ELO player, you can see trends of weaponry in use.. whether it is shelving a weapon.. or trying new builds. There is the oft chance that I may go back to using the flamer if/when it gets a buff. When I'm trying it out because I've got nothing better to do.. it is literally an anomaly for the stats... usually those get ignored in statistics. If the flamer gets popular, expect more people to be using it... it's really that simple.

Heck, if you were to track LRMaggedon, you would use LRM usage on a sharp rise.. and the stats probably bear this out. It doesn't take a genius to figure out that unbalanced weapons will have a corresponding usage change.

Anyways... I don't think PGI does enough "looking into" when it comes to weapon/item trends. You have to look into the details of the usage... with the proper understanding and context. That is why they have yet to see why some of the hubbub for certain weapons is all about.

[Shumabot]

Chazer, on 09 April 2013 - 07:56 AM, said:

You are right, they didn't make any tweak to missle damage when it was found that it was doing excessive damage.

Oh, yeah, clearly after waiting through seven months of having missiles do up to 12 times the damage they were supposed to do they dramatically nerfed them, temporarily, and admitted that they did no internal metric testing for the nerf.

If that's the kind of "balance tweak" this game is going to have you better order a tombstone for this train wreck.

Edited by Shumabot, 09 April 2013 - 09:22 AM.

[Shumabot]

Deathlike, on 09 April 2013 - 09:12 AM, said:

The reality is that I think there's an odd... personality when it comes to PGI and balance.

1) When there's math to be involved.. we talk about "usage" and "application"... the key important aspects about how a weapon is used in real time play. So when you see the current rage on the UPness or OPness of a weapon, it's because it is blatantly obvious that they haven't used it enough to get the proper grasp.. or context on how effective a weapon is based on the numbers.

2) A lot regarding min-maxing builds is about the #s... but yes you still have to take them into a match to see how it feels. We already have numbers on flamers and how bad it was that it causes the user to heat up more than the target.

If we take a proper context of say a player's gameplay or mech selection, you can get a decent idea of how they are in live gameplay. Think baseball when it comes to the metrics. If you track a decent ELO player that uses a different mech... don't be surprised a bit if their ELO tanks.. in order to adjust to the grinding they are doing in a new mech. The current ELO system only considers the weight class and not a particular chassis.. which is not that useful. If I grinded with a 3L (with it mastered), I'd probably do much better than a Jenner-D that I have yet to grind. However, ELO is factored in the same way... except that the game uses my 3L ELO and says "hey, you can do just as well in a Jenner-D", which I don't believe will be the case for a while anyways.

You can also figure out trends.. assuming PGI has tracked me from the start, I have used flamers in the past. When stats have gone live, these stats are not there... and for good reason. If you were to track an average ELO player vs a high ELO player, you can see trends of weaponry in use.. whether it is shelving a weapon.. or trying new builds. There is the oft chance that I may go back to using the flamer if/when it gets a buff. When I'm trying it out because I've got nothing better to do.. it is literally an anomaly for the stats... usually those get ignored in statistics. If the flamer gets popular, expect more people to be using it... it's really that simple.

Heck, if you were to track LRMaggedon, you would use LRM usage on a sharp rise.. and the stats probably bear this out. It doesn't take a genius to figure out that unbalanced weapons will have a corresponding usage change.

Anyways... I don't think PGI does enough "looking into" when it comes to weapon/item trends. You have to look into the details of the usage... with the proper understanding and context. That is why they have yet to see why some of the hubbub for certain weapons is all about.

The flamer doesn't even heat up enemy mechs. The large pulse laser has the same range as a medium but does less damage per ton and generates more heat. Streaks were free killing lights for a year solid. Small pulse lasers are objectively inferior to medium lasers on everything short of a 6spl jenner, and that's not even a good mech. The AC2 generates more heat per damage per ton than anything else in the game and does less sustained damage before shutdown than any other autocanon in the game.

PGI isn't "not noticing trends" PGI is "not addressing balance issues because it's hard". Every time they address a balance issue it either takes them literally months to change a spread sheet value or they break something when they make the change. Their code base must be some sort of dark fortress of thorn bushes and land mines for them to be this intransigent about balance issues. It's clear that PGI doesn't actually believe balance is a priority.

And honestly? I don't blame them. This community can't recognize balance issues for what they really are. It's got Stockholm syndrome to the battletech franchise and they're free to bleed it dry forever because of that.

Edited by Shumabot, 09 April 2013 - 09:28 AM.

[Zyllos]

Shumabot, on 09 April 2013 - 09:26 AM, said:

The flamer doesn't even heat up enemy mechs. The large pulse laser has the same range as a medium but does less damage per ton and generates more heat. Streaks were free killing lights for a year solid. Small pulse lasers are objectively inferior to medium lasers on everything short of a 6spl jenner, and that's not even a good mech. The AC2 generates more heat per damage per ton than anything else in the game and does less sustained damage before shutdown than any other autocanon in the game.

PGI isn't "not noticing trends" PGI is "not addressing balance issues because it's hard". Every time they address a balance issue it either takes them literally months to change a spread sheet value or they break something when they make the change. Their code base must be some sort of dark fortress of thorn bushes and land mines for them to be this intransigent about balance issues. It's clear that PGI doesn't actually believe balance is a priority.

And honestly? I don't blame them. This community can't recognize balance issues for what they really are. It's got Stockholm syndrome to the battletech franchise and they're free to bleed it dry forever because of that.

Flamers numbers just need changes.

The Large Pulse Laser (as does all pulse lasers) need their "On" duration reduced to 0.5, so that they fire 100% quicker than regular versions. The Large Pulse Laser also dealt less damage per ton than a Medium Laser in the TT, but in the TT, you gained a +1 to hit. The best thing to bring that advantage over is to greatly reduce the "On" duration to 0.5s (instead of 1.0s for a normal laser). Also, the Large Pulse Laser will need it's heat increased again with this change, I would suggest 8.0.

SSRMs needs it's locking mechanics seperated from LRMs so that they can completely lock differently (and not have Artemis IV effect it, along with other equipment that helps LRMs but not SSRMs). Then balance SSRMs based on different mechanics, like losing the SSRM locking mechanic when it is fired.

The Small Pulse Laser also falls into this boat but for a different reason. It needs it's damage increased by 0.5 (dealing 3.5 damage).

The AC/2 needs it's heat decreased to 0.3 heat per shot, so that it is equal to the AC/5 in heat.

I personally think a lot of the above issues is moot when you can converge all your weapons onto a single point. Fix that issue first, then balance everything else.

And the reason why PGI is taking a LONG time to balance is because they want to test, retest, and reiterate the test. But the problem is that with games, that puts too much strain on your community. No other software program has to worry about if parts of the program makes the user feel underpowered or overpowered. It either works or it doesn't, so that means you take your time to make sure it works.

In the gaming world, if the game makes the user feel underpowered or overpowered, it's most likely the fault of the underlying mechanics and values, which, most of the time, is not the fault of the program. Sometimes mechanics and values get translated incorrectly by the program, but most of the time, it's because the mechanics or values are incorrect. And those should not be taking 2 to 4 weeks to test.

And this is why the community is getting fed up with balance. There is not enough balancing changes happening. Make a single change to a weapon or mechanic, then let it run for 1 week. If the community doesn't like it, then go back to change it again for the next week. And do not be afraid of making several weapon/mechanic changes together, especially if they are independent of each other.

A good example of this is fixing pulse lasers and then fixing SSRMs. Both of those should have been changed ages ago but we are getting slow, iterated changes which change balance slowly.

This is also why it would be good to have a Public Test server. Yes, we are in beta, but PGI seems timid on making many changes quickly. So, add a Public Test server, that can change daily for live players to test. It is just impossible to test balancing issues in a closed environment with a specific set of minds.

[RoboPatton]

Game is still technically in beta. It's changed quite a bit from the original closed beta. I think they are focusing on content right now. I worked in the video game business for almost 6 years, I know the cycles, I know how they handle bugs and "balance." Much of what we are collectivley QQ'ing over will likely be gone or improved by the official launch. If not, I close my wallet and go elsewhere. I don't suspect I will have to.

So, yeah all those "3L are op" posts have probably already led them to the conclusion they will eventually implement (when they are ready to start putting in fixes, instead of content).

Edited by RoboPatton, 09 April 2013 - 10:40 AM.

[blinkin]

it's rather amazing. i don't agree with the OP, but he did make an effort to (at least at the very start) keep this constructive. despite this effort the thread has immediately spiralled of into flame wars and rants about how the develeopers haven't fixed "what i want fixed" so obviously are ignoring everyone everywhere.

i have seen plenty of fixes.

• they limited max engine sizes on mechs so i couldn't have a 99kph catapult (yes that was very possible for a while in closed beta).
  
• they nerfed jump jets so that a single jump jet couldn't be used to fly up a thousand feet.
  
• then they buffed jump jets so that they could actually jump (it was nice to have JJ be useful again)
  
• they made it so lasers could more effectively hurt light mechs and defeat lag shield.
  
• they completely reworked both of the vision modes (i thought they were just fine).
  
• they reduced the heat on PPC/ERPPC because people complained they were useless (again i disagree here)
  
• they made countless changes to streaks (even if they did end up mostly back where they started)
  
• they increased ammo counts on ballistic weapons
  
• they made gauss rifles more fragile to reduce all of the sniper QQ.
  
• they shrank the atlas cockpit. (it used to be the whole head, not just the eye)
  
• they buffed the flamer and machine gun a few times (even if none of the changes helped enough)
  
• countless reworks on mech hit boxes
  
• light mechs no longer take leg damage just from running
  
• mechs that wander into the caldera on caustic don't immediately shut down and explode
  
• caustic and river city both have had several major level reworks to balance the sides more
  
• heat sinks have had several reworks to balance them
  
• countless changes to the net code in an effort to reduce lag shield (this latest iteration is not good in my opinion)
  
• and i don't have the patience to list any more

there have been many changes that have been lifted straight out of the suggestions forums. just because they are not catering to your specific whining does not mean that everything being said is being ignored.

and they don't exactly get rewarded for changing things (whether it be fixes, rebalances, or anything). shortly after any major change the forums get spammed with "OMG developers are we todd ed everyone will quit playing for evar!!1!!!1111!!!!!" by hoards who have absolutely no understanding of how game development works and are angry because the game isn't EXACTLY every thing that they want.

honestly if i were them i would have given up on any sort of interaction with the forums long ago.

[Deathlike]

blinkin, on 09 April 2013 - 12:01 PM, said:

honestly if i were them i would have given up on any sort of interaction with the forums long ago.

Most of the time I think they are like the forum posters... where TL;DR is the norm.

The norm IMO is that I'm starting to think is that they don't have enough of a testing base (I mean, their in-house testing) even time or effort into balancing. It's probably easier saying "see if this new feature can be broken" and test it, instead of trying to tweak "existing" and "working" solutions to get a more concrete feel of the matter.

They did buff the AC20 health after all... so I guess you could give them "some" credit.

[Zyllos]

blinkin, you are right. PGI has done a lot of balancing.

The problem is that many of those changes should have happened within 4 weeks. It shouldn't take 4 months to do all those changes.

Especially the very small, incremental changes. Doing many, but small incremental changes lets you really get balance to where it should be. And efficiently, without taking a ton of work.

The only time several weeks should be taken to balance something is when it's fundamentally flawed. Things like ECM, SSRMs, LRMs, BAP, ect. Those balancing changes most likely need mechanic changes. ECM needs to do less, so how much less? BAP needs to do more, so how much more? SSRMs just needs their entire locking mechanic changed. These kind of changes require rework to get right, so it would be best to determine what they should be doing first before reworking them.

But what PGI has done is they take months to rework a mechanic and taking weeks to just add/subtract a number from a XML file. If this is truely a beta, changes need to be happening quickly, especially since we are getting closer to launch.

[Xipe Totec]

I've stated this at length elsewhere, but suffice it to say that good statistical analysis is really hard to find on a tight budget, especially as PGI requires bilingualism.

I suspect this is a skill that they are hoping to pick up as they go along, but one of the reasons it is a difficult to find skill is that it is actually quite difficult to tell good work from hucksters.

My guess is that they are getting data-dumps from toolmaps and trying to make heads or tails of them, then fixing them post-hoc. It works over time to do it that way, but it lacks finesse isn't how most big players do it. Most big players have big budgets (and big prestiege) though, so don't hold your breath for some windfall of complex statistcial/economic modelling.

[Kattspya]

Xipe Totec, on 10 April 2013 - 02:32 AM, said:

I've stated this at length elsewhere, but suffice it to say that good statistical analysis is really hard to find on a tight budget, especially as PGI requires bilingualism.

I suspect this is a skill that they are hoping to pick up as they go along, but one of the reasons it is a difficult to find skill is that it is actually quite difficult to tell good work from hucksters.

My guess is that they are getting data-dumps from toolmaps and trying to make heads or tails of them, then fixing them post-hoc. It works over time to do it that way, but it lacks finesse isn't how most big players do it. Most big players have big budgets (and big prestiege) though, so don't hold your breath for some windfall of complex statistcial/economic modelling.

But not getting good statistical analysis is no reason not to get even bad statistical analysis. At the moment and throughout open "beta" they have not shown to make use of any statistics. There are still a bunch of weapons that are practically useless and that must reflect in the stats. All they would need to do is check the prevalence of a weapon in matches to see that it is underpowered compared to others or maybe not fun. But they aren't seen doing that. They aren't seen doing much of anything when it comes to balance. What they do are these large changes and then they don't revisit that change for a long while. I would much rather have quick oscillations going from underpowered to overpowered or even better from underpowered to less underpowered to balanced or almost balanced.

They aren't doing advanced statistical analysis they aren't doing amateur analysis and they aren't bruteforcing balancing. What are they doing?

[Deathlike]

Balancing team says "it's fine".