312 replies to this topic

[Kiiyor]

Aim64C, on 16 July 2014 - 10:39 PM, said:

As I've been playing over the past week or so, I've noticed a considerable shift in how clan mechs play. While the meta should be expected to shift, to some degree, I am wondering if it is going to stagnate on PPCs and LRM/SRM builds.

I've come into the discussion a bit late - but it might be more useful to study the blowout matches in terms of discovering imbalancing factors.

I've seen more 'blowouts' the past couple nights than I have in the previous week and a half (or two - haven't been tracking that closely). Pretty much all of the games I played were a near shut-out victory of <=3 losses for the victor.

Good point. It's my aim to add the stats from blowouts in the next rush of data, but also some specific imbalance metrics like

% chance of winning If you have

1 more clan mech
2 more clan mechs
3 etc
1 more IS
2 more etc etc.

I think those will be valuable, possibly. Ken has already sent me some data and asked for the same thing

Also, I'm all for having SRM's a dominant force on the battlefield, Clan or otherwise. Anything that gets people closer for fisticuffs can only be a good thing.

[Lynx7725]

Kiiyor, on 16 July 2014 - 10:40 PM, said:

Spoiler

Yeah that's totally how they build the Locust in the factory.

Back on topic, in a few weeks the figures for clan mechs may start to shift again, as the mechs become more readily accessible for the players. We'd likely see a drop in performance.. are you going to track trends over time?

[Kiiyor]

Lynx7725, on 16 July 2014 - 10:51 PM, said:

Yeah that's totally how they build the Locust in the factory.

Back on topic, in a few weeks the figures for clan mechs may start to shift again, as the mechs become more readily accessible for the players. We'd likely see a drop in performance.. are you going to track trends over time?

Yep, i've got around 150 matches of my own, though they're not all post 3/3/3/3, and as long as people keep sending me data, i'll keep updating the results.

In my initial analysis, I was tracking performance over time, but now with the community contributing, that will be harder.

Hmmm... most screenshots have dates though, so it might be viable after all...

[Haakon Magnusson]

Joseph Mallan, on 24 June 2014 - 04:22 PM, said:

Actually my point was that Clan Mechs are not tough enough yet. I have said it many times I want the Boogieman, I want to sweat when I see a Timber Wolf and above... I don't have that yet.

Well, that's not what PGI wants.. they want equal footing and 12v12. So unfortunately they can't be any tougher (Or less)
Although I hope they won't change IS ACs to burst (I still maintain that it would be good for IS ultras as well... flavorvise setting them apart) because that is definite equalizing factor versus clan dot damage


EDIT:

Additionally I don't believe setting (c) mechs apart really does anything good to numbers (except distorts them), it just gives an easier way of spotting some of the new players that have been tossed to the deep end of the pool. There are others there, but they are simply not wearing their (c) colors, but trying to make do with that one mech they own. Which won't either have elites open, but you can't spot it's better/worse performce.
Also, there are some long time players in their freebie (c)enturions and other mechs.

Edited by Haakon Magnusson, 16 July 2014 - 11:22 PM.

[SgtKinCaiD]

Thanks for your work !!!

[Kiiyor]

Haakon Magnusson, on 16 July 2014 - 11:11 PM, said:

Well, that's not what PGI wants.. they want equal footing and 12v12. So unfortunately they can't be any tougher (Or less)
Although I hope they won't change IS ACs to burst (I still maintain that it would be good for IS ultras as well... flavorvise setting them apart) because that is definite equalizing factor versus clan dot damage


EDIT:

Additionally I don't believe setting (c) mechs apart really does anything good to numbers (except distorts them), it just gives an easier way of spotting some of the new players that have been tossed to the deep end of the pool. There are others there, but they are simply not wearing their (c) colors, but trying to make do with that one mech they own. Which won't either have elites open, but you can't spot it's better/worse performce.
Also, there are some long time players in their freebie (c)enturions and other mechs.

That's true about decent players in (C) mechs, but the damage and kill numbers are pretty telling - and the standard deviation is too.

154.54 average (C) damage, with a standard deviation of 118.41 (this is HUGE!) vs
256.81 average damage, with a standard deviation of 61.61

The (C) deviation means there are wild swings in the data - so the -118.41 players would likely be the newest, and the +118 players more experienced, as that puts them close to par with average damage overall.

I added the metric because it had a very noticeable impact on overall scores. If the Clans had (C) mechs, things would likely balance out, though I also think clan mechs are harder to get the hang of initially than IS starter mechs.

I'm trying to think of the best way to make a fair analysis of overall performance, and I believe that (C) mech performance is just too erratic to be anything but detrimental to overall discussion. Plus, it's also a nice topic for debate

[Haakon Magnusson]

Kiiyor, on 16 July 2014 - 11:53 PM, said:

That's true about decent players in (C) mechs, but the damage and kill numbers are pretty telling - and the standard deviation is too.

154.54 average (C) damage, with a standard deviation of 118.41 (this is HUGE!) vs
256.81 average damage, with a standard deviation of 61.61

The (C) deviation means there are wild swings in the data - so the -118.41 players would likely be the newest, and the +118 players more experienced, as that puts them close to par with average damage overall.

I added the metric because it had a very noticeable impact on overall scores. If the Clans had (C) mechs, things would likely balance out, though I also think clan mechs are harder to get the hang of initially than IS starter mechs.

I'm trying to think of the best way to make a fair analysis of overall performance, and I believe that (C) mech performance is just too erratic to be anything but detrimental to overall discussion. Plus, it's also a nice topic for debate

Yes, but if you are dropping (c) mechs because of the newbs in them, you would have to have some way of identifying newbs in other mechs as well, because they will be there influencing numbers as well, just downing into the mass (ie reducing std dev) because of greater number of veteran users in those mechs.

But in all honesty, elo of the people in the end of match reports influences wildly your results. If you'd take only your results, then you would know that at least your own elo is in the same ballpark all the time, you will still get averaged elo bucket for a team... since you can't see where others are at, you are missing facts.

Interesting fact, did you know that ice-cream sales are up come summertime. Also did you know number of drowning deaths are also increased? Coincidence, I think not!! (Just kidding, but still, like this example. We are missing information which might explain numbers)

[Kiiyor]

Haakon Magnusson, on 17 July 2014 - 12:10 AM, said:

Yes, but if you are dropping (c) mechs because of the newbs in them, you would have to have some way of identifying newbs in other mechs as well, because they will be there influencing numbers as well, just downing into the mass (ie reducing std dev) because of greater number of veteran users in those mechs.

But in all honesty, elo of the people in the end of match reports influences wildly your results. If you'd take only your results, then you would know that at least your own elo is in the same ballpark all the time, you will still get averaged elo bucket for a team... since you can't see where others are at, you are missing facts.

Interesting fact, did you know that ice-cream sales are up come summertime. Also did you know number of drowning deaths are also increased? Coincidence, I think not!! (Just kidding, but still, like this example. We are missing information which might explain numbers)

This is true. I've considered ELO, and I have no idea how to account for that, so I assume the MM does it. Also, while there are (probably) generally n00bs in (C) mechs, there will undoubtedly be n00bs piloting Clan machines also.

In my mind, the data will be influenced strongly by the lack of data - the majority of matches are from myself, one other major contributor, and around 6 others.

Hopefully, the community pitches in and buries me in enough EOM screenshots to dilute the effect of my own matches. More data tends to be more truthful than less data, after all.

Also, this is taken over a handful of weeks - another contributing factor.

[JimboFBX]

Wins matter the most. Trial or no trial. Getting credit for a kill is kind of luck. Clans can take more damage before dieing since they require both side torsos to be destroyed before they die, so damage is also an unreliable metric. The extra 50 damage a mech for clans is probably from the ER LL and ER ML, which have a significant range advantage and allow a clan mech to cover a significant area, which I've already made a post about that people have ignored.

Instead of dealing with averages, median values might also be a better metric, once you throw away the noobs.

[Kiiyor]

JimboFBX, on 17 July 2014 - 12:19 AM, said:

Wins matter the most. Trial or no trial. Getting credit for a kill is kind of luck. Clans can take more damage before dieing since they require both side torsos to be destroyed before they die, so damage is also an unreliable metric. The extra 50 damage a mech for clans is probably from the ER LL and ER ML, which have a significant range advantage and allow a clan mech to cover a significant area, which I've already made a post about that people have ignored.

Instead of dealing with averages, median values might also be a better metric, once you throw away the noobs.

Yep - but although wins are very much the end state and ultimate metric for games in MWO, I'm trying to assess balance on a more intimate faction vs faction scale. Lots of people have been crying that the Clans are OP/Just Fine/Underpowered (lol) whatever, and with the crushing numbers of them dropping in matches, they most assuredly are OP on the merit of numbers alone.

I'm after a mech vs mech statistic. Think of it as trying to gauge what balance would be like if it were pure Clan vs pure IS matches. While we can't get that with mixed factions, I feel that trying to cherry pick closely faction weighted matches is the closest we'll get until CW, and faction vs faction.

Also, i'm going to release some win related metrics in the next batch of data. Also², I do love me some median action, so i'll crunch that too, methinks.

[ApolloKaras]

Damn fine work mate.

[TheCrazySteve]

Great stuff. What follows is not much of a criticism, just some thoughts. Also, I completly commend your commitment and am in no way attempting to argue you haven't done enough, you have already done allot more than everyone else here.

One big issue is how to account for ELO. Assuming people play one faction the majority of the time, eventually ELO will balance out the faction divide. Perhaps the most telling data then would be from those who play equally between the two factions. While a single player wouldn't provide much meaningful data, if you could find 30-50 players who play say 20 rounds in like an IS heavy, 20 rounds in a clan heavy, it would be interesting to see the results.

[bluepiglet]

Leaving aside the Lights which are fubar and has became quite irrelevant since the last 2 patches, It's not surprising that only the IS Assault class is less dominated by their Clan counter parts. Because currently only the way IS to compete with Clan is taking utilizing the high Alpha damage of their Ballistic and Missile, only the ones capable of carrying large amount of such weapons stand any change, aka the Assault class.

[Alistair Winter]

Thanks again. This thread needs to go in the Hall of Fame of MWO forum threads.

[Charons Little Helper]

Of note - while I appreciate the #'s - the whole thing is somewhat invalidated by the existence of Elo. (yes it does work - no it doesn't work perfectly)

If a clan heavy mech causes you to win significantly - it'll bump up your Elo. Therefore, even if the Clan heavy is better, Clan heavy mech pilots will be pushed up in Elo to go up against IS pilots who're more skilled, canceling out much of the statistical differences you're trying to peg.

[ravineh]

Kiiyor, on 24 June 2014 - 04:04 PM, said:

If you remove any matches with at least one (C) mech, the clan are just under 10% more effective than the IS

I guess some people are missing someting here if u say it like that.

Yes with avg dmg/mech/match being 270 for IS and 327,5 for CLAN, the % differene is 9,63.
But how much more dmg on avg CLAN mechs do compared to IS? - 21,2%

With avg kills/mech/match being 0,66 IS and 0,76 CLAN, the % difference is 7,06.
Meaning the avg kill/mech/match for CLANS is 15,15% higher.

Thats not so small actually.

[Angel of Annihilation]

I like the study but your conclusions are a bit skewed.

What I see is that more clan mechs at a higher tonnage produce higher damage and kills. Seems logical to me.

Clans have a 2% numerical advantage overall but also a 7% tonnage advantage ( more mechs + more tonnage = more weapons = more damage).

This 2% numerical and 7% tonnage advantage produces roughly a 8% advantage in total damage and 6.5% advantage in total kills.

Seems pretty damn balanced to me so I have no clue how your coming up with the clans having roughly a 10% total advantage across the board.

[Mystere]

bluepiglet, on 17 July 2014 - 04:06 AM, said:

Leaving aside the Lights which are fubar and has became quite irrelevant since the last 2 patches, It's not surprising that only the IS Assault class is less dominated by their Clan counter parts. Because currently only the way IS to compete with Clan is taking utilizing the high Alpha damage of their Ballistic and Missile, only the ones capable of carrying large amount of such weapons stand any change, aka the Assault class.

Doing such a thing would be a bias-induced mistake. As such, it's something that should not be done in a general IS vs. Clan performance analysis.

[bluepiglet]

Mystere, on 17 July 2014 - 04:48 AM, said:

Doing such a thing would be a bias-induced mistake. As such, it's something that should not be done in a general IS vs. Clan performance analysis.

As you wise then, I believe your maths is capable enough to see obvious. Yeah, that's how underpowered Clan meches are, particularly the Light Class right?

[Koniks]

Kiiyor, on 17 July 2014 - 12:17 AM, said:

This is true. I've considered ELO, and I have no idea how to account for that, so I assume the MM does it. Also, while there are (probably) generally n00bs in (C) mechs, there will undoubtedly be n00bs piloting Clan machines also.

In my mind, the data will be influenced strongly by the lack of data - the majority of matches are from myself, one other major contributor, and around 6 others.

Hopefully, the community pitches in and buries me in enough EOM screenshots to dilute the effect of my own matches. More data tends to be more truthful than less data, after all.

Also, this is taken over a handful of weeks - another contributing factor.

Thank you for the analysis. It's certainly interesting.

But you're right that it's nowhere near scientific. Even diluting the data you contributed to the set with EOM screenshots from other players isn't necessarily going to improve that because those players will have self-selected. So there's likely going to be a lot of bias in the results. This is still more like a lot of anecdotes than good data.

PGI should be more transparent with the data it has available.