32 replies to this topic

[Tarogato]

Also posted to reddit here -----> https://www.reddit.c...rd_event_stats/

---

I was starting to think we'd never get a heavy mech leaderboard, and that my little collection of leaderboard stats posts would forever have a gaping void. But alas, much to the matchmaker's dismay, we got the final piece of the puzzle.


The above image shows the table sorted by average score. If you'd like to see the table sorted in other manners, see below:

- sorted by tonnage: https://docs.google....it#gid=66854569
- sorted by average score and normalised by tonnage: https://docs.google....#gid=1546054644

Yes, those are just two tabs in the same document. And there's other tabs if you're feeling particularly nosey.

---

What is this "normalised by tonnage" jazz anyways?

This means 20 tons = 100 tons. How I do this is plot out all the data points on a graph,

- Graph - top 75 scores from all chassis:



... and then I just level it off. This is done by applying a formula to all scores, (score × inverse tonnage) + (score ÷ some arbitrary weighting factor) ... The resultant scores represent nothing on their own (it's not as simple as score÷tonnage or anything like that), it is only the relationship between the scores that actually matters. Here is the result:

- Graph - top 75 scores from all chassis, normalised by tonnage:



You can see there how the red trend line now lays nearly flat. I opted to leave it banking a little upward toward the assault side of things. The "arbitrary weighting factor" is just a number I pick that makes the trend line roughly level, and the value is entirely dependent upon the data collected (because all sets of data will have a different trend line that requires a different amount of slope correction to be made level).

---

Special note:

- The light mech leaderboard ran for 7 days.
- The medium mech leaderboard ran for 5 days.
- The heavy mech (+NTG) leaderboard ran for 6 days.
- The assault mech leaderboard ran for 6 days.
- The Viper leaderboard ran for 8 days.
- The Cyclops leaderboard ran for 7 days.

The Viper, Cyclops, and Night Gyr were the only three mechs that didn't have a "chassis" leaderboard, but were rather broken up into individual "variant" leaderboards. What I did was take the top 75 scores across all variants and use that as my data for these two chassis. Also, I used the Viper data from the 7th day, and the Cyclops data from the 6th day, so NOT the final event scores for those chassis. This is closer to fair because their events lasted longer and they would have otherwise had more time to farm scores up. It's perhaps worth noting that "omg, but the Night Gyr scores are so high because of this, it's basically cherry picked!" ... no, not really. Because I did the same thing for the Viper and the Viper had abysmal scores, lol. ( ͡° ͜ʖ ͡°)

---

EXTRA!

There was an event that I missed - I forgot to post it to Reddit like all the others. But you can see it here: Weight Of The World Leaderboard STATS

Now here's a little bonus. The final rank of each chassis from all of the chassis leaderboard events compared with the final rank of each chassis that made it onto the Weight Of The World event, all sorted by the average of everything:

- RANK ALL THE THINGS: https://docs.google....t#gid=873915851

Edited by Tarogato, 28 September 2016 - 03:35 AM.

[Disapirro]

Very nice work and thank you for putting it together.

[Eregion]

Thanks Tarogato, for your effort and time.

Hope that PGI have someone like you, to do the math... or at least that they have someone who can "read" the data.
Very very usefull.

[Hit the Deck]

Thanks to Tarogato for compiling the data as always.

Thanks to PGI for not excessively pre-buffing the Night Gyr at release. It's a solid true to the lore 'Mech which one can be proud of owning not because of mega quirks it has!

Edited by Hit the Deck, 28 September 2016 - 04:04 AM.

[Bud Crue]

Taro the god of statistical analysis and hit box mapping.
As always, thanks for the effort; it is fascinating to peruse.

Edit:

GamerGirlGundam, on 28 September 2016 - 04:08 AM, said:

that pre-nerf post-resize nova thou..

Mechs in the middle of the Rank Of All Things chart...some of their performances were night and day.
What happened with the Summoner or the Orion IIc or example? And the Arctic Cheetah is all over the place.
BK variance due to rescale I assume? Again fascinating stuff. Amazing effort.

Edited by Bud Crue, 28 September 2016 - 04:33 AM.

[Y E O N N E]

Can you refresh us on what the Std. Dev. normalized by tonnage is?

[Tarogato]

Yeonne Greene, on 28 September 2016 - 04:51 AM, said:

Can you refresh us on what the Std. Dev. normalized by tonnage is?

It's just the standard deviation of the scores when normalised by tonnage. Really, I don't think it's all that useful, but some people always ask to see standard deviations, so...

[El Bandito]

All I can say from what I saw the tournament scores, is that ARCHERS NEED BUFFS. Seriously, 3000 top score is pathetic for a Heavy mech, especially considering how other 70 tonners have fared.

[Big Tin Man]

You and Tahawus should talk Tarogato. He is working on his stat crunch for the heavies as well. Well done.

http://mwomercs.com/...sts-are-op-but/

[Bud Crue]

I have to bump this. People, especially those arguing about comparative mech performance and balance, need to check this out, especially Tarogato's linked charts in the Extra section.

[MechWarrior414712]

El Bandito, on 28 September 2016 - 05:26 AM, said:

All I can say from what I saw the tournament scores, is that ARCHERS NEED BUFFS. Seriously, 3000 top score is pathetic for a Heavy mech, especially considering how other 70 tonners have fared.

Could also mean that missile weapons need a buff / rework (lrm)

[El Bandito]

I O O percent KongLord, on 28 September 2016 - 08:46 AM, said:

Could also mean that missile weapons need a buff / rework (lrm)

Definitely. And the first step starts with ECM rework.

Edited by El Bandito, 28 September 2016 - 09:30 AM.

[Jman5]

El Bandito, on 28 September 2016 - 05:26 AM, said:

All I can say from what I saw the tournament scores, is that ARCHERS NEED BUFFS. Seriously, 3000 top score is pathetic for a Heavy mech, especially considering how other 70 tonners have fared.

I can't help but wonder how much of these scores are due to straight imbalance and how much is from the number of people competing in each leaderboard.

For example, the Orion was an uncompetitive leaderboard. I was watching it in the first day and it took a long time until it even filled up to 75 players. The end result was a predictably low scoreboard.

The archer and catapult are pretty similar in design and playstyle. Yet the catapult leaderboard beat the pants off the archer leaderboard. If this was pre-resizing you could make a quirk argument but the catapult got its hitpoint quirks nerfed hard. In fact the Archer blows it out of the water in this regard.

Is the archer a truly bad missile-mech or is it simply an underplayed one?

[MechWarrior414712]

El Bandito, on 28 September 2016 - 09:29 AM, said:

Definitely. And the first step starts with ECM rework.

And the second step for fixing LRM is going back to the pre-artemis closed beta nuke LRM with like ½ the projectile speed of current LRM but twice the damage.

Jman5, on 28 September 2016 - 10:48 AM, said:

I can't help but wonder how much of these scores are due to straight imbalance and how much is from the number of people competing in each leaderboard.

For example, the Orion was an uncompetitive leaderboard. I was watching it in the first day and it took a long time until it even filled up to 75 players. The end result was a predictably low scoreboard.

The archer and catapult are pretty similar in design and playstyle. Yet the catapult leaderboard beat the pants off the archer leaderboard. If this was pre-resizing you could make a quirk argument but the catapult got its hitpoint quirks nerfed hard. In fact the Archer blows it out of the water in this regard.

Is the archer a truly bad missile-mech or is it simply an underplayed one?

Take note some of those cats can be K2s which are "ok", and A1 has much better convergence when it comes to SRM bombing. Archer has missiles in 4 different parts. Not good.

Edited by I O O percent KongLord, 28 September 2016 - 11:31 AM.

[Tarogato]

Jman5, on 28 September 2016 - 10:48 AM, said:

I can't help but wonder how much of these scores are due to straight imbalance and how much is from the number of people competing in each leaderboard.

For example, the Orion was an uncompetitive leaderboard. I was watching it in the first day and it took a long time until it even filled up to 75 players. The end result was a predictably low scoreboard.

The archer and catapult are pretty similar in design and playstyle. Yet the catapult leaderboard beat the pants off the archer leaderboard. If this was pre-resizing you could make a quirk argument but the catapult got its hitpoint quirks nerfed hard. In fact the Archer blows it out of the water in this regard.

Is the archer a truly bad missile-mech or is it simply an underplayed one?

I'm a believer that leaderboard performance is fairly indicative of true performance due to the fact that people will generally flock to better performing mechs and fewer people will play weaker mechs. So sure, maybe the Archer/Orion didn't have many big names competing, but perhaps that was for a reason.

[Y E O N N E]

I O O percent KongLord, on 28 September 2016 - 11:28 AM, said:

Take note some of those cats can be K2s which are "ok", and A1 has much better convergence when it comes to SRM bombing. Archer has missiles in 4 different parts. Not good.

I think it more likely the Jester was helping to juice the Cat score rather than the K2. The A1, Jester, and Butterbee were probably doing the heavy lifting.

[Davegt27]

what are we hoping to learn from these stats ?

my impression is the vast majority of Mechs are fairly close when you look at the scores

which should be a shocking statement

another thing is tonnage---does tonnage mean anything in this game

I would think fire power means more

[Y E O N N E]

Davegt27, on 28 September 2016 - 04:20 PM, said:

what are we hoping to learn from these stats ?

my impression is the vast majority of Mechs are fairly close when you look at the scores

which should be a shocking statement

another thing is tonnage---does tonnage mean anything in this game

I would think fire power means more

The average suggests how good the chassis is relative to its class peers, but the standard deviation will provide a notion on how trustworthy that average is. Higher standard deviation will tell you that it is a more inconsistent performer.

So, while it looks like the spread is fairly close and roughly correlated to tonnage, the standard deviations suggest otherwise.

Edited by Yeonne Greene, 28 September 2016 - 04:29 PM.

[Tahawus]

I will try to get my stats updated this evening. They'll provide another approach to looking at the leader board results.

[Hit the Deck]

Yeonne Greene, on 28 September 2016 - 04:28 PM, said:

The average suggests how good the chassis is relative to its class peers, but the standard deviation will provide a notion on how trustworthy that average is. Higher standard deviation will tell you that it is a more inconsistent performer.

So, while it looks like the spread is fairly close and roughly correlated to tonnage, the standard deviations suggest otherwise.

Just a note, be careful when interpreting those SD/standard deviations. Two equal SD values of two different averages doesn't mean that they have the same spread.