13 replies to this topic

[Texugo87]

I was jonesing to do some data entry today, and couldn't pass up having a look at the dataset provided by the outcome of the leader board challenge. Hopefully it goes without saying that the challenge structure and environment undoubtedly introduced confounding variables, so this data is to be taken with a grain of salt. Also, bear in mind that the top 15 Atlas players may only represent the top 1% of Atlas pilots, while the top 15 players of some less popular chassis might represent a much higher % of that population, dragging their average down (I'll come back to this later).

Also, I only looked at IS mechs, as that seems more relevant with the Quirks coming out next week.

Mechs ranked by mean score, with Standard Deviation given
Battlemaster, 2198, 115.4
Atlas, 2196, 64.6
Stalker, 2181, 56.5
Banshee, 2150, 112.9
Jenner, 2116, 106.4
Awesome, 2116, 84.1
Highlander, 2112, 118
Orion, 2096, 68.9
Jagermech, 2074, 97.2
Catapult, 2070, 58.4
Firestarter, 2069, 82
Raven, 2067, 67.8
Commando, 2056, 92.3
Blackjack 2025, 108
Cataphract, 2022, 72.1
Cicada, 2005, 84.3
Quickdraw, 1988, 121.8
Kintaro, 1986, 67.1
Victor, 1981, 112.8
Spider, 1972, 112.9
Shadowhawk, 1967, 104.5
Thunderbolt, 1964, 77.6
Locust, 1958, 100.5
Griffin, 1954, 107.9
Centurion, 1941, 96.2
Trebuchet, 1930, 84.3
Dragon, 1927, 81.7
Wolverine, 1915, 117.9
Hunchback 1903, 82.3
Vindicator, 1832, 95.3

Lights alone:

Jenner, 2116, 106.4
Firestarter, 2069, 82
Raven, 2067, 67.8
Commando, 2056, 92.3
Spider, 1972, 112.9
Locust, 1958, 100.5

Not a lot of surprises here, and likely in line with how most players would rank the chassis.

Mediums Alone:


Blackjack 2025, 108
Cicada, 2005, 84.3
Kintaro, 1986, 67.1
Shadowhawk, 1967, 104.5
Griffin, 1954, 107.9
Centurion, 1941, 96.2
Trebuchet, 1930, 84.3
Wolverine, 1915, 117.9
Hunchback, 1903, 82.3
Vindicator, 1832, 95.3

Poor, poor Vindicator.

Heavies Alone:


Orion, 2096, 68.9
Jagermech, 2074, 97.2
Catapult, 2070, 58.4
Cataphract, 2022, 72.1
Quickdraw, 1988, 121.8
Thunderbolt, 1964, 77.6
Dragon, 1927, 81.7

Orion on top is a bit surprising, especially with such a low SD. Bear in mind that it is the Heaviest heavy... I'll address this a little lower down.

Assaults Alone:

Battlemaster, 2198, 115.4
Atlas, 2196, 64.6
Stalker, 2181, 56.5
Banshee, 2150, 112.9
Awesome, 2116, 84.1
Highlander, 2112, 118
Victor, 1981, 112.8

Awesome better than Victor and Highlander? Battlemaster the best Assault? WTF is going on here?

Anyway...

Remember what I said earlier about portion of the population the top 15 of each chassis captures being different, and how that might be problematic? I think it is somewhat safe to make the assumption that the larger the population of each chassis, the lower the standard deviation between the top 15 scores should be.

With this in mind I looked for a correlation between SD and mean. A strong, negative, correlation (Higher the SD, lower the mean score) would mean that the ranking between chassis might be reflecting chassis popularity more than anything else. However, I only found an overall weak correlation of R=-0.18. I also looked at this within chassis class (light, assault, etc.), and also found weak to moderately weak negative correlations with Assaults at -0.43, Heavies at -0.30, Mediums at -0.06, and lights at -0.38.

This means that IF our assumption of SD as indicating population is sound, population isn't strongly influencing the ranking. But what else might be?

I decided to look at chassis weight next. Overall, I got a moderate correlation of 0.54, meaning heavier chassis perform somewhat better overall on average. The picture changes a little when I broke it down by chassis however, Lights show a strong relationship of 0.72, Heavies moderately strong at 0.69, and assaults show a moderate relationship at 0.54. Mediums, weirdly, show a weak, negative correlation of -0.09.

Finally, I looked at Class averages: Assaults were the highest at 2133, then Lights at 2040, Heavies at 2020, and mediums at 1946. This pretty much nails my experience and opinion on chassis balance in the current state of the game.

TLDR: Don't feel bad taking a Locust, in the right hands they are better than Griffins, Centurions, Trebuchets, Wolverines, and Dragons, and only a little behind the Victor. Speaking of Victors, you should probably trade yours in for an Awesome if you don't want your team to lose!

EXTRA BONUS STAT: There is practically as strong of a correlation between alphabetical mech name rank and score (-0.52) as there is mech weight and score (0.53)! Zeus is DOA, you heard it here first!

Edited by cjmurphy87, 28 October 2014 - 11:03 AM.

[Belorion]

Salt makes food taste better so I will take my grain.

One interesting comparison would be to look at the median scores based on the IS tier the mech was placed in vs Clan as a whole.

[Jman5]

Quote

Median SD Hunchback 2074 97.4

Maybe I'm not understanding it right, but I calculated the median value of the hunchback and got about 1883 with a standard deviation of about 82. The only person to score at or higher than 2074 was me.

[Texugo87]

Jman5, on 28 October 2014 - 10:23 AM, said:

Maybe I'm not understanding it right, but I calculated the median value of the hunchback and got about 1883 with a standard deviation of about 82. The only person to score at or higher than 2074 was me.

Good catch, I duplicated the Jager's dataset, correcting it now. I also typed median when I meant mean.

Edited by cjmurphy87, 28 October 2014 - 10:59 AM.

[Thorqemada]

Both the Hunchbacl as well as the Blackjack can use the AC20 as well as being Laser-Boats, the Cicada can ECM or Laserboat and the Kintaro is a Missile-Monster while all of them probably be somehow lower priority targets compared to a SHD which is a Medium with higher target priority and a size not to overlook.

The Thunderbolt simply is that weak - to slow and to big to be a frontline Mech and way to weak soaking up hits...my SHD is more durable with less Armor than my Thunderbolt - the Dragon is even weaker and the Quickdraw has no good shape and size also.

The Battlemaster i guess lives in this Challenge form being a supreme Lurmboat as it is weaker than the other Assaults in any other category.

And Clans be a league of their own - cant wait to process this Bio-Mess called Clans into Soylent Green.

[Funkadelic Mayhem]

I can 1 + 1 = doesn't matter also.

[Artgathan]

There are many errors in the presented values.

For example (Mech Name - Mean Value)

• Atlas - 2169
  
• Battlemaster - 2167.6
  
• Commando - 1908.6

I would suggest going over them again and then re-evaluating some of your conclusions. Even then, this data is not particularly useful. These numbers would suggest that a Dragon is on par with a Shadowhawk, both of which are "just as good" as a Locust.

[Ultimax]

cjmurphy87, on 28 October 2014 - 08:27 AM, said:

Awesome better than Victor and Highlander? Battlemaster the best Assault? WTF is going on here?

1) JJs nerfed.

2) Torso Twist Speeds way over-nerfed.

3) PPCs nerfed

4) Too Few Energy Hardponts

5) Laser Meta (directly impacted by #4)

Edited by Ultimatum X, 28 October 2014 - 12:02 PM.

[Adiuvo]

You're trying to inject analysis into a flawed data set For many reasons, these tournaments, and especially this one, are completely pointless to determine anything. They're just for fun. Elo plays a factor, individual pilots are the ones who drive the scores up, etc..

[orcrist86]

You do realize your only sampling the right hand tail of the curve and you need a lot more data.... Also because the tourney is opt in you have to account for skill bias.

[Lily from animove]

I made a excel sheet earlier today

https://docs.google....dit?usp=sharing

Edited by Lily from animove, 28 October 2014 - 12:26 PM.

[Sorbic]

I enjoy my locusts and can often do better in it than some other heavier mechs. However I probably never get one shot after doing just 60 dmg in my Victor... It's interesting to see how well each chassis is capable of competing in the tourney but the limited data they give us doesn't show the average score, just the good games.

[Gaussfather]

Thanks CJ, it was fun reading your analysis even if there are some errors to fix.

I personally enjoy piloting Victors (and Highlanders) more than other mechs for PUG matches... but as Ultimatum X said, then came the nerfs on those 2 mech chassis with the Victor getting it the worst. I managed to get 7th place in Victors but I think the in-match competition was a bit skewed toward LRMs boats and lasers. Victors are all about ballistics (for me) and PPCs, with paltry LRMs or SRMs as backup. I wound up switching out my SRMs on my Vic builds to LRMs to try to score higher in my matches (more assists and damage).... maybe in group warfare it would be different since the Victor is more of a skirmisher/hit and run mech with good dakka and ok energy back-up.

The fact that Kintaro's (my worst mech which I desperately wait for new quirks on) are ranked higher than Shadowhawks (the best medium in my opinion) is probably because you can put a lot of lurms on them. Same goes with Battlemaster and Awesomes.

Most Clan mechs (besides the DW) are also laser and lurm boats because their ACs are nerfed and tough to aim.

But in a Vic you have a decent chance against a moving light mech compared to most assaults and its a good brawler when properly outfitted/piloted. I just hope they remove the negative quirks now that Clan mechs are here and are clearly superior by weight class (except for the lights).

Jagermechs and Cataphracts (and DWs) are an exception to the "lurm rule" because you can Dual Gauss and Dual AC20 and have some amazing 10 match scores (i.e. lots of kills) if you play enough matches.

With the Victor you can make a 40 alpha-shot build with dual PPCs and an AC20. Sometimes it worked great on the right map/team, sometimes it sucked.

So all in all I enjoyed your thread and analysis, it confirms to me what is a problem with the latest iteration of this game:

Too much fire and forget and hardly any targeting skill (lasers are instant hits... lrms lock-on).

Would be great if PGI hired you to do some analysis with their dataset! So we could see what weapon builds performed the best.

[Vaderman]

What's going on is Lurms.

Almost every battlemaster I saw during the tournament was carrying nothing but lurms.

In fact, there were a lot of people doing nothing more than chain firing lrm5s.