312 replies to this topic

[Kiiyor]

Science!

Edit: New data!

Here's the old data, if you're interested:

Spoiler

So, it's not really Science, it's more of a questionable statistical analysis of a (probaly too small) small sample size of matches.

Firstly, I've found the entertainment value present in all the CLANS ARE OP/CLANS ARE FINE threads to be almost as great as piloting these awesome new Clan mechs.

After a while though, I began to wish for some actual hard data to throw at people. "Kiiyor, you take screenshots of almost every EOM screen. You should collate that info!" the ice cube in my Scotch whispered to me the other night.

10 minutes later, having transferred barely half of one result to excel, my Scotch and I both agreed it was a job for future me. Also, being slightly inebriated, I was more interested in using the REPT function in Excel to make wang charts. HILARIOUS.

Anyhoo, after almost a full freaking day working out Photoshop batching, and with the help of a passable piece of free OCR software, I had the results from 110 matches stored in Excel.

The result? Some (still very questionable) SCIENTIFIC ANALYSIS. No graphs though. Just some lovely numbers.


WARNING!!!

Some caveats about the below data:

• 100 matches is barely viable as a sample size. It probably isn't viable. SERIOUSLY. At the most it’s an indication. But SCIENCE.
  
• My own details are recorded in every match, which guarantees one Clan mech per battle, as I have been deeply involved in a torrid and passionate affair with the StormCrow. In a sample size as low as this, it will definitely skew the results. Well, the MM should counter that, actually.
  
• In a perfect world, I would have counter-run 110 matches in some Sphere hardware.
  
• To try and keep things more viable, I ran only in my StormCrow, figuring that mediums would have less of an impact on the battlefield. My own kills/match and damage/match were well above the average though, so that will also skew the results.
  
• My ELO in mediums is questionable. I was a terror in the 4SP when it reigned supreme, but shelved it for aeons when enemy mechs became capable of punching Jenner sized holes in it with one Alpha.
  
• Mixed battles... it's far harder to do a decent comparison when the samples are mixed. If there was full separation, each factions strengths/flaws would probably become far more apparent.
  
• OCR SOFTWARE! I used some text recognition software to gather the data. It had some trouble (sometimes) working out if a 6 was an 8, 3 or 9. I paid particular attention to this, but some errors may have slipped through.
  
• Almost 100% pug. 4 or 5 matches with a mate.
  
• The data discusses damage/killin' mostly, and not utility (ECM, AMS etc)

THE RESULTS!


(note, they are an image. I couldn't format them as text here to save myself)






So. How bout them shiny numbers? Note that i've got some top 10's (most prevalent mechs, best killin' machines, biggest death traps etc) but I haven't been bothered formatting them as yet. I may upload them later. Edit: top 10's added. GO TEAM STALKER!

Some analysis of the results:

• The Clans were more prevalent on the battlefield.
  
• The Clans were VERY top heavy.
  
• The average damage for the Clans was far higher than the IS, but the IS were far more efficient at killing (less damage / kill).
  
• IS lights are... not doing well, but really, neither are the Clan lights.
  
• Clan heavies are everywhere! Ain't no party like a TimberWolf party.
  
• The biggest competition against the clans seem to be IS Assaults.
  
• Go Stalker, you big metal phallus of DOOOOM!

Note that these figures are not normalized. I had a crack at normalizing the data for kills and damage, but really can't remember how to interpret the results. I've put this in a spoiler, because it's mathy.


Regarding the Charts!

• I think my ELO in mediums was questionable, so the increases in the charts could be attributed to me murdering my way up the ELO brackets.
  
• Also, it's over about 5 days. I'm recording more data to upload, so we'll see how this looks over the next few weeks.

Personal observations:

• The IS are still learning how to combat the clans... but mixed matches isn't helping. It's hard for a new meta to develop when neither side is forced to confront each other's specific fighting style.
  
• Having said that, a grace period getting to learn the enemy is probably very helpful. Ease into it.
  
• The Clans, if they out-maneuver you, are lethal. They suck at peek and shoot; they need quality time with the IS under their crosshairs.
  
• The IS is FAR more effective at defence. That's not from the figures, it's from personal experience.
  
• The IS needs to stick and move; fire off an alpha, back to cover. Standing toe to toe is asking for limb removal.
  
• AMS is ludicrously effective against clan LRM's, but no-one carries it!
  
• Clan heavies seem to be the best bang for buck. Also; killing machines.
  
• Forcing the clans to maneuver can expose holes in their defence, as their fixed engine sizes is detrimental to their assaults (but not their heavies!).

Things I would like:






More data! Does anyone else collect EOM screenshots? Have any factions been using private matches to host their own IS vs CLAN grudge matches? DOES ANYONE CARE?

Anyone who wants to send me EOM screenshots (and if you do... please, please be objective - no selective collections of one sided stomps please, or no edits of the data to make all the scores spell 80085 (boobs - hilarious!) you can email them to:

sendmemwojunk@hotmail.com

Overall:

Overall, I think the clans are remarkably well balanced when compared to IS mechs, and that some of the disparate figures above are more indicative of the IS needing to develop a new meta to combat the way the Clans are built to wage war. At the moment though, the clans do seem to have an advantage, but it isn't as great as many people seem to think it is. I took the bull by the horns for a few matches and tried to force any IS heavy groups into defence, and we seemed to do very, very well, the usual pug-related one man charges and the like notwithstanding. Many Clan mechs tend to have low slung weapons, and have to expose lots of their profile to unleash their furious barrages of doom.

Also, these are new machines and new weapons; ongoing balance will undoubtedly help bridge any gaps in faction viability.

THOUGHTS?

Edit: uploaded damage and kill charts.
Edit 2: Actual charts and trends added. LOOK AT ALL THE SQUIGGELY LINES.

Thanks to the community and my unflinching devotion to mech science, I've got close to 500 matches worth of cold, hard numbers now.

Instead of haphazardly totaling numbers, I had a dig through the raw data this time, and found some interesting things:

Interesting thing #1; outliers. While a large portion of matches had similar(ish) counts of IS and Clan mechs on each team, some… didn't. This posed a problem, being that it was difficult to get an idea of proper faction performance if the numbers weren't weighted correctly. So, I added some new columns, worked out the weighting of faction mechs on each side of every battle, and found that not all matches were created equal.

More interesting things below:

Note: This is mostly prior to 3/3/3/3. I'm going to run totals for subsequent batches of data based on that.

Note²: most of the below data is in image form. I couldn't format this properly here to save myself. There's also a TL;DR at the bottom, because OH GOD this thread needs it.











Up next: top 10's. Ives wrapped this one in a spoiler, because it's lengthy:

Spoiler

Well, that was in depth.

Observations:

So how about those numbers?

• I think i'm on to something with the contributing factors with the performance of Inner Sphere mechs. The disconnects and (C) mechs make a big difference.
  
• Looking at the sample above with no (C) or 0 damage mechs, it seems as though things are very nearly balanced. I'd love to have enough matches to solidify this, but I think there would have to be waaay more data than I have now, as decent, even matches are fairly rare.
  
• It seems that IS mediums are being stomped into the dirt and having their lunch money stolen before being hung from a flag pole by their underwear. Someone Photoshop that for me please. It may be due to the IS having so many mediums, with only a couple of truly viable chassis.
  
• Clan heavies are dominating. Correction, the MadCat is dominating.
  
• I believe things will be far better balanced if the Clans ever get (C) mechs themselves.

A note on the data:

I've since converted about 100 extra matches, and the Clan kill/damage gap has widened by about 1.5% over the IS. It'll be interesting to see how that pans out.



What I would like:


Anyone who wants to send me EOM screenshots (and if you do... please, please be objective - no selective collections of one sided stomps please, or no edits of the data to make all the scores spell 80085 (boobs - hilarious!) you can email them to:

sendmemwojunk@hotmail.com

I would also dearly love to get my hands on some team stats, just to see how things compare to Pugs.



Conclusion:

I think that the Clans and IS are very, very close to balanced, but the IS throws a spanner in the works with Newbie mechs and disconnects. Any future balance adjustments should be considered carefully, though I do believe that instead of nerfing the Clans, the IS should be buffed somewhat.

Also, it's nice to say that (C) and 0 damage mechs can be removed from analysis, but their existence is inescapable on the battlefield, unless you 12 man of course.

THOUGHTS?

Edited by Kiiyor, 16 July 2014 - 05:28 PM.

[Joseph Mallan]

Well except for the over abundance of Clanners the numbers look a bit off. Clanners should be out damaging us Spheroids by a greater margin.

[Kiiyor]

Joseph Mallan, on 24 June 2014 - 04:07 PM, said:

Well except for the over abundance of Clanners the numbers look a bit off. Clanners should be out damaging us Spheroids by a greater margin.

I thought so as well. That may be due to the small sample size though, or maybe ELO. I went over the damage and kill numbers from matches with a fine tooth comb though.

[Mcgral18]

Joseph Mallan, on 24 June 2014 - 04:07 PM, said:

Well except for the over abundance of Clanners the numbers look a bit off. Clanners should be out damaging us Spheroids by a greater margin.

Kiiyor, on 24 June 2014 - 04:10 PM, said:

I thought so as well. That may be due to the small sample size though, or maybe ELO. I went over the damage and kill numbers from matches with a fine tooth comb though.

More damage per kill for Clans could also be related to burst ACs and longer burn lasers. This gives more room for the target mech to twist and spread damage, while the IS ACs don't give that option, and their pulse lasers are considerably harder to spread compared to Clan pulses.

So, that stat makes sense.

[Joseph Mallan]

Mcgral18, on 24 June 2014 - 04:19 PM, said:

More damage per kill for Clans could also be related to burst ACs and longer burn lasers. This gives more room for the target mech to twist and spread damage, while the IS ACs don't give that option, and their pulse lasers are considerably harder to spread compared to Clan pulses.

So, that stat makes sense.

Actually my point was that Clan Mechs are not tough enough yet. I have said it many times I want the Boogieman, I want to sweat when I see a Timber Wolf and above... I don't have that yet.

[1453 R]

Good work, and interesting to see. Stunningly enough, Clan heavies seem to be somewhat over-represented in the matchmaker at current. I wonder why...

[jaxjace]

These numbers dont seem too scary, what ive found in personal experience many times that the only thing keeping me alive against them is my superior piloting which i say modestly because i am not that great, but many clanners i feel are just scrubs who tried to p2w but are now suffering for it, I am afraid of what happens when the clanners learn how to use their damn mechs.

Im loving the data on how efficient IS kills are, hopefully we will fine tune and get better at that to combat the clans learning curve and ridiculous damage potential.

[Kiiyor]

Joseph Mallan, on 24 June 2014 - 04:22 PM, said:

Actually my point was that Clan Mechs are not tough enough yet. I have said it many times I want the Boogieman, I want to sweat when I see a Timber Wolf and above... I don't have that yet.

I sweat when I see a TWolf, heh - especially if it's missing it's ears. Means it's loaded with SRMS.

[1453 R]

Kiiyor, on 24 June 2014 - 04:30 PM, said:

I sweat when I see a TWolf, heh - especially if it's missing it's ears. Means it's loaded with SRMS.

Depending on how the TC works out, my best/favorite Timber Wolf's going to be the most evil Marauder analogue in the Inner Sphere. Two C-ERPPCs over two C-ERSLs in the arms, a C-UAC/5 in the RT, and a Mk. V TC in the LT. Maybe make space for an Active Probe and do ALL the heavy scouting.

No ears? Check. Horrifying murder machine? Also check

Edited by 1453 R, 24 June 2014 - 04:34 PM.

[R Razor]

Thanks for taking the time to post those stats.........they are interesting. That said, I played a few games on the test server as well as on the open server tonight, and I didn't see a significant numerical difference nor a significant firepower advantage on the part of the clans. Again though, only a few games were played.

What I notice more than anything since the clans came out is that LRM's are back with a vengeance, almost every clan mech on the battlefield packed at least one rack. Didn't seem to overly impact the outcome of the game though.

[Kiiyor]

R Razor, on 24 June 2014 - 04:37 PM, said:

Thanks for taking the time to post those stats.........they are interesting. That said, I played a few games on the test server as well as on the open server tonight, and I didn't see a significant numerical difference nor a significant firepower advantage on the part of the clans. Again though, only a few games were played.

What I notice more than anything since the clans came out is that LRM's are back with a vengeance, almost every clan mech on the battlefield packed at least one rack. Didn't seem to overly impact the outcome of the game though.

The ripple fire is really a very interesting mechanic. AMS really, really does eat them alive. Even 2 mechs with AMS and the overload module makes them borderline insignificant.

The trouble is that the clanners pack so much gun, and most IS builds are so tight that justifying the extra 1.5 tonnes is tough.

I wish there was a way to get some data on weapons and loadouts. I'd be able to have a crack at determining how much effect AMS has on all LRMs. There would have to be a lot of data though.

[1453 R]

Kiiyor, on 24 June 2014 - 04:45 PM, said:

The ripple fire is really a very interesting mechanic. AMS really, really does eat them alive. Even 2 mechs with AMS and the overload module makes them borderline insignificant.

Which is balanced out by the fact that Clan 'Mechs can load an LRm launcher and a ton or two of ammo onto a 'Mech damn near as an afterthought. My Warhawk-B loads thirty tubes of LRM and four tons of ammo as a secondary armament. Personally I love the ripple-fire mechanic and think we need to see more like that. The Clan launchers just feel better and are flat-out more fun to use.

Really though, guys...AMS. It's a thing you should take if you don't like the Clans. It's a thing you should take even if you do.

[Thiak]

dont forget that trial mechs are only IS

imho the lasers (range) are a bit overpowered the rest feels pretty balanced for me

Edited by Thiak, 24 June 2014 - 04:56 PM.

[Bartholomew bartholomew]

Thanks for making this. It seems closer than I expected. But matches some things I have seen.

However I think your small sample size is pretty close to the truth.

And most nerfherding OP whiners are just that.

[Levi Porphyrogenitus]

Caveat: I have no screenshots, everything I'm about to say is purely anecdotal and based solely on my flawed memory.

I play only IS. I have no plans to pick up any Clan mechs.

It's been my experience that IS mechs are easily able to match or exceed Clan performance. In all the games I've played since the Clan patch went live, I've only had a handful where I failed to get a kill, and nearly every match in which I did kill something at least one of the deceased was a Clan mech. Many of my matches I wound up at or near highest damage and kills, often beating out Clanners of all weights. I've done this in lights, mediums, heavies, and assaults.

Sure, maybe it's the fact that a lot of Clan mechs still aren't through Basic yet, let alone Mastered. Maybe it's that most Clan pilots are having a hard time adjusting to the burn times of their lasers, or to the burst-fire AC mechanics. Still, my overall impression so far has been that Clan mechs are very dangerous if you stand and take hits, but are far more vulnerable to damage-mitigating maneuvers than a comparable IS build. Many tend to be over-gunned, giving them great single-alpha potential but terrible combat durability (you're often better off dropping guns for more heat sinks or ammo, to maximize performance of a sustainable payload).

[Kiiyor]

Levi Porphyrogenitus, on 24 June 2014 - 05:20 PM, said:

Still, my overall impression so far has been that Clan mechs are very dangerous if you stand and take hits, but are far more vulnerable to damage-mitigating maneuvers than a comparable IS build. Many tend to be over-gunned, giving them great single-alpha potential but terrible combat durability (you're often better off dropping guns for more heat sinks or ammo, to maximize performance of a sustainable payload).

This is my experience also. IS poptarts or heavy alpha assaults in particular seem best able to match the Clans. It's the frontloaded damage.

The data seems to point to this too. The IS don't need to do as much damage to kill. I'm guessing IS pilots will need to start playing more cautiously to properly combat clan tech.

[BillyM]

Great data, but one thing to note is that it's likely that 3x as many IS mechs are elited as compared to the Clans in those numbers. Additionally, the pilots have been running the IS builds for 3x as long on average, EASILY.

...once these folks have a few weeks of use under their belt and fully skill-tree'd mechs, WATCHOUT!

--billyM

[Tombstoner]

Joseph Mallan, on 24 June 2014 - 04:22 PM, said:

Actually my point was that Clan Mechs are not tough enough yet. I have said it many times I want the Boogieman, I want to sweat when I see a Timber Wolf and above... I don't have that yet.

Agreed. but its very unlikely that much of anything will come down the road. I can see a few tweaks but over all Clans are looking to be in a good spot. The question then becomes what happens if IS goes 1-1 with clan in CW. I'm not convinced IS FLD will be a sufficient advantage when clans also have FLD weapons. I don't see splash damage being sufficient to mute clan FLD firepower. Disclaimer i don't run clan's yet but i have been on the receiving end of a number of T-wolves.

I see nothing wrong with this data and ~110 sample points is a reasonable starting point for assessing people in your elo.
sample size is dependent on desired precision and player population and that's basicly unknowable out side of PGI. they have the real numbers, lets hope the data base is free of bugs..

[Kiiyor]

BillyM, on 24 June 2014 - 05:34 PM, said:

Great data, but one thing to note is that it's likely that 3x as many IS mechs are elited as compared to the Clans in those numbers. Additionally, the pilots have been running the IS builds for 3x as long on average, EASILY.

...once these folks have a few weeks of use under their belt and fully skill-tree'd mechs, WATCHOUT!

--billyM

That's a good point. If anything, It would be interesting to run the data again in a week or two to see how much of an effect the eliting actually has.

[BillyM]

I can tell you right now, doubled basics makes an easy +10% effectiveness for me (10% less deaths, 10% more dmg, 10% more kills) and wouldn't be surprised to see that specifically on the direwolf most of all.

--billyM