Jump to content

Is Vs Clans - With Science!

Balance

237 replies to this topic

#1 Kiiyor

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Big Daddy
  • Big Daddy
  • 5,565 posts
  • LocationSCIENCE.

Posted 06 September 2014 - 05:35 AM

Posted Image

Data currency: 06/09/2014



PLEASE BE AWARE, THESE STATS ARE OVER A YEAR OLD!




FIRST UP, I would like to apologize in advance to scrollwheels everywhere.

So, it's not really Science, it's more of a questionable statistical analysis of a (probably too small) small sample size of matches.

Firstly, I've found the entertainment value present in all the CLANS ARE OP/CLANS ARE FINE threads to be almost as great as piloting these awesome new Clan mechs.

After a while though, I began to wish for some actual hard data to throw at people. "Kiiyor, you take screenshots of almost every EOM screen. You should collate that info!" the ice cube in my Scotch whispered to me the other night.

10 minutes later, having transferred barely half of one result to excel, my Scotch and I both agreed it was a job for future me. Also, being slightly inebriated, I was more interested in using the REPT function in Excel to make wang charts. HILARIOUS.

Anyhoo, after almost a full freaking day working out Photoshop batching, and with the help of a passable piece of free OCR software, I had the results from 110 matches stored in Excel. I posted the results of that first analysis in another thread.

Since then, the community has pitched in, and I now have around 1500 matches of data sitting in excel, waiting to be twisted into whatever dark shapes my mad mind desires.

The result? Some (still probably questionable) SCIENTIFIC ANALYSIS.


About the thread:

Many of the awesome pictures you see arrayed below you are from the spectacularly under-appreciated Lordred and Konniving's perfect screenshot thread.

Wait, this is a new thread? Why not edit the old one?
  • I changed the name of the last thread three times, breaking three sets of links to it. I figured I'd make one new thread, and never rename it.
  • The numbers discussed from the old thread have changed dramatically, because the sample size I have for pre 4x3 matches is 4 times higher than it was. Most of the discussion around those numbers is plain ol' wrong now.
  • I couldn't be bothered editing the old one.
Also, some of the old data was patently wrong, and a lot of discussion was based around those numbers. How wrong?




Spoiler


You may notice that the format of these numbers has changed from the previous thread, partly because I wanted to make things more legible, and partly because adding things to the old numbers broke excel spectacularly, corrupted the sheet and I had to pretty much start from scratch. Then, when I added a tournament score to all mechs after finishing with the new format, it exploded again. I'd saved often, but hadn't been saving with different file names. My rage was... incalculable. I wrote a letter to microsoft. Having re-read it, I wish i'd sent it under an assumed name. Anyhoo, the new version has 63 sequential files now though, and counting.

About the data:

Warning!!! Some caveats about the below data:
  • Though the sample size is far larger than it was previously, there are still outliers and anomalies that will influence the data, so please don't take these numbers as gospel. SERIOUSLY. At the most it’s an indication. But SCIENCE.
  • The mechs of those awesome and valiant pilots that contributed are included in many of the counts. This will undoubtedly have an effect on the data, though it won't be as pronounced as it was in the last thread.
  • Mixed battles... it's far harder to do a decent comparison when the samples are mixed. If there was full separation, each faction's strengths/flaws would probably become far more apparent. I managed to grab a few screens in the recent CLANvsIS tests, but not nearly enough to crunch numbers with.
  • OCR SOFTWARE! I used some text recognition software to gather the data. It had some trouble (sometimes) working out if a 6 was an 8, 3 or 9. I paid particular attention to this, but some errors may have slipped through.
  • As far as mech contributions are concerned, the data discusses damage/killin' and other exciting stuff mostly, and not utility (ECM, AMS etc).
  • The Excel worksheet behind this has also… suffered somewhat. It's lived through several mutations, and one or two repairs. It's possible that either it or I have screwed something up.
  • Please let me know if you find anything, and I'll correct it.
  • All the data is prior to the start of the clan nerfs. ERLL were still awesome in this sample.
  • The Data is also all prior to any Clan mech MC or CBILL releases.
  • I refer to chassis efficiency and viability in this data PURELY by kills/damage/death and destruction. There is no way to account for other useful factors in mechs. FOR EXAMPLE, Spiders and Locusts are at the bottom of most tables in terms of damage and kills, yet they can be invaluable for scouting, nuisance capping, breaking up teams (SQUIRREL! SQUIRREL!) and causing mayhem.
Now, THE RESULTS!




(Note, much of this is displayed as an image. I couldn't format them as text here to save myself. I have, however, furnished each image with small triangles in the upper right and lower left of each image, so you can see which parts you can quote, if you so desire.)

***Further note on this: Turns out the forums have an image limit. I blew through it spectacularly. As a result, some of the images I had separated had to be joined together.

Posted Image


My god, are there more numbers. There's a whole plethora of science below. I've had a few goes at separating it logically, as my first few attempts looked like a robot unicorn had vomited numbers all over an Excel spreadsheet. I've settled on the format below.

I've started with some general and interesting game data, followed by my list of REASONS and contributing factors to the method behind the analysis. Towards the bottom, I compare Clans and IS BATTLE-STAT numbers, and finally offer some winner-take-all matchups between the higher performing chassis from each weight class and each faction.

Let's start with some interesting numbers. Some of them get expanded in their own space later, others are left to languish here. ROLL PRE MOVIE CREDITS:

Posted Image
Posted Image
Posted Image
Posted Image
Posted Image

SO, wow. Matches aren't nearly as close as they were pre 4x3, and that unique player number is a doozy!

Onwards! Actually, before we hit the mind numbing monotony of the mad math below, though, a diversion, with something I hit upon that I like to call interesting thing #1:

Posted Image

While I was trying to think of some of the contributing factors for the figures my math produced, I hit upon the realization that the IS mechs have something the Clans don't…

Posted Image

(C) mechs? (C) mechs. In the name of Sun Tzu and dubious SCIENCE, I have collated a list of matches that had at least one (C) mech, and compared their performance to their non-trial brethren in said battles.

(NOTE, Standard Deviation (StdDev) is a measurement of variance. A higher number (relative to the average it's measured against) means that the numbers in the sample have wilder swings above/under the average). A lower number means more consistent figures make up the sample.

Posted Image
Posted Image

Interesting? Are you depressed about (C) mechs? Well, you may be happy to hear there are mitigating circumstances…

Posted Image

New guys! Do yourself a favour! DRIVE THE GIANT METAL PHALLUS OF DESTRUCTION. Avoid the ponderous DoomedLander. And the Atlas. And stop giving the FireStarter a bad name!

AAARGH! WHAT THE CRAP AM I LOOKING AT, KIIYOR?

Well, overall, the POINT is…

Any trial (C) mech that drops in your team will generally be less than 50% as effective as a non-trial IS mech. (Admittedly, this is MUCH better than I thought it would be).

Trial mechs seem to account for just over 10% of IS mech numbers. There's about a 75% chance one will drop with your team. There's an 80-freaking-2% chance that either your team or the enemy will have more (C) mechs than the other. I have no idea if the MM attempts to balance (C) mechs out somehow with some of that dark ELO sorcery, and it's probably impossible to draw a concrete conclusion from these figures, but it seems that the (C) would definitely have an impact on the overall IS vs CLAN effectiveness debate.

HOWEVS… If you cornered me and forced me to pluck a number from the nether, I could try and math:

Posted Image

= the average (C) mech lowering team efficiency by around 7%. The numbers are diluted a little with more non(C)'s, being about a half percent increase in overall performance per extra vanilla IS mech. In smaller IS teams the effect is greater, being about a 10% reduction if one out of 5 IS mechs is (C), and 12% if one out of 4 is (C). Yikes.


This isn't a huge figure, but it certainly makes the Clan numbers look better.

Finally, and EXTREMELY importantly:

I have seen quite a bit of angst thrown at new players. A little is fine, as it helps provoke the CHALLENGE ACCEPTED receptors in the hypothalamus.

I have, however, seen some truly vile things thrown into chat. New players are an easy target for all the frustration leaking out of your pores after a particularly frustrating defeat. We can't brutalise our new players! We NEED them! You can think carry harder, but keep it internal!

If you are worried about (C) mechs dragging you down, maybe find a decent starter thread on the forums, and throw that at them instead of vitriol. Something like this:

"Hi new guy - you should jump on the forums and go to the guides and strategies pages. There's some really good stuff there."

Or - "genericMWOfaction is recruiting - we take all skill levels, and can help you learn the ropes. Look for our thread on the Factions pages on the forums"

+++ END INTERMISSION +++

NOW, Interesting thing #2!

Posted Image

I have tracked disconnects, but decided not to report on them, as it's the end result that matters - 0 damage. Some disconnects occurred after players had already dished out some hurt. Some players were obviously distracted by something shiny, and failed to score a hit, without disconnecting.

Also… some of those 0 damage mechs would invariably have been some poor (probably light) pilots who crested a hill, and ran into either a Cataphract, DS, WarHawk, Daishi, MetaCat, Gauss Yager or any other freaking mech loaded for prehistoric mega bear, and watched in dismay as 4/5ths of their tonnage was SPLATTERED ALL OVER THE FREAKING GRIDSQUARE BEHIND THEM. Ahem.

In the end, It's all part of the same 0 damage crap sandwich.

Why are 0 dmg games important?

If you have a DC, or someone asleep at the controls, your team misses out on vital firepower, and a target for the enemy to fire at. This means that your team will very likely be doing less damage, and your enemies will have an easier time of killing. It also means that mechs on your side are not performing optimally, which is important for balanced statistics. ANYHOO:

Posted Image

So. The Clanners are fewer at the moment, but at least they can drag themselves out of bed. Originally, I thought the majority of these 0 damage games would have been legitimate disconnects (and I'm sure some are) but it appears that a larger portion of IS pilots are hitting the ""**** THIS NOISE"" button before the Clanners do.

Maybe they're leaving to drink space mead with the space vikings.


Posted Image
Below is the meat of the analysis of CLANS vs IS. This is where the numbers really start to weigh down the thread, especially with new addidions. It's also here that we start to look at the SCIENCE that tells tales of death and glory, of hateful contest between deadly foes: damage, kills, death rates, chassis efficiency, well designed mechs, deathtraps, fencing, fighting, torture, revenge, giants, monsters, chases, escapes, true love, miracles, everything.

Posted Image

There are numerous contributing factors involved with current mech/weapon/faction dominance. Disconnects and 0 damage mechs, (C) mechs, faction and mech class balance can all drastically affect the outcome of a battle. More Clan mechs = more winning. Also, very few matches had evenly balanced faction numbers per team per match. It's hard to predict how each side compares against each other without comparable faction numbers.

With this in mind, I've tried to eliminate enough of these factors to leave us with the most (reasonably) ideal and even conditions to assess faction balance, and have created a sample of matches that are bereft of (C) mechs, 0 damage mechs, and wild faction number imbalance. I've also created a sample of fair matches with 0 DMG and (C) mechs left in, for comparison between the two samples.

The idea with this sample is to look at situations where Clan and IS machinery can expect to perform evenly against each other, and see how close they actually get to doing so. THE NUMBERS:

Posted Image
Even = The number of matches with a 6 Clan and 6 IS mechs per team.
Close = The number of matches with an imbalance of ± 1 faction mech on EITHER TEAM.
ET CETERA.

Note that the count of each imbalance in the charts here do not include the count above (i.e. close match counts do not include even match counts), but the actual samples do.

The even battle count here, with matched numbers of hallowed IS machines and sleek and deadly Clan murder robots on each side, offers a much fairer assessment of faction balance, but is still burdened with (C) and 0 damage mechs fuzzying the reults. HENCE:

Posted Image
Yikes. Those (C) mechs are everywhere.
Posted Image


EGADS, my perfectly balanced comparison theorem has hit a snag, in the form of a puny and underdeveloped (yet tenacious) small sample size.

Even matches are out, there's only a handful of those. Curses. Close matches are also too small, expecially when broken down between pug and team. That leaves close(ish) matches to save the day. The sample size is probably still too small, but we won't let something as crucial as an effective sample size get in the way of SCIENCE.

Our second sample is therefore the Closeish no (C) no 0 DMG count of:


156 matches.
(57 pre 4x3, 99 post)


The comparison (control) sample of closeish matches, with (C) and 0 DMG mechs left in is:
784 matches.
(288 pre 4x3, 496 post)


We now have our samples, one comprehensive yet flawed, another scientifically sound (with REDUNDANCY!), yet threadbare.

LET'S MAKE THEM FIGHT EACH OTHER.


Posted Image

Gather 'round, intrepid pilots, and let us see how numerous each faction is, and verily, let us see if it really is the size of the fight in the mech that counts.

Posted Image

This tonnage imbalance would have a very large effect on the overall damage output of IS mechs. If the Clans are bringing 10% more tonnage to matches, it will place the more prevalent IS mediums at an even greater disadvantage. Let's see how this works with DMG/Tonne later in the show.

Posted Image

Here, you can see how faction numbers/match played out through the sample. The numbers on the left of the chart represent the number of faction mechs making up the entire match. The higher the blob, the more IS weighted the numbers are.

Posted Image

Matches were more closely faction balanced prior to 4x3, but that's likely due to the Clan numbers around that time being much higher than they are now post 4x3, where matches are dominated with IS numbers.

Posted Image

Let's get ready to rrrrrrumble!

Here's the meat of the analysis. Here, I've collected some interesting stats on damage, kills and all their variations.

I'd initially listed kills and damage in separate sections, but the screen real estate used was getting ludicrous. Both figures are also better contextually if viewed together.

Things to keep in the back of your mind:
  • In the fair match sample, even with comparatively equal mech numbers, the Clans still have on average about an 10% tonnage advantage.
  • Generally, Clan weapons are DOT (damage over time) and do more damage than IS weapons, but not necessarily to the one location on an enemy mech.
  • IS weapons however, have higher FLD (front loaded damage). This means an IS mech is more likely to land lots of damage on a single component, which is a more efficient way to kill.

Do the two methodologies balance each other out? NO! Well, not quite anyway.

Now, i've had to wrap a LOT of the data below in spoilers, for sanity reasons. Data is mostly sorted starting with the broadest metrics, then working down to a more granular level, but I've decided to post what I believe to be the best metrics to indicate balance first:

Posted Image

DMG/mech and Kills/mech are to me the best indicators of mech performance. KDR is OK, but in reality, a death is of little consequence if you've slaughtered half the enemy team... or if you're powering down to preserve a ratio.

In a match with reasonably balanced faction numbers, these figures can give you an indication of how much a faction mech can be expected to contribute to a battle.

Note also that this sample has a comparison with data from the control sample, so you can see the effect (C) and 0 DMG mechs has on performance.

BEHOLD! THE RESULTS OF KIIYOR'S BALANCED MATCH THEOREM:

Posted Image
Post 4x3, IS DMG is lower, but more consistent. Clan damage is higher, but less consistent.


Control sample comparison (detail):
Spoiler


Posted Image
Already, that is much closer than the all match sample, but still unbalanced. Also, all IS classes bar Lights seem to have a sizeable disconnect with the performance of their Clan counterparts. Spoiler for the mech stats below though; it's all StormCrow, MadCat and DireWolf, majestic overachievers that they are.

Posted Image

Control sample PUG/team comparison (detail):
Spoiler


Posted Image

Check that out! Stats are almost identical between PUG's and teams here! Clan lights seem to suck a lot more in teams though.

Posted Image

There is a HUGE difference here from pre to post 4x3. Huge.


Control sample comparison (detail):
Spoiler


Posted Image

Well, aren't you the over-achievers, Clan assaults.

Posted Image

Control sample comparison (detail):
Spoiler


Posted Image

IS Lights, I bow to thee.

Apart from the IS lights, things are pretty close to Identical between teams and pugs. IS mediums are slightly better in teams, and IS heavies are better in PUGS. Clan Lights seem to be meat in teams, though.

Now, that's it for the fair match DMG/kill/mech sample. Some interesting stuff there. I'm spoilering the main topics from here on in, in a laughable attempt to compact this post.

Posted Image
Spoiler


Posted Image
Spoiler


Posted Image
Spoiler


Posted Image
Spoiler


Posted Image
Spoiler


Posted Image
Spoiler

Posted Image
Spoiler


Posted Image
Spoiler


Posted Image

Spoiler


PHEW.

How 'bout them numbers? If you look at the balanced /mech numbers, you can see that balance is far closer than it is in the other samples, but the Clans still seem to have a definite advantage. MAIN POINTS:
  • IS mechs kill more efficiently, but this does not quite counter-balance the extra firepower in Clan mechs.
  • Clan lights are terrible at killing and damage. They have lots of utility in battle though, which these figures don't take into account.
  • Initially, I was prejudiced against IS mediums. In my mind, they were all garbage. If you look at kills/mech though, IS mediums are just as competitive as IS heavies and assaults. This is compounded by the fact that IS mediums have some very, very poor performers in their roster, and some of the best in the game.
  • Really, some of the best in the game. Further below are some mech totals, and the SHAWK in particular is one of the better performers in the whole 'sphere. The Griff also is pretty good, but for some reason I don't have a whole lot of drops from that in the sample to be sure about it's performance.
Before we start dissecting individual mech performance though, here's another diversion; a damage diversion. And a kills diversion.








Ever wondered how many mechs do less than 100 damage in a game? More than 1000? FEAST YOUR EYES:

Posted Image

Below is a collection of chartish type things that took far too long to put together. These abominations detail where mech damage ranges fall across the MWO mech population, and are quite interesting:

Spoiler


Posted Image

Same deal as above, but with Kills. Turns out if you kill more than 5 mechs in a game, you are in a VERY exclusive club.

Spoiler


Posted Image

IN THIS SECTION, we pit mech against mech, faction against faction, in a battle indicative of mech mastery, luck and sample size, as much as it is of overall chassis effectiveness. Who takes it? Who's lethality reigns supreme?

THERE CAN BE ONLY ONE! Overall. Also, one for each class. So there can ONLY BE 4. Or 8, really. Or more, depending on the filters I place in Excel. Read on.

Through the mystical sorcery of PivotTables, I've managed to find the top performers in each class, and will pit them against each other in numerical battle.

How did I sort to find these top performers? Primarily, Kills/mech. Some may argue that kills aren't the true indicator of performance, citing things like kill stealing etc, but as your sample size starts to increase, those sorts of cases become swallowed in the general truth of the data. There may indeed be better ways to indicate balance, but at the end of the day, it's the mech with his big metal foot planted on the chest of his fallen adversary that writes the chronicles of their glorious battle.

The numbers to be compared are:
  • DMG/Mech:
    • A measure of destructive power meted out. Not to be used on it's own, as high damage is not always indicative of overall lethality. An Ember tearing the heart out of an Atlas through it's back is arguably more lethal than the WarHawk missile boat that put fistfulls of missiles into every mech without chewing through to the fragile bits.
  • DMG/Kill:
    • Efficiency! Now we can see the whole 'efficient killing' vs 'overkill killing' debate thing in context.
  • Kills/mech:
    • Lethality! This stat describes how often a mech should be able to get kills, and is IMHO one of the better ways to judge mech goodness.
  • KDR:
    • Ah, the ePeen. Personally, I don't think it's a good balance stat, but then again I don't care If my virtual pilot is sitting in a functioning mech at the end of any given match.
    Add these stats together, and what do you get? NOTHING. Yep. I tried a few different scoring systems, including the difference in overall percentage of all the main stats averaged together, but it didn't feel right.
We do have another system though: the tournament score. I've managed to add it to all mechs (using their 10 best matches) but this was done after most of the results here were already colalted, and to be honest… I'm not a fan of the system, because it rewards outliers, and anomalies can have a huge effect on the results.






My idea…

Is to use the tournament score with the 10 results around the median - I.E. Pick the middle of the data and use that point, the 4 below it, and 5 above to do the tourney score. What do you guys think?

Caveats about the below data:
  • Mechs with less than 50 drops are exempt, because sample size.
  • Contributor mechs aren't in the sample (their performance was almost always over average).
  • 0 damage mechs are removed, as they throw out the average counts.
  • After this filtering, I found another problem. Some mechs seemed to be performing exceptionally well. For example, the first time I calculated the single best mech in the ENTIRE GAME, this happened:
Spoiler



Initially, I published it, fearing that not doing so would be inviting confirmation bias to kick me in the nads. I then looked closer at the data, and found that some mechs were predominately being piloted by the same people. This doesn't make for an effective sample, so any mechs with less than 30% unique players were also exempt.
THAT'S IT. The data isn't filtered in any other way.


Now… about the team sample. Team matches are an ENTIRELY different beast to PUGS. In team play, the mechs that run well seem to be whatever mechs the skilled players WANT to run well. Keep that in your mind as you look at the team figures.

Battle of the LIGHTS (PUG):


Let's start with LIGHTS. In my mind, it was a battle between either the Ember or JR7-F, and the Kitfox.

I was wrong!

Spoiler


Battle of the LIGHTS (TEAM):

How would the lights fare in teamville? My thoughts were Ember and Kitfox. Wrong again!

Spoiler


Battle of the MEDIUMS (PUG):

I surveyed some mates. "ShadowHawk and StormCrow", they said.

Spoiler


Battle of the MEDIUMS (TEAM):
Spoiler


Battle of the HEAVIES (PUG):

MadCat and the 'Phract, was the general consensus. We were half right:

Spoiler


Battle of the HEAVIES (TEAM):

MadCat and the 'Phract, DEFINITELY, was the general consensus. Again… wrong. OH HOW WRONG. For a while, it was the QUICKDRAW (and WELL DONE Eric and Pizza). I then instigated my purge the unclean rule, and it became….

Spoiler


Battle of the ASSAULTS (PUG):

In my mind, this was a battle pre-ordained, that would only be fought by the Stalker and the DireWolf (though having seen my last batch of data, I thought the WarHawk was in with a chance):

Spoiler


Battle of the ASSAULTS (TEAM):

Here, I though DireWolf (hands down) and the DS. It turns out that FATTIES can contribute too.

Spoiler


WELL. I bet many of you are as surprised as a drunk man in his first Thai go-go bar.

Now that class warfare is wrapped up, let's look at some...

Exhibition fights!

Let's experiment! SCIENCE!

Battle of the META (PUG):

So, what about the 2 mechs generally perceived to be the pinnacle of mech meta?

Spoiler


Battle of the META (TEAM):

ARE dragons more powerful in packs, with experienced handlers?

Spoiler


THERE CAN BE ONLY ONE! ALL matches:

So, which mechs are favoured over all others by the data? PUG AND TEAM COMBINED! Who rises above?!?!?!?

Also, I'm putting a 150 mech minimum on this one, to try and water down outliers.

Spoiler


THERE CAN BE ONLY ONE! PUG matches:

Ok, let's see how things differ at the top for PUGS and TEAMS. This one... will surprise you. I'd suggest that I need a more data, but the IS mech has a surprisingly robust sample.

Spoiler


THERE CAN BE ONLY ONE! TEAM matches:

I've had to lower the count of drops in teams. There isn't enough data, otherwise.

Spoiler


TOP VS BOTTOM! (PUG):

BEST MECH VS WORST MECH. Who can it beeeee now? CROW, vs Locust I reckon'. 50 match cutoff.

Spoiler


TOP VS BOTTOM! (TEAM):

Hard to pick, as there simply weren't enough variant drops. 30 match cutoff:

Spoiler


That's it. Awesome, hey?

AGAIN, WHY IS THE TEAM DATA SO DIFFERENT?

Well, it appears to be a combination of a few things.

Firstly, the team queues have far less unique players than the PUG stomping grounds. This means the same players are using the same comfortable mechs, over and over.

SECONDLY, I recognized a lot of decent players consistently in the team queue. Whatever they run often does well.


Now, using these numbers to argue balance is all well and good, but there are thousands of variables that would need to be in place before something like this can happen. Think about HOW these mechs lay down their damage; Is the Yager as versatile as the TWOLF? No. It's support. It's hitboxes don't let it take damage, but if it's not being targetted, it can be devestating. The Yager needs to be played to it's advantages to win. It's niche is support. The Twolf is always playing to it's advantage. It has found a niche in everything!

Also, being that these mechs aren't actually grouped together in a match, their scores come from them in their perfect environment.

Well, that's the individual mechs sorted out. Looking at these scores though, made me wonder about exactly how much of an impact crap IS mechs are having on the reputation of the 'Sphere as a whole.

What would happen if we put the top scoring mechs in each bracket in a fight against each other? NO IDEA.

On PAPER though...

INTERESTING THING #4. GOOD MECHS VS GOOD MECHS!

PUG CAGE MATCH!

Spoiler


TEAM CAGE MATCH!

Spoiler


Posted Image


EGADS, PIVOT TABLES.

Here's some tabular data for all mechs from the sample. Before you view it, you should note that the same caveats for the Battle Royale appear here:
  • Mechs with less than 50 drops are exempt, because sample size.
  • Contributor mechs aren't in the sample (their performance was almost always over average).
  • 0 damage mechs are removed, as they throw out the average counts.
  • I found that some mechs were predominately being piloted by the same people. This doesn't make for an effective sample, so any mechs with less than 30% unique players were also exempt.
  • Mechs between 30-50% unique players are probably suspect also, but are included. Just FYI.
  • Exempt mechs have their numbers greyed out, and the offending column flagged in red.
  • Data is sorted by kills/mech.
Also, all data is post 4x3.








First up, overall chassis totals. Be aware that some chassis have spectacular variants, and terrible ones, so the ones near the top of the list are either consistent performers here, or have one AMAZING variant. Individual chassis data few flicks of the scrollwheel further down.


Post 4x3 PUG mech stats - chassis totals/averages:

Spoiler


Post 4x3 PUG mech stats - all variants:

Spoiler


Post 4x3 TEAM mech stats - chassis totals/averages:

Spoiler


Post 4x3 TEAM mech stats - all variants:

Spoiler


Posted Image


Well, we have all these numbers. What do they mean for balance? Let's postulate, from a purely numbers based standpoint, with BROAD, SWEEPING GENERALIZATIONS:

At the moment, if you get a match with sort of balanced faction numbers on both sides, Clan mechs are around 30-40% more effective than IS mechs as a whole.

There are REASONS for this, however:
  • (C) mechs and 0 DMG mechs (disconnects or otherwise) really punish the IS. Without them, the Clans are around 30% more effective.
HOWEVER…








In a (very hypothetical) match full of good clan mechs, and good IS mechs, the clans have around 1.5% more effectivenes (on paper). That's minute. In Teams, the IS are hypothetically better. It's all so very situational, and those numbers do not take into account what would actually happen if you put the mechs mentioned above into a match together. Speculation FTW.

Let's put the good mech titbit aside for the moment though, and look at that 30% difference.

If the Clans are 30% more effective than the IS. We just need to nerf their effectiveness by that amount, right?

WRONG!

More hypothetical! If a Clan mech does 10% less damage before it dies, there's a good chance this will allow an IS mech to do 10% more. An increase the TIME TO KILL (TTK) for one mech usually means a decrease for its opponent.

In reality, the balance point is probably halfway.

So, if the clans are 30% more effective, a 15% reduction would maybe allow the two sides to meet in the middle. Maybe.

/horrible_generalization

Also, the N word. To nerf all clan mech effectiveness by 15% is freaking huge. Enormous. Hordes of players frothing at the mouth demanding refunds huge! I don't think it's possible do nerf to that degree in one go. Maybe it will happen (if it happens) as it has started to now - little cuts here and there, over enough time to let them scab over before adding the next wound. Awesome analogy is awesome.

So how do we balance?

Well, it's tricky. You can't balance for perfect conditions if they never happen.

Should we give the Clans (C) mechs? Absolutely. That will be long in the future if it ever happens, though.

12 IS vs 10 CLAN? I am… actually leaning towards this. Thematically, the Clans have always had an advantage. Right now, they are probably OP, but they FEEL good. Look at the changes to the ERLL. Did it make the weapon appear more balanced? Probably. That 2 second duration though, made the weapon feel laughably weak.

If people are worried about stuff like KDR for less opponents, then make Clan mechs worth 1.1 kills a piece. Hell, i'd be happy with a complete kill overhaul that gave half kills. If you do more than 50% damage to a mech, and someone else gets the kill, you should be awarded half a kill each. Fighter pilots did it in WWII. Precedence, your honours!

OR... do we try and reserve judgement for now? I'm leaning most towards this. I'd like to keep running these numbers, and do another huge analysis AFTER all the Clan mechs are available for Cbills.

Posted Image

YE OLDE SUMMARY:

Well, that's (almost) it. Thank god. Here was where I was going to jump into a more detailed analysis of the data above, but by now even I was sick of reading what I had written. When I tried to compact the summation, I found that there were only a couple of points that really bothered me about all this clan vs IS stuff:

The most drastic and telling figure out of all of this, to me, is the match result change. Fights are far more one sided now. Crapper matches are crapper for everyone. How can this be?

I used to blame IS mediums, but not all IS mediums are bad; just the bad ones. Which is most of them. The non-bad ones are actually great. So what is it?

In my mind, there are a few possible reasons: (I’m spoilering this, because it’s not really a dot point any more. It has mutated).

Spoiler


Posted Image
  • Clan mechs, in your average matches, out damage and out kill IS mechs.
  • Clan mechs are performing better than they were pre-4x3, and the 'Sphere is doing worse. I think 4x3 and New players are partly to blame for this, as detailed in the big-worded bit above.
  • However…
  • The IS absolutely have mechs that can compete with Clan mechs.
  • The good clan mechs (Crow, Wolves) are very, very good. They're at the top, looking down in derision at everything else.
  • Good IS mechs are also very good.
  • The Clans have some under achievers. So do the IS. The trouble is, IS under achievers are LEGION, and bad Clan robots will decimate bad 'Sphere robots, any day of the week, playing with one hand tied behind their back whilst being half ****faced on pure, hallucinogenic Absinthe.
  • IS mechs kill with less damage (more efficiently).
  • The Clans straight up do more damage.
  • In a match between good Clan mechs and good IS mechs, these two figures almost balance each other out.
  • The hate on IS mediums appears to be entirely misplaced (for the good mediums, anyway). There doesn't appear to be any one IS weight class letting the team down, and I was very, very surprised at how closely each weight bracket performs in terms of kills/mech. The trouble is the Clan heavies and Assaults - they are going from strength to strength.
Sort of TL;DR: my humble opinion - elevator speech:


While I think the Clans have a definite advantage, I don't think it's as large as everyone believes. For the IS, there is a combination of BAD ROBOTS (to quote lord Vassago) and new IS players that are making the Clans look better than they are. When all the Clan mechs are released for CBills, they may very well have the same new player problems the 'Sphere currently has. Will players be patient enough for this? ...LOL - probably not.

OH GOD LET IT BE THE END:

That's really it now. Almost. If you've read all the post, I commend you, I really do. I put this together to allow my fellow pilots to have a numerical appreciation of the current state of balance, and to maybe put numbers to arguments some of you may have had.

If you see any mistakes in the data (and there very well could be) please let me know.

What I want from you:

EOM SCREENSHOTS.

Give them to me! Get into the habit of screening at the end of every match! If you can be bothered sticking around. CRUSH MY INBOX:

sendmemwojunk@hotmail.com

As an added extra to the warm, fuzzy feeling of contentment you are likely to get knowing you are furthering the cause of SCIENCE, any screens you send me will be lovingly torn from their moorings in their JPEG homes and forced to convert to text, before being stuffed into an Excel spreadsheet and SENT BACK TO YOU for your own dark amusement. I get more data, you get your data Awesomeafied by at least 34%. Everyone wins.

I will also never post your names and specific match data here, and will never provide your name data to others.

Error checking!
If you see any mistakes in the data (and there very well could be) please let me know.

Discussion!
But please, try and avoid the purer forms of vitriol. I'm all for passionate discussion, not uncontrollable fires. If you're going to become em-passioned, please cite some data!

Also, If you don't agree with the numbers or data, then send me EOM screenshots! The more data I have, the more truthful this SCIENCE will be.

Future stuff:

I'll leave the format as is now, and add new data in as I get it. I'll update the currency of the data at the top of the thread.

If there are drastic data changes, (especially to individual mech stats) I'll update the thread with said changes… somewhere.

I've already begun the process of adding timestamps to the data I have, so I'll eventually be able to plot changes over time. This will be VERY useful for seeing the effect of balance adjustments, if we can plot balance dates on charts. Plus, I love charts.

I'm going to add tournament score to some of the data, but I'm still toying with creating a different scoring type in addition to this, using the tournament system, but based around the median of the data, rather than the very pinnacle.

I'm also taking requests!
If you want data crunched in specific ways, let me know, and I'll try to accommodate you.





So there you have it, my pretties - you now have some numbers to arm yourselves with! Time to use them to support your own barely formed arguments, twist them to your own desires and belligerently stick to your guns even if proven wrong!

I need a drink.


EDIT: 07/09/14 - Added the correct overall mech stat tables - the initial ones were missing the count of unique players in each mech.

Edited by Kiiyor, 11 October 2015 - 02:28 AM.


#2 El Bandito

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • Big Daddy
  • Big Daddy
  • 26,736 posts
  • LocationStill doing ungodly amount of damage, but with more accuracy.

Posted 06 September 2014 - 06:18 AM

Yep, Clan mechs are better, as any sane person can see. Two of the four best IS mechs that can do well against respective Clan class are Hero Mechs--which is MC only, kinda making it pay to...you know the drill.

View PostKiiyor, on 06 September 2014 - 05:35 AM, said:

Now, using these numbers to argue balance is all well and good, but there are thousands of variables that would need to be in place before something like this can happen. Think about HOW these mechs lay down their damage; Is the Yager as versatile as the TWOLF? No. It's support. It's hitboxes don't let it take damage, but if it's not being targetted, it can be devestating. The Yager needs to be played to it's advantages to win. It's niche is support. The Twolf is always playing to it's advantage. It has found a niche in everything!


This point is important. The best performing IS Heavy, the FB, (or the Jager in general) are only limited to few niche roles and only preforms well when no one is looking at his direction, while the Timbie is a jack of all and master of all.

View PostKiiyor, on 06 September 2014 - 05:35 AM, said:

The Clans have some under achievers. So do the IS. The trouble is, IS under achievers are LEGION, and bad Clan robots will decimate bad 'Sphere robots, any day of the week, playing with one hand tied behind their back whilst being half ****faced on pure, hallucinogenic Absinthe.


Also this point. Can't believe there is a person in the forums who thought RVN-2X or SDR-5V can have a real chance against the Kit Fox or the Adder and made such argument about it against me.

Edited by El Bandito, 06 September 2014 - 06:27 AM.


#3 SgtKinCaiD

    Member

  • PipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 1,096 posts
  • LocationBordeaux

Posted 06 September 2014 - 06:20 AM

Man, you're great !

#4 Kiiyor

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Big Daddy
  • Big Daddy
  • 5,565 posts
  • LocationSCIENCE.

Posted 06 September 2014 - 06:32 AM

View PostEl Bandito, on 06 September 2014 - 06:18 AM, said:

Yep, Clan mechs are better, as any sane person can see. Two of the four best IS mechs that can do well against respective Clan class are Hero Mechs--which is MC only, kinda making it pay to...you know the drill.



This point is important. The best performing IS Heavy, the FB, (or the Jager in general) are only limited to few niche roles and only preforms well when no one is looking at his direction, while the Timbie is a jack of all and master of all.



Also this point. Can't believe there is a person in the forums who thought RVN-2X or SDR-5V can have a real chance against the Kit Fox or the Adder and made such argument about it against me.


Oh yeah, the Misery and Ember surprised me, expecially the Ember, though I will say that I think the Banshee will rank a lot higher once I get more data for it.

There was lots of stuff that opened my eyes, especially where IS mediums were concerned. I thought they were the worst IS performers, turns out they are awesome.

I really do think part of the problem is BAD IS MECHS. It's not easy to argue from a balance standpoint though, as they exist, they will drop in matches, and they will (most likely) die ineffectually, and there probably isn't a lot we can do about it.

#5 LaserAngel

    Member

  • PipPipPipPipPipPipPip
  • The Angel
  • The Angel
  • 889 posts

Posted 06 September 2014 - 06:50 AM

I would also like to thank Ice, Dude, and Slug.

#6 Varik Ronain

    Member

  • PipPipPipPipPipPip
  • 219 posts

Posted 06 September 2014 - 07:18 AM


Kiiyor so how can I record data and send it to you to help increase the sample size?


#7 Ultimax

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 6,979 posts

Posted 06 September 2014 - 07:20 AM

Players might lament the Jager, but I still love mine.

The Jager only has one role, and that role is removing targets from the field efficiently. (Hint: this is a good role to have).

It's also TEN TONS lighter than the TBR, most builds run "delicate IS XL engines" - yet the mech still performs.

My theory?

Take a look at Kiiyor's line up of the bestest from the IS:

Misery: High mounted hard points.
Jager: High mounted hardpoints.
Shadowhawk: High mounted hardpoints.
Ember: Ridiculously fast, hard to pin down, supreme battlefield opportunist.


Most interesting, but not surprising to me in the least.




To Kiiyor,

This thread & your effort is amazing, I've sent a tweet to Russ & Paul saying you deserve...something...


One thing that I see as a tough to deal with snag is that a DWF-Prime can be nearly identical to a DWF-B, the same goes for all of the Clan mechs and moreso for the ones that do not have unique CTs.

So while I'm unsure how you could deal with this, I also think having say "TBR-S" split off from "TBR-Prime" in results doesn't really tell the whole story since they could basically be clone builds.

#8 Jonny Taco

    Member

  • PipPipPipPipPipPipPip
  • Bridesmaid
  • 706 posts
  • Locationan island

Posted 06 September 2014 - 07:22 AM

"12 IS vs 10 CLAN? I am… actually leaning towards this. Thematically, the Clans have always had an advantage. Right now, they are probably OP, but they FEEL good. Look at the changes to the ERLL. Did it make the weapon appear more balanced? Probably. That 2 second duration though, made the weapon feel laughably weak."

Very nice to see that your extremely detailed analysis has lead you "at least somewhat" to this conclusion.

Furthermore, your comparison of good is vs good clan teams shows exactly what many of us who have tested them extensively w/o plebs involved have been saying for a while.

Thanks again for the thread/post, you've certainly earned much of this often hostile and polarized community respect with your posts over the past few months.

Edited by lartfor, 06 September 2014 - 07:23 AM.


#9 Kiiyor

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Big Daddy
  • Big Daddy
  • 5,565 posts
  • LocationSCIENCE.

Posted 06 September 2014 - 07:22 AM

View PostVarik Ronain, on 06 September 2014 - 07:18 AM, said:


Kiiyor so how can I record data and send it to you to help increase the sample size?


Take screen captures of your end of match results. Send them to me at

sendmemwojunk@hotmail.com

I've got a photoshop batch set up that edits and crops the images, then dumps them into OCR text extraction software. I'll even send you your match data in an Excel spreadsheet at the end!

#10 Varik Ronain

    Member

  • PipPipPipPipPipPip
  • 219 posts

Posted 06 September 2014 - 07:23 AM

View PostKiiyor, on 06 September 2014 - 07:22 AM, said:


Take screen captures of your end of match results. Send them to me at

sendmemwojunk@hotmail.com

I've got a photoshop batch set up that edits and crops the images, then dumps them into OCR text extraction software. I'll even send you your match data in an Excel spreadsheet at the end!


Turns out after removing the clutter it was 185 ;)


edit: numbers

Edited by Varik Ronain, 06 September 2014 - 07:45 AM.


#11 Ultimax

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 6,979 posts

Posted 06 September 2014 - 07:23 AM

View PostKiiyor, on 06 September 2014 - 06:32 AM, said:

I really do think part of the problem is BAD IS MECHS. It's not easy to argue from a balance standpoint though, as they exist, they will drop in matches, and they will (most likely) die ineffectually, and there probably isn't a lot we can do about it.


Unfortunately I have been saying that all Clan Teams:

1) Will likely consistently out ton IS teams.
2) IF 3/3/3/3 then are likely to have at least 1 KFX for ECM.
3) IS has too many crapmechs dragging down the top performers.
4) Clan team speed will be more uniform, aside from the Dire Wolf who will get there when he damn well pleases and kill several mechs when he does (or he will be killed at his spawnpoint in River City skirmish).


There are a lot of posters with some of the worst confirmation bias and agenda mongering I have seen in any game (I've played a lot of games, that contained a lot of unsavory internet trolls).

Edited by Ultimatum X, 06 September 2014 - 07:25 AM.


#12 Tkhaw

    Member

  • PipPipPipPipPipPip
  • Moderate Giver
  • Moderate Giver
  • 264 posts

Posted 06 September 2014 - 07:32 AM

View PostVarik Ronain, on 06 September 2014 - 07:18 AM, said:


Kiiyor so how can I record data and send it to you to help increase the sample size?



Same here.

I really salute your efforts in getting the data out in statistics, tables and charts.

Most other clan vs IS threads were to me, anecdotal at best.

Keep it up!

#13 Tkhaw

    Member

  • PipPipPipPipPipPip
  • Moderate Giver
  • Moderate Giver
  • 264 posts

Posted 06 September 2014 - 07:44 AM

View PostKiiyor, on 06 September 2014 - 07:22 AM, said:


Take screen captures of your end of match results. Send them to me at

sendmemwojunk@hotmail.com

I've got a photoshop batch set up that edits and crops the images, then dumps them into OCR text extraction software. I'll even send you your match data in an Excel spreadsheet at the end!



I am actually worried about bias,

what do I need to do in order to avoid bias?

#14 Scurry

    Member

  • PipPipPipPipPipPip
  • Bridesmaid
  • 375 posts

Posted 06 September 2014 - 07:51 AM

If anything, I'd say mediums are competitive because of their reputation. People always aim for the heavies or assaults to take them out first - but ignore the mediums. However, the mediums can still do plenty of damage while people aren't shooting at them.

The Stormcrow is at an amazing sweet spot when it comes to weapons tonnage/speed/armor. Heavy class weaponry, superior speed, decent armour, all in a somewhat awkward-looking package that is weird to hit, cockpit-level torso hardpoints (WYSIWYS), medium class "awareness filter" and the weird bobbing animation. If the bloody thing had jump jets, it would be a no-contest.

Now I'm going to have trouble bringing myself to use my Stormcrow since it's apparently so horribly superior. *sigh* And I was finding it so much easier to grind using it too.....

#15 Kiiyor

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Big Daddy
  • Big Daddy
  • 5,565 posts
  • LocationSCIENCE.

Posted 06 September 2014 - 07:52 AM

View PostTkhaw, on 06 September 2014 - 07:44 AM, said:



I am actually worried about bias,

what do I need to do in order to avoid bias?


Not much - i'm relying on honesty a bit for this one. Just screen as many as you can, no matter how you perform. It can be hard sometimes, especially if you die early and then have to sit and watch mech hide and freaking seek if one chump powers down to preserve their KDR!

#16 Tkhaw

    Member

  • PipPipPipPipPipPip
  • Moderate Giver
  • Moderate Giver
  • 264 posts

Posted 06 September 2014 - 07:54 AM

View PostKiiyor, on 06 September 2014 - 07:52 AM, said:


Not much - i'm relying on honesty a bit for this one. Just screen as many as you can, no matter how you perform. It can be hard sometimes, especially if you die early and then have to sit and watch mech hide and freaking seek if one chump powers down to preserve their KDR!


A reason I asked was because most of the screenshots I have are those that I performed well in :)

#17 Varik Ronain

    Member

  • PipPipPipPipPipPip
  • 219 posts

Posted 06 September 2014 - 07:58 AM

View PostKiiyor, on 06 September 2014 - 07:52 AM, said:


Not much - i'm relying on honesty a bit for this one. Just screen as many as you can, no matter how you perform. It can be hard sometimes, especially if you die early and then have to sit and watch mech hide and freaking seek if one chump powers down to preserve their KDR!


Oh dont worry buddy ;) I sent the good with the bad.. plenty of horrible matches in there where I was on the bad side of a roflstomp. I just finished sending 4 batches of screenies.

Edited by Varik Ronain, 06 September 2014 - 07:59 AM.


#18 Carrie Harder

    Clone

  • PipPipPipPipPipPipPip
  • 678 posts
  • LocationCarrying pugs up Mount Tryhard

Posted 06 September 2014 - 08:00 AM

Shadow Hawk represent!

#19 Tkhaw

    Member

  • PipPipPipPipPipPip
  • Moderate Giver
  • Moderate Giver
  • 264 posts

Posted 06 September 2014 - 08:04 AM

Sent about 24 that I had.

I will send more next time :)

#20 Bishop Steiner

    ForumWarrior

  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • The Hammer
  • The Hammer
  • 47,187 posts
  • Locationclimbing Mt Tryhard, one smoldering Meta-Mech corpse at a time

Posted 06 September 2014 - 08:21 AM

View PostUltimatum X, on 06 September 2014 - 07:20 AM, said:

Players might lament the Jager, but I still love mine.

The Jager only has one role, and that role is removing targets from the field efficiently. (Hint: this is a good role to have).

It's also TEN TONS lighter than the TBR, most builds run "delicate IS XL engines" - yet the mech still performs.

My theory?

Take a look at Kiiyor's line up of the bestest from the IS:

Misery: High mounted hard points.
Jager: High mounted hardpoints.
Shadowhawk: High mounted hardpoints.
Ember: Ridiculously fast, hard to pin down, supreme battlefield opportunist.


Most interesting, but not surprising to me in the least.




To Kiiyor,

This thread & your effort is amazing, I've sent a tweet to Russ & Paul saying you deserve...something...


One thing that I see as a tough to deal with snag is that a DWF-Prime can be nearly identical to a DWF-B, the same goes for all of the Clan mechs and moreso for the ones that do not have unique CTs.

So while I'm unsure how you could deal with this, I also think having say "TBR-S" split off from "TBR-Prime" in results doesn't really tell the whole story since they could basically be clone builds.

don't forget crit factor for the Ember.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users