312 replies to this topic

[Mystere]

bluepiglet, on 17 July 2014 - 05:10 AM, said:

As you wise then, I believe your maths is capable enough to see obvious. Yeah, that's how underpowered Clan meches are, particularly the Light Class right?

Ahem. Read my response again. You're discounting 25% of available Clan mechs from a Clan vs IS analysis. That was my point and nothing more.

And if you think those numbers you showed tell the whole story, especially considering how Kiiyor is doing things, you have a whole lot to learn when it comes to scientific analyses.

So try again.

Edited by Mystere, 17 July 2014 - 05:23 AM.

[Kiiyor]

Viktor Drake, on 17 July 2014 - 04:42 AM, said:

I like the study but your conclusions are a bit skewed.

What I see is that more clan mechs at a higher tonnage produce higher damage and kills. Seems logical to me.

Clans have a 2% numerical advantage overall but also a 7% tonnage advantage ( more mechs + more tonnage = more weapons = more damage).

This 2% numerical and 7% tonnage advantage produces roughly a 8% advantage in total damage and 6.5% advantage in total kills.

Seems pretty damn balanced to me so I have no clue how your coming up with the clans having roughly a 10% total advantage across the board.

It's situational. I've had a crack at finding a perfect number of matches (same # lights/medium/heavy/assault) and similar tonnage, but it just doesn't happen. I think there were 2 matches out of the whole sample that ran into that. The trouble is the huge saturation of MadCats.

You'll notice that when I removed the 0 damage mech matches, the numbers were a lot closer, but I'm doubting myself because of the small sample size.

The truth is, it's impossible to get perfect data without proper Clan vs IS battles.

Aaaactually, you know what I didn't add with this one? DMG/KILLS/Tonne. It was in the last one, why the heck did I leave it out of this??? To the EXCEL machine.

[Kiiyor]

Charons Little Helper, on 17 July 2014 - 04:16 AM, said:

Of note - while I appreciate the #'s - the whole thing is somewhat invalidated by the existence of Elo. (yes it does work - no it doesn't work perfectly)

If a clan heavy mech causes you to win significantly - it'll bump up your Elo. Therefore, even if the Clan heavy is better, Clan heavy mech pilots will be pushed up in Elo to go up against IS pilots who're more skilled, canceling out much of the statistical differences you're trying to peg.

True, but there's no way these numbers can pull out such intricacies, and they'll always be limited like that. I will say though, that the sorcery of ELO tends to be balanced across a team (apparently), so you'll usually have samples from many ELO brackets in one match.

While my ELO experience will have an effect on the data, I can't imagine my experience (or any of the other 6/7 contributors) being so incredibly different. It's part of the reason i'm keen for more data, so I can dilute the ELO issues somewhat.

This was really never intended to be a thesis, or something that could be printed and waved in people's faces as IRREFUTABLE PROOF AAARGH ZOMG of one faction's superiority, but more of something to provide a general overview of performance, within the constraints of the data available to me.

[Haipyng]

Well done Kiiyor! That was a lot of work to put together. Thank you. It would be nice if PGI gave you direct numbers on a wide sample range to analyze for the community.

I am guessing these are solo drop numbers. I'd be really interested to see what happens currently in group MM drops.

Edited by Haipyng, 17 July 2014 - 05:55 AM.

[Goose of Prey]

Kliyor I have a question for you.


Do you keep track of ECM use in any of the matches? I think the IS is holding it's own now due to its ECM advantage, and not necessary good balancing by PGI. Just interested in your opinion on the matter as you have been doing a great job on gathering data. Thanks.

[Macksheen]

Love the later analysis too.

Doesn't surprise me - Stormcrow has been for me the best of the clan mechs thus far, w/ the Timber Wolf second. It makes me sad for my IS mediums, except for my SHD and GRF ... and possibly Blackjack.

[cleghorn6]

This is beautiful work, well done. I take screenshots of all my EoM screens, I'll upload them somewhere and send you a link.

How do you pull the numbers out? I'd love to do some stats on my results but spending a week in front of Excel is not my idea of fun.

[Kiiyor]

Goose of Prey, on 17 July 2014 - 05:55 AM, said:

Kliyor I have a question for you.


Do you keep track of ECM use in any of the matches? I think the IS is holding it's own now due to its ECM advantage, and not necessary good balancing by PGI. Just interested in your opinion on the matter as you have been doing a great job on gathering data. Thanks.

I don't - I started to, really I did - I also tried to attribute a primary weapon designation to people I noticed (LURM BOAT/******* POPTART) etc, but after just 2 matches I thought BUGGER THAT.

If you were in an assumptiony mood, you could say that all DDCs mount ECM, etc.

Hmmm.... maybe I could look at something like "% chance of winning with this mech"

[Angel of Annihilation]

Kiiyor, on 17 July 2014 - 05:27 AM, said:

It's situational. I've had a crack at finding a perfect number of matches (same # lights/medium/heavy/assault) and similar tonnage, but it just doesn't happen. I think there were 2 matches out of the whole sample that ran into that. The trouble is the huge saturation of MadCats.

You'll notice that when I removed the 0 damage mech matches, the numbers were a lot closer, but I'm doubting myself because of the small sample size.

The truth is, it's impossible to get perfect data without proper Clan vs IS battles.

Aaaactually, you know what I didn't add with this one? DMG/KILLS/Tonne. It was in the last one, why the heck did I leave it out of this??? To the EXCEL machine.

Thanks for your response. Yes you are right that it is going to be impossible to get a perfect test result because only PGI has access to the necessary resources. Still you have done an excellent job.

However I just wanted to make sure to point out how the larger number of clan mechs, combined with that fact that Clanner are leaning toward heavier chassis is going to account for a significant amount of that 10% overall superiority in terms of kills and damage output.

Definitely curious to see those numbers you forgot to add because I think we will see that overall the Clan and IS mechs coming up very balanced.

[Ted Wayz]

Kiyor, PGI keeps data by weapon type. If players would submit their archived stats to you would you be willing to crunch the numbers? Be great to see what weapon systems are leading to the clan advantage in data, although I suspect the answer is lasers.

[Ted Wayz]

And I hate that the mechs would be balanced. I would rather see 2 clan lances (10) vs 4 IS lances (16) once CW is implemented. Maybe a separate area for CW from the current arcade?

[Kiiyor]

cleghorn6, on 17 July 2014 - 06:16 AM, said:

This is beautiful work, well done. I take screenshots of all my EoM screens, I'll upload them somewhere and send you a link.

How do you pull the numbers out? I'd love to do some stats on my results but spending a week in front of Excel is not my idea of fun.

I first run the screenshots through a batch job I have in Photoshop, that crops all the crap away from the numbers, and shifts them closer together a bit, while removing a lot of colour to make things stand out better in contrast.

I then use a fairly premier piece of OCR software, that I've got on my work machine. It can extract to an Excel table - you can even specify where the gridlines will be, which is handy for Excel. These can be batched also.

The big issue is that the while the OCR is amazing at numbers, it doesn't get it right all the time for mech names:



... So I have to edit those by hand. It sucked for a start, but I'm old hat at data entry, so it ended up pretty painless.

Then I found out you could specify what font to look for in my OCR. I added the MWO font, and now it works like a charm.

Viktor Drake, on 17 July 2014 - 06:16 AM, said:

Thanks for your response. Yes you are right that it is going to be impossible to get a perfect test result because only PGI has access to the necessary resources. Still you have done an excellent job.

However I just wanted to make sure to point out how the larger number of clan mechs, combined with that fact that Clanner are leaning toward heavier chassis is going to account for a significant amount of that 10% overall superiority in terms of kills and damage output.

Definitely curious to see those numbers you forgot to add because I think we will see that overall the Clan and IS mechs coming up very balanced.

I'm keen to see how 3/3/3/3 will affect this also. I'm predicting that the Cats will still be everywhere, but maybe not as significantly as they are now.

[Xeno Phalcon]

Ah yes, Science.



(But seriously, these forums need more Mentats)

[Kiiyor]

Ted Wayz, on 17 July 2014 - 06:21 AM, said:

Kiyor, PGI keeps data by weapon type. If players would submit their archived stats to you would you be willing to crunch the numbers? Be great to see what weapon systems are leading to the clan advantage in data, although I suspect the answer is lasers.

Yep, I would indeed. Maybe I could even correlate some form of efficiency based on battle performance also - but that's starting to sound like swirly analysis.

I've already got a spreadsheet for my own weapon data, and it's also happily missing all the missed shots from start of match test firing that I used to do before the last couple of stats resets

[Jonny Taco]

"Any future balance adjustments should be considered carefully, though I do believe that instead of nerfing the Clans, the IS should be buffed somewhat."

Nope, it should be 12v10 3 is lances, vs 2 clan stars...

[rageagainstthedyingofthelight]

Possible the best post ever. In part because the statistics bear out my pre-conceived notion that playing an IS mech right now is only slightly better than disconnecting.

[Cion]

Excellent post OP, thank you for taking the time to do this "for SCIENCE!!".

PGI should hire you for analysis and community outreach, or at the least give you a ton of MC.

[lpmagic]

Nice work here, thank you for taking the effort to put this information out there.

There are some mitigating factors to take into consideration, and they are not end all be all.

A large number of higher Elo players do play with Clan mechs, but much less so then IS as a goodly portion of higher Elo players are in the "competitive scene" and that particular set of players is not allowed (in competition) to utilize clan mechs. now by no means is this going to be a larger number of people then are available in this study, but I would suggest that this might sku things a bit as some of the subjectively "better players" drive IS quite a bit more then Clan and not really by choice. but because they need to continue to practice what they will be driving in competition. not the biggest factor by any means

Next, I would add that leaving (c) mechs and disco's in the equation would be potentially the best measure, as these are variables that are part of the vagaries of the game itself, weather everyone drove a clan mech or not, farmers will still farm and bad connections will still be bad connections, additionally, new players will always be involved in the equation until such point as the game stagnates enough to not draw new people in to the fold, IMHO these factors are a natural part of what you are doing.

I would also be interested in the difference between the "solo" queue and the group queue or if there is a variance.

I really love that you took the time and effort to do this, it is wonderful!!
Thank you!

[Sprouticus]

Some notes and thoughts:

1) Great work. THANK YOU. I will try to get you some screenies.
2) I think it would be awesome to see what the sample size would be in matches WITHOUT specific mechs. ie no timberwolves vs no timberwolves or no stormcrows vs no stormcrows. I am guessing that the size would be absurdly low for the two mentioned, but maybe if you got enough feedback you could look at a statistically significant sample.

**The idea being to compare numbers without mechs that are currently percieved as OP (TW and SC). **
3) I think assuming that all of mech X has ECM is fine for basic analysis. Who the heck brings an ECM mech to the party and does not bring ECM.
4) As menitoned above, the balancing factor of people in better mechs being pushed up in Elo and thus lowering the value of those mechs is real and should be in your caveats. There is no way to fix that of course, but yea.
5) I think the performance of the SUMMONER is actually the biggest surprise in your data. For as much as eveyrone complains, that is some decnet performance. That having been said, it might be coming from just a few dedicated summoner pilots which might skew the data.
6) The ERML may not be the root cause of the clan advantage as indicated, otherwise the Nova would be much higher in your lists IMO.
7) For the folks sayning that the ratio of good clan mechs to good IS mechs matters....No. That just means that PGI has gotten better at making good, balanced mechs. If you look at the mechs put out in the last year, almost none of them have been terrible, IS OR clan. Yes there are some bad IS mechs. Really bad ones. But if you look at the number of mechs taken, few people are using the crappy IS mechs.


Again, thanks so much for all your hard work. Excellent info.

[Kiiyor]

lpmagic, on 17 July 2014 - 07:19 AM, said:

Nice work here, thank you for taking the effort to put this information out there.

There are some mitigating factors to take into consideration, and they are not end all be all.

A large number of higher Elo players do play with Clan mechs, but much less so then IS as a goodly portion of higher Elo players are in the "competitive scene" and that particular set of players is not allowed (in competition) to utilize clan mechs. now by no means is this going to be a larger number of people then are available in this study, but I would suggest that this might sku things a bit as some of the subjectively "better players" drive IS quite a bit more then Clan and not really by choice. but because they need to continue to practice what they will be driving in competition. not the biggest factor by any means

Next, I would add that leaving (c) mechs and disco's in the equation would be potentially the best measure, as these are variables that are part of the vagaries of the game itself, weather everyone drove a clan mech or not, farmers will still farm and bad connections will still be bad connections, additionally, new players will always be involved in the equation until such point as the game stagnates enough to not draw new people in to the fold, IMHO these factors are a natural part of what you are doing.

I would also be interested in the difference between the "solo" queue and the group queue or if there is a variance.

I really love that you took the time and effort to do this, it is wonderful!!
Thank you!

At some stage I'll be adding everything back in, and tallying all match results also. I'm trying to avoid it until I've got all the kinks worked out though, because I didn't do the exhaustive double checking I normally would (working VERY late hours ATM, but I'm one of those bizarre people who finds data crunching cathartic). Normally I'd get one of our interns to go over it

Once I've tidied things up, I'll be happier to start filling the first page with kilometers of data. I might have to start using more spoiler tags.

The reason I threw in the data without outliers was to try and get a mech vs mech result. Now I'm worried people will drop into a match, see a (C) mech and go "well, I'm screwed. I'm gonna file this match under R".