29 replies to this topic

[DEMAX51]

Why do people assume that trained statisticians wouldn't know that they have to account for variations in player skill level during these tests?

I mean, c'mon guys - all they have to do is say "ok, the Clan teams had an average 100 point Elo advantage, which would give them a predicted win ratio of about 65% - it looks like the Clans are winning ~73% of the time, so they're still overperforming."

You don't think that's occurred to them?

Edit: And also, regarding people intentionally throwing matches in an effort to manipulate the data: this will probably result in that particular datum being labeled an outlier, and thrown out. I'm sure they're taking that into account as well.

Edited by DEMAX51, 23 September 2014 - 12:47 PM.

[Angel of Annihilation]

Kevjack, on 23 September 2014 - 11:41 AM, said:

Why is there this belief that everyone who bought the Clan packs with real money are expert, pro players?

It is the investment amount. Typically someone brand new to a game just wanting to test it out or someone who is a casual player isn't going to invest up to $240 in mechs. Also casual players just by virtue of having alot less practice than dedicated players will typically not be as skillful at playing, practice makes perfect after all.

Therefore it is pretty obvious that those players willing to invest significant amounts into Clan mechs, are, on average, going to possess a higher skill level than those who have not invested into Clan mechs.

[Angel of Annihilation]

DEMAX51, on 23 September 2014 - 11:57 AM, said:

Why do people assume that trainedwouldn't know that they have to account for variations in player skill level during these tests?

I mean, c'mon guys - all they have to do is say "ok, the Clan teams had an average 100 point Elo advantage, which would give them a predicted win ratio of about 65% - it looks like the Clans are winning ~73% of the time, so they're still overperforming."

You don't think that's occurred to them?

Edit: And also, regarding people intentionally throwing matches in an effort to manipulate the data: this will probably result in that particular datum being labeled an outlier, and thrown out. I'm sure they're taking that into account as well.

Your assuming alot. I honestly, sincerely doubt that PGI has a dedicated statistician being paid $50-60k a year just to plan out how to balance clan mechs. Most likely they have some programmer who may have taken a statistics class or two in college and is mathematically inclined looking at the numbers and making a "judgement call" based on his observations.

[FupDup]

One of my concerns about the tests is that we aren't getting a breakdown of chassis/variant/weapon contribution. All we're told is just % of wins. Okay, so they won more often, but what CAUSED those victories? PGI might interpret those numbers as "clearly the LB 2-X and Adder caused the 73% win rate!" whereas in reality it would probably be due to things like Mad Cats. Just a blanket statement of how often they won isn't nearly detailed enough to understand the true state of the factions. We need to see how much each individual mech/weapon CONTRIBUTED to that win rate.

[Mazerius]

Kevjack, on 23 September 2014 - 11:41 AM, said:

Why is there this belief that everyone who bought the Clan packs with real money are expert, pro players?

Misconception really, there seems to be a large group of folks that were waiting for the clans to be released before buying or even playing that much, at least thats the feel I get from the pug queue, seems at times the guy in that shiny Timber wolf is just as likely to leeroy as the guy in the mech with the (C)

[DEMAX51]

Viktor Drake, on 23 September 2014 - 01:01 PM, said:

Your assuming alot. I honestly, sincerely doubt that PGI has a dedicated statistician being paid $50-60k a year just to plan out how to balance clan mechs. Most likely they have some programmer who may have taken a statistics class or two in college and is mathematically inclined looking at the numbers and making a "judgement call" based on his observations.

"Dedicated Statistician" may be the wrong term - perhaps I should have used the phrasing "someone on staff more highly trained and skilled in data analytics than the average forumite."

Still, I really wouldn't be at all surprised if they had someone on staff whose background was very focused on statistical analysis. Would this person be solely tasked with analyzing Clan balance? No, most likely not. But they'd probably run analytics on game balance as a whole, in addition to PGI's business performance. I find it very hard to imagine that they don't have a staffer on hand to analyze their sales numbers, player retention numbers, free-to-play players to paying-players conversion rate, marketing success rates, and a ton of other stats I'm not familiar enough with the business world to name.

All I'm saying is that the people working on this aspect of the game probably have a much better understanding of what they need to do than the people on the forums, and the concerns raised in this thread would not be foreign concepts to them. Taking into account outside factors that may skew your data (such as varying player skill level), and identifying outliers (players intentionally throwing games) are things you learn in a 100 level college stats class.

Edited by DEMAX51, 23 September 2014 - 01:53 PM.

[Gyrok]

Zyllos, on 23 September 2014 - 10:12 AM, said:

The results, it being bad, is going to affect the outcome in what way?

In that Clan mechs will receive more nerfs or that Inner Sphere mechs will be underpowered?

The problem with labeling the current results as bad is that I see them as correct. CW, which allows IS vs Clans, will contain all the above type of players, those who run "bad" variants (FYI, there is no bad variants), those who are still grinding out skills, and those who are in trial mechs.

So, to get a complete picture, you have to allow those who want to run these types of mechs into the results to get a complete picture.

In what world are IS mechs underpowered at this point? Because I am not seeing it...the TW is good, but watch it get ROFLstomped by an IS assault with a STD engine...think you cannot take 2 ST off a TW before he can chew through your shield side on a HGN?

Think again...I killed a completely fresh DW the other night in my 733C and he managed to chew through my shield arm entirely before he went down, but armor was not cracked on LT front or rear...(4xUAC5 + 6xERML).

Clan mechs are UP at this point. The reality is, they were OVERNERFED...

[Gyrok]

DEMAX51, on 23 September 2014 - 11:57 AM, said:

Why do people assume that trained statisticians wouldn't know that they have to account for variations in player skill level during these tests?

I mean, c'mon guys - all they have to do is say "ok, the Clan teams had an average 100 point Elo advantage, which would give them a predicted win ratio of about 65% - it looks like the Clans are winning ~73% of the time, so they're still overperforming."

You don't think that's occurred to them?

Edit: And also, regarding people intentionally throwing matches in an effort to manipulate the data: this will probably result in that particular datum being labeled an outlier, and thrown out. I'm sure they're taking that into account as well.

Considering that nearly every match my guys played in the last round had minimum 1, often 2-4 guys on the IS side just suiciding to get another match...what do you think the odds are that the data was skewed significantly by that data...?

Karl has done the most statistical analysis from what I gather, and even he does not have all the crucial answers I asked questions about regarding that data.

This concerns me, because frankly the clan mechs are about to get completely screwed over if they keep listening to the morons with pitchforks and torches in this place.

[KhanCipher]

Zyllos, on 23 September 2014 - 10:12 AM, said:

(FYI, there is no bad variants)

I disagree with this statement, unless you can tell me the reason i should run a STK-4N over a STK-3F, AS7-K over a AS7-D, RVN-2X/4X over a 3L, BLR-1G/3M over a BNC-3S, or use the A or B dire whale arms over the prime arms (other than to elite the mech or get the pitiful xp bonus on omnis). Untill then there is such a thing as bad mechs and bad variants.

[DEMAX51]

Gyrok, on 23 September 2014 - 01:58 PM, said:

Considering that nearly every match my guys played in the last round had minimum 1, often 2-4 guys on the IS side just suiciding to get another match...what do you think the odds are that the data was skewed significantly by that data...?

Probably very little, because they'll be clear outliers.

Quote

Karl has done the most statistical analysis from what I gather, and even he does not have all the crucial answers I asked questions about regarding that data.

Maybe he just hasn't had the time to analyze the data fully yet? I'm sure he's got a lot on his plate right now.

Quote

This concerns me, because frankly the clan mechs are about to get completely screwed over if they keep listening to the morons with pitchforks and torches in this place.

Does this include the "morons" from House of Lords and Steel JaguaR, who have said, at this point, they're likely to play as Clan factions in CW because that's the way balance is leaning?