133 replies to this topic

[SaltBeef]

There are other factors that lead to the tears a falling. Like when the MM puts 1 team with no ecm mechs against a team with 4 or 5. That is what stinks. Clan mechs are a little better but Is has better weapon heat , damage grouping, less shake, more customization for thier mechs. ALL CLANS HAVE IS RANGE AND Lasers. They have been gimped to stop the endless whining. Is that 2 thirds more mech chassis and Nooobs are ushered into Is mechs until recently.

[Asyres]

Mcgral18, on 24 September 2014 - 08:53 AM, said:

*snip*

You expect those types of players to get accurate results? Notice the crashing there?
Get 100 competent players and do the same thing, it's a better result than PUGs.

In the case of this game, people being bad, people bringing bad builds, bad mechs, and whatever else is essentially random error. At least some of those things will still happen if you pit a few 12 man teams against each other. while it's possible to reduce the influence of this type of error by reducing instances of it (by using better players as a testing population), that's not as statistically effective as just running more tests.

That also disregards the simple fact that running the tests in 12 mans introduces systematic error, since the game is played differently in that setting.

Now, I don't know how big PGI's sample size is, but if it's in the low 1000s (which seems reasonable), that number is likely something like 73 +/- 2%. Lets say your 100 players (lets call it 120) play 20 matches each - that's 100 matches - and get a clan win rate of, let's say, 60%. Sounds great, right? Well, unfortunately, the margin for error on that sample size is 10%, which means that 73% could be more or less accurate, or that the true win rate is around 50% - you don't know, because you don't have enough data. The fact that you've eliminated some sources of error becomes irrelevant, because you don't have a statistically representative sample.

[Revis Volek]

Hoax415, on 24 September 2014 - 08:46 AM, said:

No a test with very few players is not a better dataset. You people do not actually understand testing and how it works and how you get robust useful data.

I guess I will try to explain even though its a waste of time with people like Mcral18 who are obviously just pushing an agenda with no basis in reality and its a double waste of time because someone else is just going to start another one of these idiotic threads with the same bad thesis in a couple of days.

Lets say you run your 24 player test. You get two teams of players who are all within 100 elo of eachother per the MM. You have them play 10 matches. 5 where team A is clan and 5 where team B is IS.

According to the logic here that is some kind of super test. Those 10 matches are much more valuable data compared to 200 matches with trial mechs, bads, bad builds, bad chassis, elo difference etc. etc.

Right?

Wrong.

Less players and less matches is not going to get you a better set of data. That's not how it works.

Will you also be controlling for maps? Just run the test on one map right? No wait what if one team is much much better on that map than the other team.

Ok how about we up the test to 16 matches, 4 matches on 4 maps each team plays as each faction twice?

Well we still have the problem of what if one team is stronger on one type of map and you run 3 of those type. Or one team does much better on hot maps vs cold maps because reasons. And so on.

We haven't even talked about controlling chassis selection, controlling builds, gamemodes, ping, playstyle compatibility -or are these two teams actual competitive teams? which is 100% flawed from the get go by the way.

I'm going to stop here because someone with half a brain should start to see it by now. You use large randomized testing when you can't possibly hope to control every variable. Because once your dataset gets large enough all the random things become background noise to the overall aggregate trend.

tl;dr OP is wrong. As usual when it comes to the "throw out the tests PGI has done because I don't want to believe the results!" crowd.

Why cant we control the variables? Please explain this to me? We can control chassis' (did to an extent, one team was Clan one was IS) We can control PING to an extent (use players from similar regions in similar test), we can control game modes (i do this every day), and how can you say a team comprised of comp players is flawed? That statement doesn't even make sense? I can go get 12 guys who play comp, have played MCW, CNC 6 mans, and the Official MWO tourney....how do you get any more comp then that? How do you claim someone is or is not comp? What is your criteria for such a statement?

No one said their idea was 100%, we are just throwing IDEAS out. Sorry you dont know how to think tank and just shoot everyone's ideas down but a 24 player control with 6 low ELO and 6 HIGH ELO on each team seems pretty valid to me. It would also allow you to find people who maybe are not BIAS? Which IMO goes hand and hand with PLAY STYLE. If you are trying to preserve a tactic or technique sure you want in on the test servers and some say in the changes.

Also, lets not make statements about people you know nothing about. McGral18 is my Clan mate and a great pilot, courteous, calm and more capable then most i see on a daily basis. There is no agenda here, he hardly even uses clan mech from what i can tell.....WubShee is king in McGrals Stable!!!

Either way, your lack of appreciation for others ideas and rudeness really is unnecessary and toxic to the player base you are trying to fix the game for.....I think you need to get your priorities straight. You wanna balance the game and make it better but your actions on the forums show me otherwise. You could have educated them on such things if you are so intelligent with them rather then throwing out insults and patronizing everyone.

[SaltBeef]

Mechwarrior turned upside down from Canon.

[Revis Volek]

orcrist86, on 24 September 2014 - 09:04 AM, said:

I'm just going to their this out there. Match maker accounts for elo.... So there was no control grout necessary. The samples are not Ransome they are direct and comparable... So yeah. 73% is probably right on.

MM works until the valves open.....they forget to mention which games and when this happend.

If the valves open the rules go out the window and MM does what it has to to get the game started.....

[Appogee]

I have all the Clan Mechs and most of the IS ones.

Dire Whale, Timbie, and Storm Crow are the premier chassis in their three weight classes. IS has the advantage in the Lights, thanks to Firestarters and Jenners.

Most of the other chassis on both sides are "competitive", while a couple suck (eg. Awesomes, Adders).

I'm all for a tournament. But there's your answer.

[Revis Volek]

Hoax415, on 24 September 2014 - 09:08 AM, said:

That is why you need to get 100's of matches. The more matches you have the less the X% of them which were predetermined because of multiple afk's or a TK outbreak or one side has all horrible builds matter.

Even with a very small sample in the 73% test. You would need to believe that 20% of solo queue IS vs Clan matches had invalid results but not just invalid, invalid in favor of the clans.

You must believe that 1 in 5 matches were invalid because the Clans couldn't lose and that 0 matches ended up in a way that IS couldn't lose.

Otherwise you are just talking out of your ass as usual.

Saying a Team is defeated before they even enter the game is most "talking outta their ass" i have ever seen anyone do....where the hell did you even come up with that? You are making things up at this point and if we are going to argue "what if's" we will be here for a long, long time.

Give me one valid reason that cannot be controlled as to why the IS would have lost the game before the match even started?

Why wouldnt you just Omit that data if there was a problem?

Edited by DarthRevis, 24 September 2014 - 09:35 AM.

[Mystere]

Asyres, on 24 September 2014 - 09:25 AM, said:

Now, I don't know how big PGI's sample size is, but if it's in the low 1000s (which seems reasonable), that number is likely something like 73 +/- 2%.

The 73% figure is based on a sample size of exactly 268 matches.

Edited by Mystere, 24 September 2014 - 09:44 AM.

[FupDup]

One of the biggest issues is that the 73% number doesn't account for the chassis/weapons that CAUSED it the most. It just provides a blanket, general statement of "the Clams be OP yo." But it doesn't say which PART of them is OP. The result of this is that the people collecting data are going to make stupid assumptions such as every single Clan chassis/variant being too good, rather than just a few.

After all, I'm pretty darn sure that the Clan SRM2, UAC/10, and SPL mounted on Adders and Summoners are clearly are causing the Clans to win 73% of their matches!


Knowing the win rate is relatively meaningless unless we know what caused it to get there in the first place.

[Syncline]

I still say that their win rate numbers cannot be considered completely statistically meaningful until all Clan mechs that are involved in testing are available for C-Bills and a significant amount of time is given to the C-Bill buyers to get acclimated with, customize, and master/elite their mechs.

For example, when the last test was run, the clearly OP Clan mechs, the TWF and SCR, were not available for C-Bills.

I'm speculating here because I don't have any statistics on the amount of real-money purchases made by "casual" versus "serious" players, but I'm willing to bet that, among Clan tech users, there are more "serious" players than "casuals." These serious players are probably generally better pilots than the casuals, and have access to the most problematic mechs in the game. Creating an experiment that combines disproportionately more serious players with OP mechs, and testing them against a pool of pilots which, generally, is less skilled and which does not want to pay real money to have access to the OP tech, does not generate a statistically relevant result.

Edited by Syncline, 24 September 2014 - 10:05 AM.

[Kain Demos]

Syncline, on 24 September 2014 - 09:54 AM, said:

I still say that their win rate numbers cannot be considered completely statistically meaningful until all Clan mechs that are involved in testing are available for C-Bills and a significant amount of time is given to the C-Bill buyers to get acclimated with, customize, and master/elite their mechs.

Agreed. Every time I bring this up I've been told "STFU there was a Nova Prime trial 'mech for the test!".

[Angel of Annihilation]

Here is what PGI needs to do.

First a poll of IS Players:

Question 1: Do you feel the Clans are OP?


Question 2: Do you feel you can compete with the Clans in an IS mech?
Question 3: What is your reaction when you see your dropped in to a Clan vs IS match.

• Try even harder to show those dirty Clanners whose boss
  
• Play as normal
  
• Play as normal but play with the understanding you are going to lose.
  
• Gripe and QQ for the first 5 mins of the match about how clan are OP without concentrating on winning.
  
• Suicide rush the enemy so you can get the match over ASAP because your going to lose anyway.
  
• Rage Quit and Disconnect from the match.

Here is why this is important:

First, if only 10% of respondents answer number 5 or 6 the match now becomes the equivalent of a match between 11 IS mechs and 12 Clan mechs. Obviously a pretty huge advantage for the Clans even if they are balanced.

Second, if an additional 10% answer number 4-5, then now the matches become the equivalent of a match between 10 IS mechs and 12 Clan mechs. Obviously a massive advantage for the Clans at this point.

Also if anyone already feels the clan are OP or that IS mechs can't compete they likely already have a defeatist attitude which means they aren't playing there best.

The point that I am trying to make is any IS match against the Clans already starts as an uphill battle just due to the mentality of the player base. When you bear in mind these factors, a 73% win rate isn't even close to unreasonable. Hell even the 90% win rate shouldn't have been a surprise.

[Syncline]

Something else that I noticed is that I find it harder to see the target reticules in IS mechs than in Clan mechs when placed against backgrounds of certain colors. It's definitely a situational problem, but one that Clan mechs don't have. How much brown terrain is in the game? How much blue terrain is in the game?

[Brody319]

Syncline, on 24 September 2014 - 10:24 AM, said:

Something else that I noticed is that I find it harder to see the target reticules in IS mechs than in Clan mechs when placed against backgrounds of certain colors. It's definitely a situational problem, but one that Clan mechs don't have. How much brown terrain is in the game? How much blue terrain is in the game?

I dont have any problem with that, doubt it has any real effect on gameplay.


Look we will never ever know 100% unless we make a private server, and control every variable

[C E Dwyer]

wanderer, on 24 September 2014 - 09:20 AM, said:

You've still got the best Clan 'Mechs behind a paywall when the last Clan vs IS test came out.

While there's plenty of bad pay-pilots, there are inordinately more bad F2P ones. Right now, that means your odds of having a bad player in a heavy/assault slot for the IS is considerably higher, and thus the odds of having one of those dreaded under-100 damage team-mates that can wreck a game singlehandedly.

I'd point more towards light/medium slots being F2P-open now for Clans as the reason for 73% vs. 90% W/L...because the same bads you see in IS 'Mechs are finally being allowed to migrate into Clan chassis, meaning the same reasons for horrible 12-0 style stomps are beginning to get their chance to do so in even better machines to do badly in.

Wander has a very valid point here, I don't think on reflection, there was much validation done by the last test, and while things like trial mechs are only IS was supposed to have been taken into consideration, I think its just another big flaw in the test, until clan are trail and at least the first wave is open the tests will be difficult if not impossible to make a valid call on it

[C E Dwyer]

Syncline, on 24 September 2014 - 10:24 AM, said:

Something else that I noticed is that I find it harder to see the target reticules in IS mechs than in Clan mechs when placed against backgrounds of certain colors. It's definitely a situational problem, but one that Clan mechs don't have. How much brown terrain is in the game? How much blue terrain is in the game?

True but take into consideration that IS reticules on Is mech are far thinner and don't obsurce as much of the target for pin point mid ranged damage, and also the blue can be partly lost in the sky on some maps as Is ones can be party lost on canyon and similar

[terrycloth]

When I see an Orion, piloting an IS mech, I know that it's an opponent with guns and I might lose, but I feel pretty confident because generally Orions suck. They're slow and I'm going to hit them with all my weapons in their giant torsos, and if I'm in a light or medium there's a reasonable chance they won't be able to fire back effectively.

When I see a Timberwolf, piloting an IS mech, I know that I'm horribly outclassed. If I get the drop on one it makes sense to unload some damage into it and then try to disengage before it notices me, but it's faster than me (unless I'm in a Jenner or something) so more likely it'll be a duel to the death. My death. Because they're really hard to kill (especially when they start bunny-hopping and abusing that bug) and put out a lot of damage.

If I see a Timberwolf that's really heavily damaged, then I can be pretty confident that I can take out a side torso and at least make it easier for the third or fourth victim to come out on top.

Today I killed a TWolf in a Centurion! I mean, yeah, it was already cored, and by the time I managed to hit it enough to destroy the structure and actually kill it it had destroyed my weapon arm and stripped off most of my armor, and it was on Terra Therma and the pilot was kind of bad and overheated a bunch, but I won!

Edited by terrycloth, 24 September 2014 - 10:47 AM.

[Mystere]

FupDup, on 24 September 2014 - 09:51 AM, said:

One of the biggest issues is that the 73% number doesn't account for the chassis/weapons that CAUSED it the most. It just provides a blanket, general statement of "the Clams be OP yo." But it doesn't say which PART of them is OP. The result of this is that the people collecting data are going to make stupid assumptions such as every single Clan chassis/variant being too good, rather than just a few.

As I have been saying and hinting all along with regard to this topic, a number, without the proper context, is just a number. It does not mean anything.

Many people will grasp at any "statistic" that supports their views, no matter the quality of the data it was based on. It's really not surprising when even so-called government "leaders" claim that climate change does not exist, or that SIDA (the English words seems to be censored) is only limited to "immoral" people, because some "scientist" or "priest" "claims" as much. Humanity is ****** until such people are expunged from their positions of authority.

Edited by Mystere, 24 September 2014 - 11:14 AM.

[Asyres]

Mystere, on 24 September 2014 - 09:44 AM, said:

The 73% figure is based on a sample size of exactly 268 matches.

Blah, that's far less useful. I stand by my assertion that arranged matches would not represent improved data, however.

I wish they'd run an IS vs. Clan queue for a week or something - even if it was only opt-in - and see what data comes out of that. Once you get a few thousand points in your data set, you could start to see if specific factors disproportionately affect match outcome, rather than just "Clans are probably a little too good in general" - which is just barely actionable, and subject to misinterpretation.

[Mystere]

Asyres, on 24 September 2014 - 11:23 AM, said:

Blah, that's far less useful.

And yet people are clinging (to put things ever so politely) to that "73%" figure as if their lives depended on it.

Asyres, on 24 September 2014 - 11:23 AM, said:

I stand by my assertion that arranged matches would not represent improved data, however.

Would 2000 arranged matches using reasonably competent players be sufficient?

Given that the above might be difficult, would 2000+ matches from a group-based tournament be a better means of gauging IS vs. Clan mech quality?