Jump to content

Experiment To Test The Clan/is Skill Gap Hypothesis


58 replies to this topic

#1 Jun Watarase

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 3,504 posts

Posted 21 December 2016 - 12:12 PM

Edit : Because people aren't reading, I'm going to highlight two important points.

1. The number of games will keep increasing. The sample size is not fixed at some low number as some people have claimed.

2. No final conclusions have been drawn yet, just initial impressions. The current results do not prove or disprove anything.

End edit.

Many players have been posting about how Clans have most of the top units/players now (whether loyalist or via merc contracts) and that most FW matches are ending up with teams of IS newbies facing top clan teams.

If this is true, then it stands to reason that the majority of clan vs IS matches will show a very obvious skill disparity even if you drop solo and make no effort to stack your side full of top players before the match starts. I mean, if most of the top players are on the clan side...then most of the low and mid tier players will be on the IS side.

So i decided to run an experiment to see if real world results matches the hypothesis.

I will be taking screenshots of the end game scores and posting them as I get them. Screenshots taken during NA prime time.

Score so far : 6 clan wins, 12 IS wins.

http://imgur.com/bVsb2pa : Clan win. By 10 points in conquest. The IS could easily have won this since they wiped our first wave with no losses, but after the first wave they just lost focus, mostly milled about in the center, and we managed to barely outcap them.

http://imgur.com/8zwCZFG : IS win. IS unit proceeds to curb stomp clan randoms.

http://imgur.com/1VyacPn : IS win. IS grabs kill and cap advantage from the beginning and maintains it for the rest of the game.

http://imgur.com/OSEIuDB : IS win. IS unit proceeds to curb stomp clan randoms.

http://imgur.com/3ijMUHM : IS win. IS randoms beat clan randoms with a 19 kill advantage.

http://imgur.com/DEs6fjP : Clan win. Was even till the last 8 minutes or so, then half the IS team tried to push while the other half stayed back and Clans got the advantage.

Initial impressions : Its quite clear that the average IS player is actually better than the average clan player, not taking into account the drops where its a unit vs randoms. Not sure where all these "clan teams full of top players" or "terribly bad IS pub teams" are...the only terribly bad matchups ive seen are the ones where the IS have the advantage.

It will be interesting to see if this changes as I play more games and the sample size increases.

Edit 1 :

http://imgur.com/oSYNX6E : IS win. IS unit proceeds to curb stomp clan randoms.

http://imgur.com/NOaeTEg : IS win. Same IS unit shows up again and curb stomps clan randoms.

http://imgur.com/jzYwBdQ : IS win. IS out trades clans at long range, on alpine peaks. IS had at least one lance with a narc/LRM boat setup.

http://imgur.com/LDf1FTk : IS win. IS basically camped right outside their base and refused to move, and several mechs just sat at the spawn gates the whole game with LRM boats or sniper builds.

http://imgur.com/qMcwF4e : Clan win. From what I could tell, several clan players running laser vomit basically focused fired the IS to oblivion and the IS didn't have anyone on their level.

IS are still winning the majority of matches, especially in matches where its mostly randoms vs randoms. Still have not seen any "clan teams full of top players" or "terribly bad IS pug teams".

Edit 2 :

http://imgur.com/nuHqM4R : IS win. IS unit proceeds to curb stomp clan randoms.

http://imgur.com/zSgrGZR : IS win. Same IS unit shows up again and curb stomps clan randoms.

http://imgur.com/oUtNSZT : Clan win. Clan randoms stomp IS 3 man + randoms.

http://imgur.com/ijTBUsG : Clan win. Clan partial premade stomps IS partial premade.

http://imgur.com/sIu7SHI : Clan win. Clan partial premade stomps IS partial premade.

http://imgur.com/Wjdmixm : IS win. IS partial premades stomp clan randoms.

http://imgur.com/xeBZ6Og : IS win. IS unit stomps clan randoms.

Slightly more even results this time. IS still holds a 2:1 win ratio though (in total).

Edited by Jun Watarase, 24 December 2016 - 09:40 AM.


#2 Barantor

    Member

  • PipPipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 2,592 posts
  • LocationLexington, KY USA

Posted 21 December 2016 - 12:14 PM

Might depend on the time of day too, certain timezones might have it even, some might have it way off. Also the holiday season might skew things a bit as younger folks have off school.

#3 L3mming2

    Member

  • PipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 1,304 posts

Posted 21 December 2016 - 12:23 PM

View PostJun Watarase, on 21 December 2016 - 12:12 PM, said:

Many players have been posting about how Clans have most of the top units/players now (whether loyalist or via merc contracts) and that most FW matches are ending up with teams of IS newbies facing top clan teams.

If this is true, then it stands to reason that the majority of clan vs IS matches will show a very obvious skill disparity even if you drop solo and make no effort to stack your side full of top players before the match starts. I mean, if most of the top players are on the clan side...then most of the low and mid tier players will be on the IS side.

So i decided to run an experiment to see if real world results matches the hypothesis.

I will be taking screenshots of the end game scores and posting them as I get them.

http://imgur.com/bVsb2pa : Clan win. By 10 points in conquest. The IS could easily have won this since they wiped our first wave with no losses, but after the first wave they just lost focus, mostly milled about in the center, and we managed to barely outcap them.

http://imgur.com/8zwCZFG : IS win. IS unit proceeds to curb stomp clan randoms.

http://imgur.com/1VyacPn : IS win. IS grabs kill and cap advantage from the beginning and maintains it for the rest of the game.

http://imgur.com/OSEIuDB : IS win. IS unit proceeds to curb stomp clan randoms.

http://imgur.com/3ijMUHM : IS win. IS randoms beat clan randoms with a 19 kill advantage.

http://imgur.com/DEs6fjP : Clan win. Was even till the last 8 minutes or so, then half the IS team tried to push while the other half stayed back and Clans got the advantage.

Initial impressions : Its quite clear that the average IS player is actually better than the average clan player, not taking into account the drops where its a unit vs randoms. Not sure where all these "clan teams full of top players" or "terribly bad IS pub teams" are...the only terribly bad matchups ive seen are the ones where the IS have the advantage.

It will be interesting to see if this changes as I play more games and the sample size increases.


if u look at the war logs you will see that the clans win most maches, a quiq chec now gave 49/120 IS victory's (this is the higest i have seen IS in the 7 times i checked the logs BTW) so yes as a clan pug u will get stomped by good IS pre made's but thats the case to, and even more so when u play as a IS pug...

in short, pugs always get the short stick when in the same que as groops..

#4 Snazzy Dragon

    Member

  • PipPipPipPipPipPipPipPipPip
  • The Defiant
  • The Defiant
  • 2,912 posts
  • LocationRUNNING FAST AND TURNING LEFT

Posted 21 December 2016 - 12:25 PM

Different timezones, different players. A few of my friends I know that hate having piloting lights, and have to take two clan lights just to take two clan assaults; they dropped clan in favor of the 265 tons IS has to offer. That's two assaults, a heavy, and a medium if you distribute properly. I personally will be taking a battlemaster 2C, marauder 3R, and whichever thunderbolt it is that gets crazy large pulse laser quirks, then a cicada 3M with 2 ER large lasers to fill out the rest of the tonnage remaining.

So some of those players could be switching sides as well (mercs are OP).

Edited by Snazzy Dragon, 21 December 2016 - 12:26 PM.


#5 Jun Watarase

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 3,504 posts

Posted 21 December 2016 - 12:30 PM

Obviously if you drop in a 12 man full of top players, 99% of the time the game is won before it even began. Thats not an accurate judgement of whether there is a big skill gap between the two sides. Its like playing poker where you deal yourself two aces at the start...you cannot accurately judge the local poker scene by doing this. There are plenty of reasons for why clans would be winning most of the time...the most obvious one being that a skilled group of clan players are playing many more matches than their IS counterparts and that shifts the tug of war advantage to the clan side.

Anyone who played during the steam release should remember that 90% of the time you would have the clans face teams comprised of mostly IS newbies, even when dropping as randoms. The skill disparity was very, very obvious at the time to everyone (except perhaps PGI).

At the very least, my experience so far shows that the skill gap isnt as big as during the steam release. Of course, your mileage may differ.

Also those screenshots were taken during NA prime time.

Edited by Jun Watarase, 21 December 2016 - 12:48 PM.


#6 Kangarad

    Member

  • PipPipPipPipPipPipPip
  • FP Veteran - Beta 2
  • FP Veteran - Beta 2
  • 573 posts
  • LocationIn the Mechlab, adding more Double Heatsinks.

Posted 21 December 2016 - 12:36 PM

View PostSnazzy Dragon, on 21 December 2016 - 12:25 PM, said:

Different timezones, different players. A few of my friends I know that hate having piloting lights, and have to take two clan lights just to take two clan assaults; they dropped clan in favor of the 265 tons IS has to offer. That's two assaults, a heavy, and a medium if you distribute properly. I personally will be taking a battlemaster 2C, marauder 3R, and whichever thunderbolt it is that gets crazy large pulse laser quirks, then a cicada 3M with 2 ER large lasers to fill out the rest of the tonnage remaining.

So some of those players could be switching sides as well (mercs are OP).


oh noes IF I take 2 assaults on IS side Ill have to take mechs filling 65 tons. and one is a medium the other one is a light.

you dont have to take a KDK3 and A Scorch just because theyr the top assaults currently. you could bring your KdK3 and a heavy a medium and a light.

also check the thunderbolts again... that one got nerfed sometimes recently... its not that good anymore.

its not even unknown that currently the best mechs are aviable to the clans... (yes I do have clan mechs and yes I once played on the Smoke Jaguar side ...).

but tbh. clans still stomp IS even witht eh tonnage disadvantage... and they did before and theres no CW where IS actualy was winning the CW. even before all the "good" units switched to clan "recently" (all former clans that were Clan are bad right russ? :D )

This game does need a better way to balance clans and IS but PGI will most likely not deliver. Live with it and keep stomping the pugs.

#7 Snazzy Dragon

    Member

  • PipPipPipPipPipPipPipPipPip
  • The Defiant
  • The Defiant
  • 2,912 posts
  • LocationRUNNING FAST AND TURNING LEFT

Posted 21 December 2016 - 12:39 PM

View PostKangarad, on 21 December 2016 - 12:36 PM, said:

Spoiler



I don't think you get it. So I'll make it simple.

I

do

not

like

piloting

lights


I never said the tonnage makes IS better than clan but I will personally perform better if I'm not forced to take them and have fun even if I lose more often to op clammers.

#8 Kangarad

    Member

  • PipPipPipPipPipPipPip
  • FP Veteran - Beta 2
  • FP Veteran - Beta 2
  • 573 posts
  • LocationIn the Mechlab, adding more Double Heatsinks.

Posted 21 December 2016 - 12:43 PM

View PostSnazzy Dragon, on 21 December 2016 - 12:39 PM, said:


I don't think you get it. So I'll make it simple.

I

do

not

like

piloting

lights


I never said the tonnage makes IS better than clan but I will personally perform better if I'm not forced to take them and have fun even if I lose more often to op clammers.

then pilot mediums and heavys.

#9 C E Dwyer

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 9,274 posts
  • LocationHiding in the periphery, from Bounty Hunters

Posted 21 December 2016 - 12:52 PM

it's a nice idea, but where does the skill end, and the advantage of clan mechs start ?

Teams should beat pugs all the time regardless of who's side your on, unless the DC is bad, or the Team badly let down the DC.

If you take the top eight units and five are on one side that side should ultimately win unless the lower skilled people are vastly inferior on the side with five of the top teams.

#10 L3mming2

    Member

  • PipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 1,304 posts

Posted 21 December 2016 - 02:29 PM

View PostSnazzy Dragon, on 21 December 2016 - 12:39 PM, said:


I don't think you get it. So I'll make it simple.

I

do

not

like

piloting

lights


I never said the tonnage makes IS better than clan but I will personally perform better if I'm not forced to take them and have fun even if I lose more often to op clammers.


so 1 more T1 player for IS? its working beter tweet russ his plan is working, lets drop a other 10t of clans and maybe just maybe the win/los ratio wil be geting close to 50/50 :P

#11 Wintersdark

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • 13,375 posts
  • Google+: Link
  • Twitter: Link
  • LocationCalgary, AB

Posted 21 December 2016 - 02:46 PM

View PostJun Watarase, on 21 December 2016 - 12:30 PM, said:

Obviously if you drop in a 12 man full of top players, 99% of the time the game is won before it even began. Thats not an accurate judgement of whether there is a big skill gap between the two sides. Its like playing poker where you deal yourself two aces at the start...you cannot accurately judge the local poker scene by doing this. There are plenty of reasons for why clans would be winning most of the time...the most obvious one being that a skilled group of clan players are playing many more matches than their IS counterparts and that shifts the tug of war advantage to the clan side.

Anyone who played during the steam release should remember that 90% of the time you would have the clans face teams comprised of mostly IS newbies, even when dropping as randoms. The skill disparity was very, very obvious at the time to everyone (except perhaps PGI).

At the very least, my experience so far shows that the skill gap isnt as big as during the steam release. Of course, your mileage may differ.

Also those screenshots were taken during NA prime time.


You misunderstand the "skill gap".


It isn't that Clan players are just better than IS players, it's that the top 5 units are all Clan side (or where, when I last checked Russ's reports).

So, this leads to them winning basically all their matches, so even if all the rest of the players are evenly skilled, the top 5 units are enough players in FW to utterly dominate overall.


So, sure, IS vs Clan may be 50/50 otherwise, but Clans will still dominate overall with several units that effectively "auto-win" all their matches because there's such a skill gap between them and the top IS teams.

#12 Wintersdark

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • 13,375 posts
  • Google+: Link
  • Twitter: Link
  • LocationCalgary, AB

Posted 21 December 2016 - 02:57 PM

View PostSnazzy Dragon, on 21 December 2016 - 12:39 PM, said:

I don't think you get it. So I'll make it simple.

I

do

not

like

piloting

lights

I never said the tonnage makes IS better than clan but I will personally perform better if I'm not forced to take them and have fun even if I lose more often to op clammers.


This is why I don't like reducing the Clam tonnage. I don't care if they massively increase the IS tonnage instead.

I *hate* playing lights. Very strongly. So when tonnage is reduced, that means I can only play mediums and light heavies.

I love my assaults, but I can't use them really at all in CW now, unless I want to commit to playing lights too, but... eww.

Now, I'm sure there's some IS players getting all herp-derp about KDK-3's. I'm not looking to run swarms of KDK's. I don't want to run ANY lights, so even at 250, if I chose to run a single KDK (which, incidentally, would be my beloved KDK-4) then it'd be run alongside 3 50 ton mediums.

At 240, even running a 85t assault leaves me with 2x50+1x55, whereas formerly I'd have gotten a couple mediums, a heavy, and the 85t assault.

For me anyways, it's not about faction balance, but rather that when you start dialing back the drop deck weight, you start forcing usage of lights, and that makes me sad, because lights are gross. I'd be so, so much happier giving IS more tonnage (or really anything else). Just don't make me pilot lights.

What's funny? If I have to bring a light (and in my world, that's anything <50t), I'll minimize it, run the lightest one I can because it's a throwaway regardless, and bring MORE assaults/heavies instead of more mediums.

#13 Besh

    Member

  • PipPipPipPipPipPipPipPip
  • Survivor
  • Survivor
  • 1,110 posts
  • LocationGermany

Posted 21 December 2016 - 02:58 PM

Tbh, over the course of this day, I have on occasion made a complete ******* out of my self in inGame Chat, simply because I could not believe and stand the totally, absolutely and extremely frustrating CRAP lvl of play IS Groups displayed .

ALL the usual Stuff :

People do NOT communicate AT ALL, no VOIP, NO TXT, nothing .
They dont respsond, even if I gave clear and precise Information ( "guys, there are only 3 Mechs in f5, if we push em hard fast they are easy kills!" -> People keep standing still behind cover, not moving ) .
People seem to have NO clue of positiong, no Map awareness, many do not even seem to look at their MiniMap .

I am really at a loss as to what has been happening in those Matches. It definitely felt as if people did not have a clue, OR did not even try .

Subsequently got rolled by Clan PUGTeams, who were not very good themselves either, but definitely MORE aware of their surroundings, and MORE working together in a coordinated fashion .

Edited by Besh, 21 December 2016 - 03:04 PM.


#14 Levi Porphyrogenitus

    Member

  • PipPipPipPipPipPipPipPipPip
  • Mercenary
  • Mercenary
  • 4,763 posts
  • LocationAurora, Indiana, USA, North America, Earth, Sol, Milky Way

Posted 21 December 2016 - 03:05 PM

I think the OP is misunderstanding Russ's claims. He's not saying the Clans are winning because their PUGs have more T1-2 players, he's saying they are winning because of the top 5 units all are Clan (though per the Twitter thread apparently one unit switched recently and the win/loss numbers have closed a good bit).

In other words, the metric can't be PUG v PUG, it must be testing to see how many PUG v Unit (say, 50%+ single-tag drops against drops that don't have at least half their team with a single tag) matches go one way or the other, and on which sides the Unit drops tend to be.

#15 Wintersdark

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • 13,375 posts
  • Google+: Link
  • Twitter: Link
  • LocationCalgary, AB

Posted 21 December 2016 - 03:25 PM

View PostLevi Porphyrogenitus, on 21 December 2016 - 03:05 PM, said:

I think the OP is misunderstanding Russ's claims. He's not saying the Clans are winning because their PUGs have more T1-2 players, he's saying they are winning because of the top 5 units all are Clan (though per the Twitter thread apparently one unit switched recently and the win/loss numbers have closed a good bit).

In other words, the metric can't be PUG v PUG, it must be testing to see how many PUG v Unit (say, 50%+ single-tag drops against drops that don't have at least half their team with a single tag) matches go one way or the other, and on which sides the Unit drops tend to be.


As well, when it was all 5 top units, that's a large number of unit vs. unit matches that are going to be very one sided as well.

#16 Johnny Z

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Knight Errant
  • 9,942 posts
  • LocationDueling on Solaris

Posted 21 December 2016 - 03:29 PM

Don't need any tests to see the combat may be close but one side has a massive speed advantage that can help dictate range, area of engagement, evasion and everything else speed offers.

This speed division is on the heavy end. IS lights can compete in the speed area even if under gunned a bit.

Edited by Johnny Z, 21 December 2016 - 03:30 PM.


#17 Jun Watarase

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 3,504 posts

Posted 21 December 2016 - 04:18 PM

Im not referencing Russ's claims at all, im referencing the many post by different people on the forums that most of the top players are playing clans atm and IS pugs are full of terribly bad players.

Original post updated. Up to 3 clan wins (2 of which were close wins) vs 8 IS wins (most of which were curb stomps).

If there is such a massive disparity in skill/population as people have claimed, then i should be seeing matches on the level of steam release where it was basically 12-0 for the first wave and the rest were clans spawn camping IS newbies in trial mechs. Not seeing this at all, even IS randoms are beating clan randoms most of the time, especially with the tonnage advantage. Really sick of seeing 5 LPL battlemasters in the first wave.

Edited by Jun Watarase, 21 December 2016 - 04:19 PM.


#18 Chuck Jager

    Member

  • PipPipPipPipPipPipPipPipPip
  • The 1 Percent
  • The 1 Percent
  • 2,031 posts

Posted 21 December 2016 - 04:42 PM

SO if I say I do not like driving the speed limit, this is a valid reason to not make changes so that others are not hurt.

I love playing clan mechs. Why, because I like functionally good "kit". But be honest. Before this change I could run a kodiak, ebj and Hunch2 and never even need to touch my light mech.

This change is not just a Clan vs IS balance, it is a mechanism to motivate folks to play for the weaker side despite the mechanics.

PS I find most of clan heavies a great deal more enjoyable to play than almost all of the IS Assaults - specifically from the reliability to have an impact on every game. The Kodiak also plays like an over armored heavy.

#19 Tristan Winter

    Member

  • PipPipPipPipPipPipPipPipPip
  • Ace Of Spades
  • Ace Of Spades
  • 2,530 posts
  • LocationNorway

Posted 21 December 2016 - 04:44 PM

The sample size you'd need to say anything remotely conclusive is just crazy. We used to have guys who made Science threads here, but I don't think they do that stuff anymore.

You need to compile lots and lots of post-match screenshots. Preferably from different tiers, preferably from both solo and group queue.

#20 Wintersdark

    Member

  • PipPipPipPipPipPipPipPipPipPipPip
  • 13,375 posts
  • Google+: Link
  • Twitter: Link
  • LocationCalgary, AB

Posted 21 December 2016 - 06:06 PM

View PostJun Watarase, on 21 December 2016 - 04:18 PM, said:

Im not referencing Russ's claims at all, im referencing the many post by different people on the forums that most of the top players are playing clans atm and IS pugs are full of terribly bad players.

Original post updated. Up to 3 clan wins (2 of which were close wins) vs 8 IS wins (most of which were curb stomps).

If there is such a massive disparity in skill/population as people have claimed, then i should be seeing matches on the level of steam release where it was basically 12-0 for the first wave and the rest were clans spawn camping IS newbies in trial mechs. Not seeing this at all, even IS randoms are beating clan randoms most of the time, especially with the tonnage advantage. Really sick of seeing 5 LPL battlemasters in the first wave.



Well, if you're just addressing random vapid forum posts and "what people say", there's not really much point in the discussion.

Fundamentally, there's no difference between a Clan player and an IS player; hell, most - even the newest - are both Clan AND IS players, after all. After all, I've got a Clan tag, but I garauntee I've more IS mechs than most IS players do =)

In terms of actual results, as Mr. Winter says above, you'd need a MUCH larger sample size to say anything even remotely useful. A dozen matches or so is so small as to be utterly useless. I'm not saying your right or wrong with this, mind, just that your... 11? matches above are just random.

After all, we've all had those days where you lose a dozen QP matches, or win a dozen, when nothing really changed. Just the way the cookie crumbled.




All the above said, proof is in the pudding. Seems to me that Clans are winning more matches overall. Or am I wrong? Someone posted stats about this elsewhere, but I'm not sure where they got it from.





3 user(s) are reading this topic

0 members, 3 guests, 0 anonymous users