147 replies to this topic

[Cimarb]

In reference to:

Spoiler

Russ Bullock‏@russ_bullock

See this by example of our new MM command center pic.twitter.com/FImD5RD5NA

I found some interesting points from these 25 matches.

In regard to 4x3:

Here are the average #s from that sample data:
Lights ------ 2.44
Mediums -- 2.64
Heavies --- 3.48
Assaults --- 3.44

Outliers. Lights are actually NON-EXISTENT in one match. That means least 3 valves where released by them alone in that match! Mediums are almost as bad, since they have 3 matches where there was only a single Medium. On the other hand, there is a match where there are FIVE Heavies, as well as a match where there are FIVE Assaults.

50% of matches where 4x3. That is decent, but makes valves being released the norm, not the exception. Russ did state that this is one of the things they want to quickly fix.

In regard to Variance:

Now, something we cannot get a full view of is the Variance (far left column). We can only see 14 out of the 25 matches. Still a decent number to look at, though.

Elo is from 0-2800. While the average difference in variance is 241.27, match #24 in the list has a HUGE 762 variance. That is 30% of the whole Elo range, meaning one team is roughly in the middle of the mid-Elo "bucket", while the other team is roughly in the middle of the LOW-Elo "bucket". I would be willing to bet that was a horrid, horrid match.

Also, 9/14 matches were in triple digit variance, 8/14 were 200+ variance, and 4/14 were 350+ variance.

What makes this even worse is that these team variance's are the average Elo of the players on that team. Not only is there a large amount of difference between teams, but you also have the very possible issue that has been brought up before: High+Low vs. Medium.

What this means is that you have a few very High Elo players offsetting a bunch of very low Elo players, balanced against a full team of average players. Even though their combined Elo averages may be "close", the skills of those individual players are horribly "not close", which is what causes a lot of stomps to begin with.

Lessons:

• Role Warfare (rewards) needs to be implemented ASAP. Trying to force players to play lighter mechs is very obviously not working, and never will work. Until players are rewarded for playing what they WANT to play, they will never play them regularly. The vast majority of players would rather wait a couple minutes than play a mech they get little to no rewards playing.
  
• Data Transparency. This is exactly why PGI does not like releasing specific data...

UPDATE:

Karl Berg, on 05 July 2014 - 11:34 AM, said:

Hey Cimarb, unfortunately those variance numbers are not true statistical variance values. Out of lazyness I was simply calculating highest Elo on team - lowest Elo on team, rather than a true normalized sum of differences from mean.

Those screenshots were also taken within the first hour of launch of the new matchmaker. We've gone through extensive tuning passes since, both to bring wait times vs percentage 3's games in-line with stakeholder expectations, and to significantly tighten up Elo matching as well.

Edited by Cimarb, 05 July 2014 - 11:40 AM.

[FupDup]

Cimarb, on 05 July 2014 - 10:20 AM, said:

• Role Warfare (rewards) needs to be implemented ASAP. Trying to force players to play lighter mechs is very obviously not working, and never will work. Until players are rewarded for playing what they WANT to play, they will never play them regularly. The vast majority of players would rather wait a couple minutes than play a mech they get little to no rewards playing.

It's what many people have been saying for a loooooooooooooooooooooooooooooooooooooooooooooooooooooong time.

[Sephlock]

Cimarb, on 05 July 2014 - 10:20 AM, said:

What this means is that you have a few very High Elo players offsetting a bunch of very low Elo players, balanced against a full team of average players. Even though their combined Elo averages may be "close", the skills of those individual players are horribly "not close", which is what causes a lot of stomps to begin with.

But the hilarity, the HILARITY!



Who doesn't like Pro-Ams?

[Sarsaparilla Kid]

Combine that with 2-4 mans going exclusively to the group queue, per Russ's tweet, no wonder people are complaining so much about small groups getting stomped in the group queue: "FYI - since server reset the MM was reset so groups of 2-4 may now get tossed into the solo Queue, useful data to watch that will help us."

[Sandpit]

This is why I you sometimes Cim lol

[Thorn Hallis]

Good find, thank you.

[Cimarb]

Sarsaparilla Kid, on 05 July 2014 - 10:36 AM, said:

Combine that with 2-4 mans going exclusively to the group queue, per Russ's tweet, no wonder people are complaining so much about small groups getting stomped in the group queue: "FYI - since server reset the MM was reset so groups of 2-4 may now get tossed into the solo Queue, useful data to watch that will help us."

I was curious what he meant by that comment. Where small groups really only going into the group queue? I want an opt-in for solos to get into the group queue, so I have little sympathy for small groups, but that does explain why so many were complaining...

Sandpit, on 05 July 2014 - 10:44 AM, said:

This is why I you sometimes Cim lol

Anything to make you happy, Sand!

BTW, just because I defend PGI and think they are doing an exceptional job lately, does not mean I never have complaints

[Sarsaparilla Kid]

Cimarb, on 05 July 2014 - 10:51 AM, said:

I was curious what he meant by that comment. Where small groups really only going into the group queue? I want an opt-in for solos to get into the group queue, so I have little sympathy for small groups, but that does explain why so many were complaining...

That was my take on it, at least. Guess we'll be getting reports from the field on how the 2-4 mans fare after last night's reset.

[KingCobra]

It is interesting to see PGI struggle after 3 years with the same problem when so many have given them the answers to there matchmaker blues.

#1 pugs should be queued pug Vs pug only in there little 12v12 free for all's

#2 premade groups should have had a lobby/launcher system where they could launch private games 1v1-12v12 for free.

#3 but like all things PGI/IGP they will smoke a joint and say lets just throw them all together in 1 big Clusterffuucckk and make them mad like angry bees.

#4 Here comes the OMG how can you say such things we must defend bad designs by PGI until the end of the game.

Edited by KingCobra, 05 July 2014 - 11:03 AM.

[Sandpit]

25 matches shown
team a
61 lights
62 mediums
82 heavies
86 assaults

291 total mechs

Ok now numbers aside
Notice anything else interesting?
There is not a single match shown that has more than 3 lights on a team. Not one. Although nearly half of them have less than three and one has none at all.

Not a single match had less than 3 heavies and assaults. None

I wanted to break it down like this because I know a lot of people have no desire to read math equations and statistical data.
The rule of 3 simple is not going to generate variety on the battlefield. It could help, IF it were used in conjunction with a few other things but there's two things to remember regarding the rule of 3

1.) It was pulled down because the MM had a helluva time trying to put players in matches resulting in all kinds of problems due to the hard restrictions it placed on force composition.

2.) Release valves were put in to alleviate the above mentioned problems. Now you're back to the exact same thing you had before.

Once the release valves were put in, it completely defeated the purpose of the rule of 3 in the first place. The release valves had to be put in because there weren't enough lights and mediums to satisfy the MM system.

I don't know how else to put it plainly, yet PGI keeps clinging to their idea that the new MM will fix things. It won't because it can't.
No amount of "forcing" or hard restrictions is going to make a player want to take a mech that they don't have fun playing in. The original idea of 3x4 along with a weight matching was much better than this because it used more than one single metric to try and match up.

I sincerely doubt PGI will accept this though and I suppose it's entirely possible that their data shows otherwise but I can tell you from personal experience that I don't see any more or less lights on the field than I used to

[Cimarb]

Sarsaparilla Kid, on 05 July 2014 - 10:54 AM, said:

That was my take on it, at least. Guess we'll be getting reports from the field on how the 2-4 mans fare after last night's reset.

I think them having supremacy during solo drops will offset their hurt feelings when they are not the top dogs in the group queue. As long as it is a balanced amount of each type, they should like it better. I still do not have any sympathy for them, though. Give me my group queue as solo opt-in, PGI! (pls)

[Sandpit]

Cimarb, on 05 July 2014 - 10:51 AM, said:

I was curious what he meant by that comment. Where small groups really only going into the group queue? I want an opt-in for solos to get into the group queue, so I have little sympathy for small groups, but that does explain why so many were complaining...


Anything to make you happy, Sand!

BTW, just because I defend PGI and think they are doing an exceptional job lately, does not mean I never have complaints

I have no problems with people who defend or enjoy the game. My only problem comes when people who do that, do it at the expense of people who are trying to use the forums to give feedback on what they aren't happy with. I've spent more than my fair share of posts defending PGI and/or the game when someone posts something I just feel isn't right or downright absurd. I just happen to call them out when they're acting like unprofessional dbags as well.

[Cimarb]

KingCobra, on 05 July 2014 - 11:03 AM, said:

It is interesting to see PGI struggle after 3 years with the same problem when so many have given them the answers to there matchmaker blues.

#1 pugs should be queued pug Vs pug only in there little 12v12 free for all's

#2 premade groups should have had a lobby/launcher system where they could launch private games 1v1-12v12 for free.

#3 but like all things PGI/IGP they will smoke a joint and say lets just throw them all together in 1 big Clusterffuucckk and make them mad like angry bees.

#4 Here comes the OMG how can you say such things we must defend bad designs by PGI until the end of the game.

I completely disagree with you, which is probably why PGI is having such a hard time with this.

For instance, I want an opt-in so I can solo drop in the group queue, which is the antithesis of your #1.

PGI did give us a launcher that allows any type of drop we want, and I think it is perfectly reasonable to require ONE person out of 24 to be at least minimally vested in the game, or two if you want to use the server resources for duelling.

#3/4 are really not even worth addressing. It is humorous you even split them apart, as they are both trying to defend your point by insulting whoever may respond.

Sandpit, on 05 July 2014 - 11:03 AM, said:

Once the release valves were put in, it completely defeated the purpose of the rule of 3 in the first place. The release valves had to be put in because there weren't enough lights and mediums to satisfy the MM system.

I don't know how else to put it plainly, yet PGI keeps clinging to their idea that the new MM will fix things. It won't because it can't.
No amount of "forcing" or hard restrictions is going to make a player want to take a mech that they don't have fun playing in. The original idea of 3x4 along with a weight matching was much better than this because it used more than one single metric to try and match up.

I sincerely doubt PGI will accept this though and I suppose it's entirely possible that their data shows otherwise but I can tell you from personal experience that I don't see any more or less lights on the field than I used to

I would not say it completely defeated the purpose, but it did prove what we were saying quite well, so I have been hounding Russ about Role Warfare Rewards since it was released.

Sometimes, you have to let them do it their way so you can say, "see what I was talking about now?"

I would assume that if their data showed otherwise they would have used a different example screenshot. That is the issue with analytics to begin with: it is VERY easy to manipulate them to show what you want them to show.

Unfortunately, I think the issue here is more in regards to an overabundance of averaging. Like I mentioned about the variance, if you average averages, you will obviously get an even MORE average. If that is your goal, then it will succeed, but it does not make for balanced matches.

[KingCobra]

I completely disagree with you, which is probably why PGI is having such a hard time with this.

For instance, I want an opt-in so I can solo drop in the group queue, which is the antithesis of your #1.

PGI did give us a launcher that allows any type of drop we want, and I think it is perfectly reasonable to require ONE person out of 24 to be at least minimally vested in the game, or two if you want to use the server resources for duelling.
__________________________________________________________________________________

#3/4 are really not even worth addressing. It is humorous you even split them apart, as they are both trying to defend your point by insulting whoever may respond.
------------------------------------------------------------------------------------------------------------------------------------------------
maybe so

First off it was always a big mistake on PGI'S part to allow premade groups in any number to drop with pugs it just made both groups hate and rage.

Second point you still cannot play MWO like PC MechWarrior was before no matter If you group up 2v2-12v12 you cannot pick the map pick a mech loadout you want and variables and drop into a match.In fact you cannot even save multiple mech configurations in mechlab to pick one without rebuilding the whole mech every dam battle.

The fact is they should have listened to the older players and made a matchmaker for everyone not a random freeforall for everyone to drop into.OMG I think I will give up the debate and just go back to playing MW4 Mektek mod exleast it was mechwarrior.

Edited by KingCobra, 05 July 2014 - 11:39 AM.

[Cimarb]

I just got a response from Karl about this, God bless his weekend-working soul:

Karl Berg, on 05 July 2014 - 11:34 AM, said:

Hey Cimarb, unfortunately those variance numbers are not true statistical variance values. Out of lazyness I was simply calculating highest Elo on team - lowest Elo on team, rather than a true normalized sum of differences from mean.

Those screenshots were also taken within the first hour of launch of the new matchmaker. We've gone through extensive tuning passes since, both to bring wait times vs percentage 3's games in-line with stakeholder expectations, and to significantly tighten up Elo matching as well.

I am hoping he will be able to share a more updated sample for us to see.

[xXBagheeraXx]

I wonder when will people realize that not everyone WANTS to drive a paper mech. Most people who play mechwarrior want something with some combination of massive firepower, speed, and armor. You get most of those good traits in the heavy to assault class. Even if there were massive rewards for playing a light mech you would be hard pressed to find me driving anything under 60 tons or so. The Only mediums I have enjoyed so far is my yen lo wang, and the Trial Blackjack. Both of those mechs pack an ac20 and are ridiculously fast....

This to me is like saying people prefer driving Ferraris over a freaking Buick.

Lets see, do i want to go nearly 90kph and pack several ballistic, energy and missile weapons with a decent amount of armor to take a hit or three, or do i want to drive around in a mech that can get one shot legged with only a few medium lasers and machineguns?

Why is everyone surprised nobody drives lights? THEY ARE FREAKING LIGHTS.

[Cimarb]

KingCobra, on 05 July 2014 - 11:37 AM, said:

maybe so

First off it was always a big mistake on PGI'S part to allow premade groups in any number to drop with pugs it just made both groups hate and rage.

Second point you still cannot play MWO like PC MechWarrior was before no matter If you group up 2v2-12v12 you cannot pick the map pick a mech loadout you want and variables and drop into a match.In fact you cannot even save multiple mech configurations in mechlab to pick one without rebuilding the whole mech every dam battle.

The fact is they should have listened to the older players and made a matchmaker for everyone not a random freeforall for everyone to drop into.OMG I think I will give up the debate and just go back to playing MW4 Mektek mod exleast it was mechwarrior.

I again do not agree. As I already said, even when I drop solo, and I played solo-exclusively for over a year (meaning never once getting in a group... at all... for that time), I PREFER to drop with groups on both teams. The quality of a highly-grouped match is immensely better than that of solo-only, and I honestly never want to drop in a solo-only queue, ever, even when I am dropping solo myself. I want to be able to opt-in to the group queue even when dropping solo.

I do agree that it would be nice to be able to save multiple configurations. That would be a very cool feature. Has nothing to do with this discussion, but I completely agree.

Have fun with MW4. I will have fun with this, which is much more "Mechwarrior" than any game that has ever been done before, and is getting better every patch now.

[Cimarb]

xXBagheeraXx, on 05 July 2014 - 11:46 AM, said:

I wonder when will people realize that not everyone WANTS to drive a paper mech. Most people who play mechwarrior want something with some combination of massive firepower, speed, and armor. You get most of those good traits in the heavy to assault class. Even if there were massive rewards for playing a light mech you would be hard pressed to find me driving anything under 60 tons or so. The Only mediums I have enjoyed so far is my yen lo wang, and the Trial Blackjack. Both of those mechs pack an ac20 and are ridiculously fast....

This to me is like saying people prefer driving Ferraris over a freaking Buick.

Lets see, do i want to go nearly 90kph and pack several ballistic, energy and missile weapons with a decent amount of armor to take a hit or three, or do i want to drive around in a mech that can get one shot legged with only a few medium lasers and machineguns?

Why is everyone surprised nobody drives lights? THEY ARE FREAKING LIGHTS.

I am a heavy+ pilot myself, and while I have mastered many lights chassis and a few medium chassis, I will probably always be a heavy+ pilot.

I understand that not everyone is like me, though, so I want light pilots to be rewarded for what they are SUPPOSED to be doing, which is NOT brawling. I want them scouting for my heavy+, capping while I keep the enemy forces busy, and sneaking around enemy lines harassing the enemy when they have nothing else to do. None of those jobs currently provides them even a fraction of the points/money they get from brawling currently.

For mediums, they should primarily be providing the same roles as lights, but with a little more skirmishing and a little less capping.

Also, while I prefer to fight in heavy+, I will pilot lights much more when we get truly rewarded for capping and scouting. I would love to bring my Locusts back out at that time.

[Screech]

Looking at the those numbers the rule of 3 is really just there to limit 12-mans. The MM can't invent people to play mechs so the 3x4 is just there to keep 1200 ton 12-man Boogeymen at bay.

[Chemistry Warden]

My admittedly small sample size in drops is even more bleak than what Russ posted on twitter the other day. Since the patch, I've seen approx 2.09 lights/match, which may be skewed by the fact that a number of my drops were in lights.
If anyone is interested, so far it has shaken out like this:

2.09 L/m
2.96 M/m
3.52 H/m
3.43 A/m

Granted, my sample size is small, but I'll continue to track this and see what happens

Edit: this is per team, the 1:1 class matching does seem to be working, though

Edited by Chemistry Warden, 05 July 2014 - 11:59 AM.