248 replies to this topic

[thievingmagpi]

thievingmagpi, on 04 November 2019 - 10:06 AM, said:

big thumbs up to Nightbird

I now regret making this statement

Edited by thievingmagpi, 31 January 2020 - 07:01 PM.

[killzone1]

Nightbird, on 31 January 2020 - 02:55 PM, said:

Yes, the community is sometimes at fault, but PGI is also at fault for creating a MM that makes bad matches 50% of the time. How about improving/reducing this to 10%?

Share the responsibility a little.

I'd say the matchmaker makes bad matches 75% of the time. Of my last 30 matches, 26 of them have been either a win or a loss with a 12-4/4-12 or worse score. 2 out of the 30 had a score of 12-10.

I get too many complete crap teams where I can do 800-1000 damage in a medium(without LRMS) and we still lose. The assaults and heavies end up with like 150 damage and no kills.

Just had another yet another 12-4 win, and Vxheous was on the other team, so I guess it's not just me, even top pros can't carry some of these messed up teams you get.

Edited by killzone1, 03 February 2020 - 05:11 PM.

[MischiefSC]

Nightbird, on 31 January 2020 - 02:55 PM, said:

Yes, the community is sometimes at fault, but PGI is also at fault for creating a MM that makes bad matches 50% of the time. How about improving/reducing this to 10%?

Share the responsibility a little.

Your matchmaker would help but, again, within a given MM segment you're not going to get great matches. Too many axis to balance on (weight, skill, build, map vs build, synergy, etc) with too few players in any given window.

However.

It would no doubt be a significant improvement, especially during peak times.

[Nightbird]

MischiefSC, on 03 February 2020 - 11:49 PM, said:

Your matchmaker would help but, again, within a given MM segment you're not going to get great matches. Too many axis to balance on (weight, skill, build, map vs build, synergy, etc) with too few players in any given window.

However.

It would no doubt be a significant improvement, especially during peak times.

If PGI shared more metrics, I can incorporate all of that into a model and generate the most optimal match every time. I do this for a living ya know. We all know PGI will not share though.

[MischiefSC]

Nightbird, on 04 February 2020 - 11:10 AM, said:

If PGI shared more metrics, I can incorporate all of that into a model and generate the most optimal match every time. I do this for a living ya know. We all know PGI will not share though.

Me too. I even did a big study into matchmaking used in games. It's why even with the huge money thrown at TrueSkill and similar big effort matchmakers for games with incredibly thick skill bands you still get, at best, mediocre matchmaking. Interestingly one big correlation for seating players with consistent accuracy was time players had in the game to make decisions. The faster paced a game is the less consistent the player performance was, same with equipment/level/character class variety.

This is way less pronounced with thick skill bands. The closer in skill level everyone on a team is the more accurate the matchmaker ends up. However high/low to a value was painfully imprecise. So translated to a value your skill might be 2,000 pts on average, right? However your ACTUAL per-match performance swings by almost 50% in performance based on everything from how much sleep you've gotten to when you ate last, not to mention things like how meta the mech you're playing is and in game synergies.

To get high/low to a value you're functionally inflating and deflating each players relative estimated contribution. Instead of everyone being 8.333% you might be valued 16% and two other players at only 4%. The result of this is that if you under perform or they over perform you've created double the skew for each person performing off expectation.

For it to work consistently you'd need everyone to be consistently valued at about 8.333%. That means reasonably close to each other in skill band.

So you'd either need to effectively solve for more variables than any matchmaker has done so far (not saying that isn't possible, just a lot of work) or you need to do what every modern game has done and accept that per match matchmaking is at best mediocre but overall player ranking works out Large Numbers style.

However, again, the model you've created, especially if PGI gives you more data, would give us significantly better matches than we have currently. Right now we don't really even have 'mediocre'. Mediocre would be miles better than what we have now. We could probably get a bit better than mediocre if you have more telemetry than you have now. If you were willing to do that for PGI it would cost them very little relative to the gain and extend the games life significantly. You could make a business case for it pretty easily.

[killzone1]

Ok, but more often than not simply splitting up teams first by skill level, then by weight, will help a lot. I've been seeing a ton of matches where it puts several top 100 players on a single side. Net effect, it's a roflstomp. Back when I was T4 and T3, I would even often see 3 or 4 cadets on one team and none on the other. Doesn't take a genius to know how that will turn out..... It's doing the same thing with T3-T1. I bet it's not even attempting to equalize the # of T1 folks on each team (forget about true ranking within T1).

The MM isn't just bad, it's downright horrific. Last 20 games have all been 12-3/3-12 or worse win or losses.

Edited by killzone1, 12 February 2020 - 10:26 AM.

[Brauer]

killzone1, on 12 February 2020 - 09:05 AM, said:

Ok, but more often than not simply splitting up teams first by skill level, then by weight, will help a lot. I've been seeing a ton of matches where it puts several top 100 players on a single side. Net effect, it's a roflstomp. Back when I was T4 and T3, I would even often see 3 or 4 cadets on one team and none on the other. Doesn't take a genius to know how that will turn out..... It's doing the same thing with T3-T1. I bet it's not even attempting to equalize the # of T1 folks on each team (forget about true ranking within T1).

The MM isn't just bad, it's downright horrific. Last 20 games have all been 12-3/3-12 or worse win or losses.

For the Nth time, tiers mean essentially nothing in this game because given enough time horrible players reach tier 1. Tier 1 includes players in the 99.99th percentile, and players in the 50th percentile. The game does balance out the number of players from each tier on each side, but when on Team 1 your tier 1 player can be a multiple time world champ and Team 2 the tier 1 player can be a 50th percentile player with a sub-1.0 wlr and kdr balancing by tiers means zilch.

[General Solo]

Here's a pic that shows why and my very simple solution at requires a reset.
Population at present maybe an issue.

[HurtySanchez]

The fundamental problem is it doesn't really matter what tier your in if it's going to group T5-T3 and T1-T3 together in games. That is just too wide of a swing. It's gotta be T5-T4, T4-T3, T2, T1.

[Brauer]

HurtySanchez, on 15 February 2020 - 10:50 AM, said:

The fundamental problem is it doesn't really matter what tier your in if it's going to group T5-T3 and T1-T3 together in games. That is just too wide of a swing. It's gotta be T5-T4, T4-T3, T2, T1.

No, the fundamental problem is the tier bar is just an xp bar. If you play enough and aren't so abjectly terrible that you drive constant losses even at T5 you will eventually be T1. Until there is a zero sum matchmaker (which won't happen) the matchmaker will be bad.

[VonBruinwald]

OZHomerOZ, on 12 February 2020 - 07:16 PM, said:

Here's a pic that shows why and my very simple solution at requires a reset.

Matchscore needs to be dynamic.

Take the highest players MS as the Maximum increase and the mean of all players as the gradual/no change boundary.

A common complaint of stomps is there's less dmg/MS to go around. This accounts for that.

[Voice of Kerensky]

Match Maker in quick play - total garbage. Sorry.
I often get furious during the game. It’s a mystery for me why I am constantly thrown into the team of losers who cannot do more than 200 damage, and my team regularly loses with a score of 12-3, at best, 12-6. There was a case, out of 13 fights held in the evening, my team won only 1 time.
Yesterday, in one fight in my team, two people could not do 100 damage, another six people could not do 200 damage. Only 4 people were able to do something. Repeatedly there are cases that in the first two (!) minutes of the battle in a team 4 people die.
Is this the balanced selection of opponents? Such a game does not bring pleasure, only disappointment and anger.
Why is emphasis placed only on tonnage and pilot skill rating (players tier)? Why is the selection of opponents not taking into account the statistics of players on the ratings of "killed / died", "won / lost"? This statistic are, This statistic are available to any player. One gets the impression that these statistic are available to absolutely everyone except PGI.

P.S. I apologize for this cry of the soul.

[Brauer]

Voice of Kerensky, on 15 April 2020 - 06:26 PM, said:

Match Maker in quick play - total garbage. Sorry.
I often get furious during the game. It’s a mystery for me why I am constantly thrown into the team of losers who cannot do more than 200 damage, and my team regularly loses with a score of 12-3, at best, 12-6. There was a case, out of 13 fights held in the evening, my team won only 1 time.
Yesterday, in one fight in my team, two people could not do 100 damage, another six people could not do 200 damage. Only 4 people were able to do something. Repeatedly there are cases that in the first two (!) minutes of the battle in a team 4 people die.
Is this the balanced selection of opponents? Such a game does not bring pleasure, only disappointment and anger.
Why is emphasis placed only on tonnage and pilot skill rating (players tier)? Why is the selection of opponents not taking into account the statistics of players on the ratings of "killed / died", "won / lost"? This statistic are, This statistic are available to any player. One gets the impression that these statistic are available to absolutely everyone except PGI.

P.S. I apologize for this cry of the soul.

The problem isn't really the matchmaker, it's the non-zero-sum tier system.

Technically, tiers already consider wins, losses, and kills because those all impact tier rating, but the tier rating means nothing. Adding more variables won't necessarily make balance better, I'd argue we just need a tier system built to produce more balanced matches so the variables aren't garbage to begin with.

The examples of bad play you cite aren't even really among the worst in QP right now with the declining size and skill of the playerbase. Just focus on your own play. PGI isn't doing anything to MWO aside running events at this point and your own play is all you can control. Best of luck in the PUG lottery.

[Dee Eight]

It would be interesting to see if they eliminated the tiers from the matchmaker altogether and just balanced by mech weight classes what would happen. It couldn't really make it any worse than it already is since we're already at the earth's core... anything from here would be going upwards...

[Brauer]

Dee Eight, on 16 April 2020 - 04:23 AM, said:

It would be interesting to see if they eliminated the tiers from the matchmaker altogether and just balanced by mech weight classes what would happen. It couldn't really make it any worse than it already is since we're already at the earth's core... anything from here would be going upwards...

I mean treating players who are so bad that they're permanently in tier 4 or 5 the same as people who demonstrate the bare shred of competence to progress beyond tiers 4 and 5 would not be an improvement. It might not make much difference given the dismal state of the tier system, but it certainly wouldn't be an improvement.

[Sniper09121986]

Dee Eight, on 16 April 2020 - 04:23 AM, said:

It would be interesting to see if they eliminated the tiers from the matchmaker altogether and just balanced by mech weight classes what would happen. It couldn't really make it any worse than it already is since we're already at the earth's core... anything from here would be going upwards...

But that is exactly how it was before tier system. Oh, it was better than before the solo/group queue split, but it was not too good either. The tier system is broken, but at least it exists.

[Red Potato Standing By]

Wouldn’t it be easier just to base the tier system off of a players average match score ( logic being better players average score should be higher)so for example tier 1 players would need say an average of over say 400 or something like that and work your way downwards for each subsequent tier. It would prevent players from being in a tier they shouldn’t be in.

[General Solo]

Disclaimer: I'm talking about the current system and a minimum effort cheap solution to the current system.)

Problem is the Match score to PSR movement conditions are, now this is the important part imo
different for wins and loses.

Now a player may contribute to wins and loses to various degrees but its the TEAMS that ultimately determine who wins and losses as no player can 12 v1 360 no scope win a game. Mostly!

So when Match score to PSR movement conditions are, different for wins and loses, not only is the players contribution used to determine PSR movement, so is the teams contribution (winning or losing) used to determine PSR movement for the player which is ultimately used to determine individual player skill not team.

So MM is using the player contribution to the match plus the teams contribution to the match to measure an individual players skill and that this imo is why players get carried upward (bias) in terms of PSR.

For example and pardon my wordage, I dont have a thesaurus
:A winner gets 100MS and a loser gets 100MS, they both contributed 100 MS to the game and in terms of skill they contributed the same as far as we can tell because as far as MM is concerned MS is how skill is measured.

In the current system the winner and loser are treated differently in terms of PSR yet they are equally skilled (bear wid me here).
The winner get favorable treatment in terms of PSR movement and this causes the upward bias.

Sure reward the 100MS winner moar than the 100MS loser.
Give them CB, XP but don't reward them with PSR as their individual skill contributed exactly 100MS same as the 100MS loser.

So they should get the same PSR movement.

So imo the ez minimal effort fix to the current MM system is make PSR conditions for the winners the same as it is for the losers PSR conditions.

This way even winners can drop PSR depending on the players individual contribution to the win.

If you look at the graphic below can you say that Mr 7, Mr 15 and Mr 20 damage on the winners team did not deserve a PSR drop. And thats where the problem lies.
One can lie to MM, but one can't lie to skillgap. Get wrekt or Get Guid or fix MM and reset it please PGI.

Edited by OZHomerOZ, 16 April 2020 - 10:32 PM.

[Jochi Kondur]

@Nightbird, could you log transform or square-root transfer distribution charts displaying positive and negative skews so that they are more easily interpreted by people who don't work with or look at asymmetrical charts and are only familiar with how a normal symmetrical distribution "bell curve" looks like.

[Jochi Kondur]

Nightbird, on 11 June 2019 - 08:19 AM, said:

The rude part is obvious, so I'll point out the idiotic part again to rub it in. I used the normal distribution for my hidden skill factor, Dr. DG here see that match score on Jarl's list is normally distributed as well, and proclaims, both are normally distributed, therefore MS is a good representative of skill! I point out that height and weight are also normally distributed, so they must represent MWO skill as well per DG

Der Geisterbaer, on 11 June 2019 - 05:36 AM, said:

Food for thought on that: Take a closer look at some of the premises made for the simulation of the current MM. Nightbird explicitly used a "hidden" normal distribution of skill as the MM supposedly cannot discern skill levels directly. But if you take a look at the global stats that the Jarl's List provides for "Average Match Score" you'll find that the graph derived from all data points on that appears to be a normal distribution as well. So the available stats do seem to reflect "skill" quite nicely and guess what? Average match score quite obviously is not the same as W/L ratio that Nightbird tries to present as "ultima ratio" in terms of suitable metrics.

I'll leave it up to you to take from that whatever you like but from a "scientific" standpoint Nightbird should certainly assess his own level of confirmation bias and other potential sources of systematic error in what he presented.

The often cited Jarl's charts on Average Match Scores and W/L Ratio are both positively skewed distributions but are being interpreted as though they were symmetrically distributed (mode=median=mean) when in fact the median and mean are pushed to the right (i.e. mode < median < mean). The cited Jarl's charts only give the meaningful fact that they are positively skewed distributions with a readily apparent mode value. However, critical information on the median, means (arithmetic mean, geometric mean and harmonic mean), standard deviation, coefficients of skewness, kurtosis, etc were missing.