72 replies to this topic

[Nightbird]

This thread is intended to pull together a few of my topics on improving the MWO Match Maker. The first half is in plain English, explaining how it works and the benefits. The second half uses programming pseudo-code and is intended to help implement the idea.

My improvement to the match maker is simply to use WLR as PSR, skipping match score (MS) altogether. This is because a multivariate analysis on MWO leaderboard data shows that match score is bad at predicting performance compared with win rate. We also can't get the necessary data from PGI to set kickers for MS correctly.

The outcome of using the WLR match maker was obtained by running a simulation. Some of you will say here, a simulation is different from real life. It is different but it can be accurate. The simulation I ran over a year ago managed to accurately predict the outcome of the current match maker (implemented 2020-07) nearly perfectly. The same simulation predicts WLR will increase the number of games with balanced teams* from 32% of all games to 51%. More importantly, it will reduce the number of games with horribly unbalanced teams** from 27% of all games down to 8%.

* creates teams with a win chance of 40-60%
** creates teams with a win chance of 0-15% or 85-100%

------------------------------------------------------------------------------
Technobabble starts here, you've been warned.

We know from asking PGI that player stats (i.e. true WLR) is not accessible by the PSR system, therefore my solution is to use a rolling average. Currently, the PSR is calculated in 3 steps.

1. Match ends, and the match server applies Jay Z's PSR formula and calculates and sends a Delta_PSR for each player to the PSR server
2. The PSR server calculates for each player NEW_PSR = OLD_PSR + Delta_PSR
3. Round to nearest integer. If NEW_PSR is less than 0, set to 0, if NEW_PSR is greater than 5000, then set to 5000

With a WLR PSR system, this becomes:
1. Match ends, match server sends RESULT for each player to the PSR server, where RESULT=1 for a win, 0 for a loss.
2. The PSR server calculates for each player the NEW_PSR, where NEW_PSR = 2500 * {299 * [(OLD_PSR/2500)/((OLD_PSR/2500)+ 1)] + RESULT} / {299* [1 - (OLD_PSR/2500)/((OLD_PSR/2500)+ 1)] - RESULT + 1}
3. Round to nearest integer. If NEW_PSR is less than 0, then set to 0, if NEW_PSR is greater than 5000, DO NOT set to 5000.

This formula does a rolling average based on the past 300 games, which is not as good as true WLR but is good enough under the constraints***. Overall, step 1 becomes much simpler and step 2 becomes a little more complicated.

To implement this system, we do not need a reset of PSR. Change the formula for calculating the further changes to PSR, but people can keep their current PSR number. This prevents the chaos of a PSR reset.

New players should be set to PSR=2000.

------------------------------------------------------------------------------
***The proposal here is made under the same scope of work allowed as the previous PSR Match Maker update. There are better systems possible, but much more work is needed. My only claim is that the benefits of the WLR Match Maker are as described at the beginning of this post, with proof from the simulations of both this system and Jay Z's system.

[w0qj]

Amazing!
Thank you for your extremely detailed analysis, and with graphs too!

By the way, have you thought of going into the field of financial analysis?
(You know, those 'rocket' folks...)

[Rkshz]

Nightbird, on 15 September 2021 - 06:43 PM, said:

This thread is intended to pull together a few of my topics on improving the MWO Match Maker. The first half is in plain English, explaining how it works and the benefits. The second half uses programming pseudo-code and is intended to help implement the idea.............

good job, it's better and more honest than what is now
but I have only one question - why complicate things?
if WLR is the most objective indicator of a player's skill, then why should WLR be tied to the PSR?
it is possible to make it easier for MM to look and balance teams for WLR

Edited by Rkshz, 15 September 2021 - 10:12 PM.

[MrMadguy]

WLR doesn't take carrying into account. Period. And carrying phenomena ACTUALLY EXISTS, as it was happening with me a lot, when old PSR exp bar was in place. What is carrying? Is when you have terrible matches, stomps, dying after 1 minute of playing with <100 dmg. But all of a sudden your WLR doesn't change, because your team can easily win even without you due to high skill gap between you and them.

[Davegt27]

define zero sum I can't remember ?

and why would you want zero sum ?

[Saved By The Bell]

I dont understand because its complicated, but I think new system should be at least tried. We can roll back anytime.

[McGoat]

Good post.

[Verilligo]

MrMadguy, on 15 September 2021 - 10:19 PM, said:

WLR doesn't take carrying into account. Period. And carrying phenomena ACTUALLY EXISTS, as it was happening with me a lot, when old PSR exp bar was in place. What is carrying? Is when you have terrible matches, stomps, dying after 1 minute of playing with <100 dmg. But all of a sudden your WLR doesn't change, because your team can easily win even without you due to high skill gap between you and them.

Carrying exists, but is a short-term phenomena. Over 300 games the effect of carrying will be negligible, not only because you have gone through that many games, but because everyone around you will have gone through that many games. As an example, you will end up paired more often with players that have a WLR of 1.0 versus those with a WLR of 2.0 or higher. Keep in mind, also, that Nightbird has already admitted this does not fully eliminate the chance of ending up in an unbalanced match on one side or the other. But going from a 27% rate to an 8% rate is a massive, massive difference.

[Mochyn Pupur]

A fine piece of empirical analysis . . . 20% improvement in MM quality is nothing to be sniffed about.

[pbiggz]

WLR is not taken into account in PSR for a good reason. The old matchmaker DID take it into account and it was nothing but whining from all of you ghouls.

[GoodTry]

It's not clear from your post whether this is replacing the tier system or not.

[Nightbird]

pbiggz, on 16 September 2021 - 04:37 AM, said:

WLR is not taken into account in PSR for a good reason. The old matchmaker DID take it into account and it was nothing but whining from all of you ghouls.

The old old match maker (before PSR) used the ELO system for individual players, but the problem is that the ELO system is made for static teams with fixed rosters, not random teams. That's why that particular system bombed, I ran the math on it and people's ELO's were going off to infinity or minus infinity.

GoodTry, on 16 September 2021 - 06:22 AM, said:

It's not clear from your post whether this is replacing the tier system or not.

It does not. It doesn't change much because of the scope of work. After running it for a while, it may be necessary to change the boundaries of each Tier depending on where players end up. For example instead of 2000-3000 for Tier 3, it may be better to use 2250-2750. There's no hurry to do that.

[Nightbird]

Rkshz, on 15 September 2021 - 10:11 PM, said:

but I have only one question - why complicate things?
if WLR is the most objective indicator of a player's skill, then why should WLR be tied to the PSR?
it is possible to make it easier for MM to look and balance teams for WLR

PSR is a system in place - the UI, Tiers. I am using PSR = 2500*WLR. It's not much different and prevents PGI from having to changes the UI and Tiers as well.

Edited by Nightbird, 16 September 2021 - 07:32 AM.

[Rkshz]

Nightbird, on 16 September 2021 - 07:32 AM, said:

PSR is a system in place - the UI, Tiers. I am using PSR = 2500*WLR. It's not much different and prevents PGI from having to changes the UI and Tiers as well.

explain to me, on what basis will MM search for a group?
for example, if PGI integrate your formula, where will the players with WLR 3+ and how will MM put together a group for them??
what is the principle?
the formula that I suggested would just sort the players into three groups, but skilled players would never intersect with newbiers - will your formula do the same?
what will be the minimum allowable ceiling for admission to play with WLR 3+

Edited by Rkshz, 16 September 2021 - 08:02 AM.

[Nightbird]

Rkshz, on 16 September 2021 - 07:59 AM, said:

explain to me, on what basis will MM search for a group?
for example, if PGI integrate your formula, where will the players with WLR 3+ and how will MM put together a group for them??
what is the principle?
the formula that I suggested would just sort the players into three groups, but skilled players would never intersect with newbiers - will your formula do the same?
what will be the minimum allowable ceiling for admission to play with WLR 3+

There isn't a change to Tiers, so everyone with PSR >= 4000 (WLR>=1.6) will be in Tier 1. There's no change to MM gates, so we'll see Tiers 1 and 2 combined, or 1, 2, and 3 during low pop hours. There's no change to anything other than the way the PSR value is calculated.

As for groups, the only change under the new system is there is no PSR ceiling, so if they have PSR=20,000 in a group of 4, the MM will put the best 12 of the remaining 20 players against them due to existing secondary lobby balancing based on PSR numbers.

Edited by Nightbird, 16 September 2021 - 08:11 AM.

[Rkshz]

Nightbird, on 16 September 2021 - 08:10 AM, said:

There isn't a change to Tiers, so everyone with PSR >= 4000 (WLR>=1.6) will be in Tier 1. There's no change to MM gates, so we'll see Tiers 1 and 2 combined, or 1, 2, and 3 during low pop hours. There's no change to anything other than the way the PSR value is calculated.

As for groups, the only change under the new system is there is no PSR ceiling, so if they have PSR=20,000 in a group of 4, the MM will put the best 12 of the remaining 20 players against them due to existing secondary lobby balancing based on PSR numbers.

poor knowledge of English did not allow me to fully understand your proposal - but now i get it, thanks

and I realized that nothing will change, because the problem is not in the TIER and PSR system - the problem is that MM does not form groups correctly - the problem is in the current formula, because the system does NOT have the correct assessment of the skill of the players

even if you to tie WLR to PSR, then nothing will change, because the MM formula will remain the same ****

you tried to kill my idea, which was supposed to reverse the current system (in which a player with an WLR of less than 1.0 will never become T1 and will never play with T1 players), but in fact, your idea is the same as it is now

chaos will continue, nothing will change, newbies will play with old farts, and stomps will remain
but I wish you the best of luck - because any change is better than nothing

Edited by Rkshz, 16 September 2021 - 08:26 AM.

[Nightbird]

Rkshz, on 16 September 2021 - 08:22 AM, said:

poor knowledge of English did not allow me to fully understand your proposal - but now i get it, thanks

and I realized that nothing will change, because the problem is not in the TIER and RPS system - the problem is that MM does not form groups correctly, because the system does NOT have the correct assessment of the skill of the players

you tried to kill my idea, which was supposed to reverse the current system (in which a player with an WLR of less than 1.0 will never become T1), but in fact, your idea is the same as it is now

chaos will continue, nothing will change, newbies will play with old farts, and stomps will remain
but I wish you the best of luck - because any change is better than nothing

I never said it will all be completely fixed. Going from 32% to 51% good matches is not the same as going to 100% good matches. Going from 27% to 8% bad matches is not the same as going to 0% bad matches. Still, it is progress. If more resources were available, better changes are possible. The only way this proposal is different from other proposals is you are transparently presented what you're getting in return.

[Nightbird]

___, on 16 September 2021 - 09:12 AM, said:

i dont know... to me it just seems like a sneaky way for you comp players to farm us casuals even harder. This isnt CSGO or LoL...

Without an example of how it can do that, it's hard to reply.

As an example of how WLR PSR helps: Today, once you've reached max PSR (PSR=5000) you can't move up anymore. It's easy to reach the max PSR with a WLR of 1.2, and once you are there, you are not differentiated from people with WLR=20 since they also have a PSR=5000. Then you get farmed by them.

[Nightcrept]

Thank you for your well thought out post. I for one support your idea.
It can't hurt to try it. lol

[RAM]

ELO > xp


RAM
ELH