Jump to content

Nightbird's Match Maker Summary


72 replies to this topic

#21 Belorion

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Legendary Founder
  • Legendary Founder
  • 5,469 posts
  • LocationEast Coast

Posted 16 September 2021 - 11:48 AM

Not a fan of this approach. I would much rather the long over due adjustments to how match score are calculated as damage currently contributes too much to match score.

#22 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 16 September 2021 - 12:28 PM

View PostBelorion, on 16 September 2021 - 11:48 AM, said:

Not a fan of this approach. I would much rather the long over due adjustments to how match score are calculated as damage currently contributes too much to match score.


And I eagerly wait for a detailed analysis of how to adjust the match score calculation and the benefits to match making it will have.

#23 Kotis77

    Member

  • PipPipPipPipPipPip
  • General
  • General
  • 245 posts

Posted 16 September 2021 - 01:48 PM

Quality of QP matches are all time low atm. 12-0 matches happening too often. Need to something. Either this thingy or at least soft reset on PSR

#24 kapusta11

    Member

  • PipPipPipPipPipPipPipPipPip
  • Little Helper
  • Little Helper
  • 3,854 posts

Posted 17 September 2021 - 12:41 PM

MWO tier distribution is linear. Actual skill distribution is exponential, which means Tier 1 is overcrowded and it leads to top 1-5% players being randomly stacked on one side or the other. Groups of exteremely good or bad players also make such stacking more likely to happen.

Take Jarl's list modified match score and change tier distribution to the following:
Top 1% - Tier 1
Top 5% - Tier 2
Top 20% - Tier 3
Top 50% - Tier 4
Buttom 50% - Tier 5

#25 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 17 September 2021 - 01:25 PM

View Postkapusta11, on 17 September 2021 - 12:41 PM, said:

MWO tier distribution is linear. Actual skill distribution is exponential, which means Tier 1 is overcrowded and it leads to top 1-5% players being randomly stacked on one side or the other. Groups of exteremely good or bad players also make such stacking more likely to happen.

Take Jarl's list modified match score and change tier distribution to the following:
Top 1% - Tier 1
Top 5% - Tier 2
Top 20% - Tier 3
Top 50% - Tier 4
Buttom 50% - Tier 5


Skill is usually a bell curve, like most thing in life actually

https://leaderboard.isengrim.org/stats

#26 GoodTry

    Member

  • PipPipPipPipPipPip
  • 268 posts

Posted 17 September 2021 - 09:00 PM

View Postkapusta11, on 17 September 2021 - 12:41 PM, said:

MWO tier distribution is linear. Actual skill distribution is exponential, which means Tier 1 is overcrowded and it leads to top 1-5% players being randomly stacked on one side or the other. Groups of exteremely good or bad players also make such stacking more likely to happen.

Take Jarl's list modified match score and change tier distribution to the following:
Top 1% - Tier 1
Top 5% - Tier 2
Top 20% - Tier 3
Top 50% - Tier 4
Buttom 50% - Tier 5


I agree with all of the others that say that Jarl's is not even close to a 1:1 representation of skill, but I think this is better than any other idea I've seen. I like it a lot.

Better still would be chuck the tier system entirely when it comes to secondary match balancing, and once a match is assembled just balance the two teams based on something equivalent to jarl's list percentage, since it is not capped like PSR. Just rank players by percentage and build the teams schoolyard-pick style (highest goes to team 1, second highest to team 2, third to team 1, etc). But I understand there are constraints that prevent such a system.

You'd actually get similar results from just uncapping PSR, making >5000 remain tier 1, and then just use raw PSR for balancing.

It just doesn't work unless it is not capped.

#27 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 17 September 2021 - 10:44 PM

View PostGoodTry, on 17 September 2021 - 09:00 PM, said:

I agree with all of the others that say that Jarl's is not even close to a 1:1 representation of skill, but I think this is better than any other idea I've seen. I like it a lot.

Better still would be chuck the tier system entirely when it comes to secondary match balancing, and once a match is assembled just balance the two teams based on something equivalent to jarl's list percentage, since it is not capped like PSR. Just rank players by percentage and build the teams schoolyard-pick style (highest goes to team 1, second highest to team 2, third to team 1, etc). But I understand there are constraints that prevent such a system.

You'd actually get similar results from just uncapping PSR, making >5000 remain tier 1, and then just use raw PSR for balancing.

It just doesn't work unless it is not capped.


You won't get anything from uncapping the current PSR because players will simply be rocketing up to infinity. What's important is that people stabilize at a certain PSR numbers that can be used to compare to other players. MS based PSR doesn't do that.

Jarl's list is not a good source either. If Jarl's list is based on skill, and we can agree that more skill = more winning, then you can go down Jarl's list and see that WLR doesn't decrease, it jumps up and down. Jarl's list only measures how good a person is at farming match score, no more no less.

Edited by Nightbird, 17 September 2021 - 10:53 PM.


#28 MrMadguy

    Member

  • PipPipPipPipPipPipPipPipPip
  • 2,193 posts

Posted 17 September 2021 - 11:44 PM

View PostVerilligo, on 16 September 2021 - 03:39 AM, said:

Carrying exists, but is a short-term phenomena. Over 300 games the effect of carrying will be negligible, not only because you have gone through that many games, but because everyone around you will have gone through that many games. As an example, you will end up paired more often with players that have a WLR of 1.0 versus those with a WLR of 2.0 or higher. Keep in mind, also, that Nightbird has already admitted this does not fully eliminate the chance of ending up in an unbalanced match on one side or the other. But going from a 27% rate to an 8% rate is a massive, massive difference.

No. I have had enough of this back in PSR XP Bar era. If WLR based MM would work, my WLR would drop due to playing at exceeding ratings due to PSR's bias towards increasing. I played at exceeding rating not even for weeks and months - for years! But it didn't happen. And therefore WLR MM isn't viable. At least now I'm able to drop one Tier after series of terrible matches and get rid of that Tier 1 guys in my matches. And this is just perfect. Nothing more is needed.

Problems with imbalanced matches are mostly caused by too drastic changes in population. Population increases during events and drops between them. This changes distributions of skills, as skill is relative thing and loss of playerbase part causes balance shift. Ratings just can't catch up quick enough. Another possible problem - release valves, that are used to fill gaps in Tier 1 matches. MM doesn't take into account, that equal ratings of low Tier players, i.e. fact, that they're equal against each other, don't guarantee, that they would be equal against Tier 1 guys. Only solution - more dynamic MM. Something like AI.

Edited by MrMadguy, 17 September 2021 - 11:48 PM.


#29 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 18 September 2021 - 09:47 AM

View PostMrMadguy, on 17 September 2021 - 11:44 PM, said:

No. I have had enough of this back in PSR XP Bar era. If WLR based MM would work, my WLR would drop due to playing at exceeding ratings due to PSR's bias towards increasing.


WLR stabilizes. As someone whose WLR is around 1.1, you'll be in the middle of Tier 3. Only people with more severe WLR differences from 1 will be pulled towards the middle as they get to play more fair matches. Both the low end and the high end.

#30 Brauer

    Member

  • PipPipPipPipPipPipPipPip
  • 1,066 posts

Posted 18 September 2021 - 11:00 AM

View PostNightbird, on 16 September 2021 - 08:10 AM, said:


There isn't a change to Tiers, so everyone with PSR >= 4000 (WLR>=1.6) will be in Tier 1. There's no change to MM gates, so we'll see Tiers 1 and 2 combined, or 1, 2, and 3 during low pop hours. There's no change to anything other than the way the PSR value is calculated.

As for groups, the only change under the new system is there is no PSR ceiling, so if they have PSR=20,000 in a group of 4, the MM will put the best 12 of the remaining 20 players against them due to existing secondary lobby balancing based on PSR numbers.


I have heard quite knowledgeable members of the community day that there is no secondary lobby balancing at this time. Do you have any proof that it exists?

FWIW I'm all for w/l based matchmaking. I don't see the point in doing much to adjust the match score kicker values.

#31 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 18 September 2021 - 11:11 AM

View PostBrauer, on 18 September 2021 - 11:00 AM, said:

I have heard quite knowledgeable members of the community day that there is no secondary lobby balancing at this time. Do you have any proof that it exists?

FWIW I'm all for w/l based matchmaking. I don't see the point in doing much to adjust the match score kicker values.


https://mwomercs.com...overies-week-1/

Secondary lobby balancing - how the MM moves people around after the gates allow 24 people into a lobby - is how I interpret Paul's post here. Note at the bottom of the post, Paul says "Average Tier separation between teams: ~300 (tightened from ~1800)" shows that when Paul says balancing for Tier, he means balancing for PSR since you can't possibly have a difference of 300 from Tier levels.

Edited by Nightbird, 18 September 2021 - 11:11 AM.


#32 Brauer

    Member

  • PipPipPipPipPipPipPipPip
  • 1,066 posts

Posted 18 September 2021 - 11:59 AM

View PostNightbird, on 18 September 2021 - 11:11 AM, said:


https://mwomercs.com...overies-week-1/

Secondary lobby balancing - how the MM moves people around after the gates allow 24 people into a lobby - is how I interpret Paul's post here. Note at the bottom of the post, Paul says "Average Tier separation between teams: ~300 (tightened from ~1800)" shows that when Paul says balancing for Tier, he means balancing for PSR since you can't possibly have a difference of 300 from Tier levels.


I wonder if this is just a function of how wide of a bucket the MM is pulling from to populate the teams. If the MM was just pulling from the T1 PSR bucket the disparity could be tighter anyway than if it was pulling from three PSR buckets. It'd be helpful for PGI to clarify this a bit as I have often heard the lack of secondary lobby balancing identified as key MM problem and a clear answer on whether or not this is true would be helpful.

Anyway, yes to making PSR about what actually wins games and not about whether or not someone pads their match score sufficiently.

#33 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 18 September 2021 - 12:39 PM

I do think that PGI has been pretty clear about how Tiers work, the MM tries to make a match with only players from one Tier, and then opens it up to 2 or 3 Tiers based on how much time has passed. Also keep in mind that it doesn't make sense to look at tonnage until you have all 24 players selected.

Regardless, I think any MM talk is moot unless we have another pop booster as big as COVID to get people into the game. As I said a while back, a good MM only better retains players. You need other things to bring people in. We've pretty much exhausted all the low hanging fruit from Cauldron updates, and the COVID population boost is tapering, so fixing the holes in the bottom of the bucket now is far less effective as we've no more water to fill it afterwards.

#34 kapusta11

    Member

  • PipPipPipPipPipPipPipPipPip
  • Little Helper
  • Little Helper
  • 3,854 posts

Posted 19 September 2021 - 01:17 AM

View PostNightbird, on 17 September 2021 - 01:25 PM, said:

Skill is usually a bell curve, like most thing in life actually

https://leaderboard.isengrim.org/stats


You're missing the point. Someone who's in the bottom 20% and in the top 80% are closer to the middle in terms of their skill than someone in the top 1%. K/D or W/L ratio of 0.5 and 1.5 are closer to the average of 1 than KD ratio of 3+.

I'll use groups to illustrate what I'm talking about, but this will randomly happen with no groups present because matchmaker can't differentiate between player performance within one tier

A group of top 0.5% players ------------------------A group of top 80% players
4x performance of an average player-------------2x performance of an average player
4x performance of an average player-------------2x performance of an average player
4x performance of an average player-------------2x performance of an average player
4x performance of an average player-------------2x performance of an average player

16x total vs 8x. Team 2 is roughly 4 top 80% or 2 top 0.5% players down.

There's nothing in the current matchmaking that will stop this scenario from happening. It sets people up for failure and is one of the reason why they leave the game.

Tier distribution should be biased heavily towards the top. Tiers also should be fixed based on your ranking. Use the average of the previous 3 seasons to rank players. Skill doesn't increase or decrease over the course of one game or dozen. It takes months to improve or get seriously rusty. In fact, most players will just reach a certain level and then stay there.

Edited by kapusta11, 19 September 2021 - 01:20 AM.


#35 MrMadguy

    Member

  • PipPipPipPipPipPipPipPipPip
  • 2,193 posts

Posted 19 September 2021 - 01:19 AM

View PostNightbird, on 18 September 2021 - 09:47 AM, said:

WLR stabilizes. As someone whose WLR is around 1.1, you'll be in the middle of Tier 3. Only people with more severe WLR differences from 1 will be pulled towards the middle as they get to play more fair matches. Both the low end and the high end.

No way. Current quality of matches is terrible? Stomps and deathball matches? Ok. Just around 10 such matches and I'm in Tier 4, where quality of matches is much better. And if MM would be WLR based, it would take ages to achieve it. Because my WLR hasn't changed even a little bit during that time. Zero-summ MS-based MM with little WLR bias - IS PERFECT. Nothing else is needed. Problem is most likely with team-forming algorithm itself - not with PSR rating system. It really seems like it has some fundamental flaw, like not being able to distribute players between teams properly, when release valves are opened. I.e. like if 1,5 teams are filled properly and then 0,5 gap in one team is filled with "cannon fodder" guys from Tier 3.

#36 Rkshz

    Member

  • PipPipPipPipPipPipPipPipPip
  • 2,866 posts
  • Twitch: Link
  • LocationOdesa, Ukraine

Posted 19 September 2021 - 09:28 AM

View PostNightbird, on 18 September 2021 - 12:39 PM, said:

I do think that PGI has been pretty clear about how Tiers work, the MM tries to make a match with only players from one Tier, and then opens it up to 2 or 3 Tiers based on how much time has passed. Also keep in mind that it doesn't make sense to look at tonnage until you have all 24 players selected.

Regardless, I think any MM talk is moot unless we have another pop booster as big as COVID to get people into the game. As I said a while back, a good MM only better retains players. You need other things to bring people in. We've pretty much exhausted all the low hanging fruit from Cauldron updates, and the COVID population boost is tapering, so fixing the holes in the bottom of the bucket now is far less effective as we've no more water to fill it afterwards.

I think that no one understands how MM works
if the PGI wanted to improve MM, then they would open the current formula for the community

btw, how are you doing with the new formula? will there be only conversations or will PGI integrate it?

#37 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 19 September 2021 - 09:35 AM

View Postkapusta11, on 19 September 2021 - 01:17 AM, said:


You're missing the point. Someone who's in the bottom 20% and in the top 80% are closer to the middle in terms of their skill than someone in the top 1%. K/D or W/L ratio of 0.5 and 1.5 are closer to the average of 1 than KD ratio of 3+.


I don't understand your post. Are you talking about the current PSR or WLR PSR? If WLR PSR, you just need to think about it some more since it already does what you're saying but is much better at it than your suggestion.

#38 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 19 September 2021 - 09:39 AM

View PostMrMadguy, on 19 September 2021 - 01:19 AM, said:

Zero-summ MS-based MM with little WLR bias - IS PERFECT.


You're entitled to your opinion, but I think 32% of all games being formed with balanced team is abysmal. It's the kind of grade where you're forced to hire a private tutor for your son/daughter. 51% is still a failing grade. I could build a PSR/MM with 95% of all matches being made with balanced teams, with no increase in wait time and with the current small population, but it is out of scope for even the previous effort PGI was willing to make. It's ironic because if they had my MM from the beginning of MWO this game would have 10-50x the population today because of how retention rates correspond very well to match making.

#39 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 19 September 2021 - 09:43 AM

View PostRkshz, on 19 September 2021 - 09:28 AM, said:

Btw, how are you doing with the new formula? will there be only conversations or will PGI integrate it?


I created this thread because there was a short talk about adding it to the Cauldron's wish list, I would put the chances at 0 though. PGI never understood match making or retention rates, they didn't even read Jay Z's proposal (the first post in the thread they created about seeking community proposals) when they shared their "options". Core 2C anyone?

#40 Rkshz

    Member

  • PipPipPipPipPipPipPipPipPip
  • 2,866 posts
  • Twitch: Link
  • LocationOdesa, Ukraine

Posted 19 September 2021 - 10:10 AM

View PostNightbird, on 19 September 2021 - 09:43 AM, said:

I created this thread because there was a short talk about adding it to the Cauldron's wish list, I would put the chances at 0 though. PGI never understood match making or retention rates, they didn't even read Jay Z's proposal (the first post in the thread they created about seeking community proposals) when they shared their "options". Core 2C anyone?

very sorry, good MM balance formula is the most important for all players (especially for beginners)
I perfectly understand that my idea has zero chance, but I wanted to shake up the community - I hoped that you and JayZ having more authority, could break this wall





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users