Jump to content

How Do We Make The Middle Tier Better?


155 replies to this topic

#101 DaZur

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 7,511 posts
  • LocationWisconsin

Posted 11 April 2022 - 11:41 AM

View PostSFC174, on 11 April 2022 - 10:26 AM, said:

Ultimately WLR is the only way that we can fix the MM given the limited resources. Yes, it'll take 100+ games for you to get sorted properly, but over a large enough sample size WLR is the best predictor of future success.

Actually W/L is as bad a metric as match score or damage done... All in application are poor measures of an "individuals" skill level.

How many times have you personally dispatched 4 opponents and carried the team and still lost the match? How many times have you dispatched those 4 opponents and not exceeded 200 damage? How many times have you dispatched 4 opponents, not exceeded 200 damage and found your PSR go down?

The problem is this is a "team" game and your personal performance is greatly influenced by the quality of the team, the breath of their individual contribution and where you fit into that composite.

That said, there is no easy fix and no amount of trying to derive a meaningful individual "skill quotient" from any one or multiple team metrics will give a real palpable measurement.

Bottom line... What we have isn't perfect... Not even close. But it's what we got...

#102 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 April 2022 - 12:45 PM

View PostDaZur, on 11 April 2022 - 11:41 AM, said:

Actually W/L is as bad a metric as match score or damage done... All in application are poor measures of an "individuals" skill level.

How many times have you personally dispatched 4 opponents and carried the team and still lost the match? How many times have you dispatched those 4 opponents and not exceeded 200 damage? How many times have you dispatched 4 opponents, not exceeded 200 damage and found your PSR go down?

The problem is this is a "team" game and your personal performance is greatly influenced by the quality of the team, the breath of their individual contribution and where you fit into that composite.

That said, there is no easy fix and no amount of trying to derive a meaningful individual "skill quotient" from any one or multiple team metrics will give a real palpable measurement.

Bottom line... What we have isn't perfect... Not even close. But it's what we got...


You are entitled to your opinion, but at the same time what we have now is hot garbage because it is formed from unsubstantiated opinions rather than hard math.

#103 DaZur

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 7,511 posts
  • LocationWisconsin

Posted 11 April 2022 - 01:06 PM

View PostNightbird, on 11 April 2022 - 12:45 PM, said:

You are entitled to your opinion, but at the same time what we have now is hot garbage because it is formed from unsubstantiated opinions rather than hard math.

LOL."Hot garbage" might be a bit harsh but I can't really argue.

Math Is hard... especially when you are metaphorically trying to distil an individuals intrinsic value from metrics derived from a team game where the individuals value is largely dependent upon the success or failure of the team.

Edited by DaZur, 11 April 2022 - 01:07 PM.


#104 SFC174

    Member

  • PipPipPipPipPipPipPip
  • The Pharaoh
  • The Pharaoh
  • 695 posts

Posted 11 April 2022 - 01:12 PM

View PostDaZur, on 11 April 2022 - 11:41 AM, said:

Actually W/L is as bad a metric as match score or damage done... All in application are poor measures of an "individuals" skill level.

How many times have you personally dispatched 4 opponents and carried the team and still lost the match? How many times have you dispatched those 4 opponents and not exceeded 200 damage? How many times have you dispatched 4 opponents, not exceeded 200 damage and found your PSR go down?

The problem is this is a "team" game and your personal performance is greatly influenced by the quality of the team, the breath of their individual contribution and where you fit into that composite.

That said, there is no easy fix and no amount of trying to derive a meaningful individual "skill quotient" from any one or multiple team metrics will give a real palpable measurement.

Bottom line... What we have isn't perfect... Not even close. But it's what we got...


I used to think the same, but the math says otherwise, and it took a bit to convince me. Over a long enough time frame the only thing that determines your win loss ratio is you.

You can put together anecdotal situations about how you played with a high level of skill and still lost, but play enough games and you're going to get the crap teams and the great teams enough times that those factors are neutralized as a determinant of WLR. The drawback is that it takes a couple hundred games to get a WLR that is an accurate representation of your skill level. That's why I've always favored some sort of additional metric to help seed players, but hey, we all went through the matchmaker changes and tier reset so we can tolerate a couple hundred games of WLR settling to get better matchmaking.

The great thing about this is that it wouldn't be particularly hard to do. WLR is already present in the statistics and Nightbird put together a proposal for how to do it within the structure of PGI's existing architecture. He's put out a lot of data on this (probably have to go back to 2020 to find some of the early posts) that's worth reading if you're interested.

#105 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 April 2022 - 01:15 PM

View PostDaZur, on 11 April 2022 - 01:06 PM, said:

LOL."Hot garbage" might be a bit harsh but I can't really argue.

Math Is hard... especially when you are metaphorically trying to distil an individuals intrinsic value from metrics derived from a team game where the individuals value is largely dependent upon the success or failure of the team.


Math IS hard, and like anything else that is hard, you leave it to the pros. If you know that option B is 3x better than option A, no one sane would pick option A.

https://mwomercs.com...th-in-business/

Do you really think that Match Score is somehow a better representation of a person's skill? That is really the question is here, if MS is better than WLR then it is fine to use to calculate someone's PSR with. Unfortunately MS is terrible which is why PSR is terrible and the MM is terrible. The main reason I suggest WLR is because creating a good formulation for MS is 10x harder.

Edited by Nightbird, 11 April 2022 - 01:17 PM.


#106 pbiggz

    Member

  • PipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 4,826 posts
  • LocationOutreach

Posted 11 April 2022 - 01:31 PM

View PostNightbird, on 11 April 2022 - 01:15 PM, said:


Math IS hard, and like anything else that is hard, you leave it to the pros. If you know that option B is 3x better than option A, no one sane would pick option A.

https://mwomercs.com...th-in-business/

Do you really think that Match Score is somehow a better representation of a person's skill? That is really the question is here, if MS is better than WLR then it is fine to use to calculate someone's PSR with. Unfortunately MS is terrible which is why PSR is terrible and the MM is terrible. The main reason I suggest WLR is because creating a good formulation for MS is 10x harder.


You're passing off WLR as a silver bullet. Its not.

#107 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 April 2022 - 01:39 PM

View Postpbiggz, on 11 April 2022 - 01:31 PM, said:

You're passing off WLR as a silver bullet. Its not.


It's 3x better than the current MM. Backed up with math.

#108 Knownswift

    Member

  • PipPipPipPipPipPip
  • Ace Of Spades
  • 269 posts

Posted 11 April 2022 - 02:01 PM

View PostDaZur, on 11 April 2022 - 01:06 PM, said:

LOL."Hot garbage" might be a bit harsh but I can't really argue.

Math Is hard... especially when you are metaphorically trying to distil an individuals intrinsic value from metrics derived from a team game where the individuals value is largely dependent upon the success or failure of the team.


This game only resolves in three ways.

1. You win.
2. You lose.
3. You tie (both teams lose)

All emergent gameplay events and every "intrinsic" value a player can have is represented by the sole metric that matters; the win loss ratio.

There are no metaphors. It isn't subjective.

In your hypothetical situation; a player that regularly dispatches 4 players a game probably doesn't have a problem winning.

If you regularly carry that absolutely shows in the WLR. If you don't... well I think you get the idea.

Edited by Knownswift, 11 April 2022 - 02:09 PM.


#109 pbiggz

    Member

  • PipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 4,826 posts
  • LocationOutreach

Posted 11 April 2022 - 02:14 PM

View PostNightbird, on 11 April 2022 - 01:39 PM, said:

It's 3x better than the current MM. Backed up with math.


And its still not a ******* silver bullet.

#110 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 April 2022 - 02:21 PM

View Postpbiggz, on 11 April 2022 - 02:14 PM, said:


And its still not a ******* silver bullet.


Well, next time you get a job offer for 3x your current pay, make sure to use that as your reason to decline.

#111 pbiggz

    Member

  • PipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 4,826 posts
  • LocationOutreach

Posted 11 April 2022 - 02:54 PM

View PostNightbird, on 11 April 2022 - 02:21 PM, said:

Well, next time you get a job offer for 3x your current pay, make sure to use that as your reason to decline.


Non sequitur

#112 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 April 2022 - 03:02 PM

View Postpbiggz, on 11 April 2022 - 02:54 PM, said:


Non sequitur


Only showing the insanity of your previous statement.

#113 Tarl Cabot

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Tai-sho
  • Tai-sho
  • 7,822 posts
  • LocationImperial City, Luthien - Draconis Combine

Posted 11 April 2022 - 04:08 PM

View PostSFC174, on 11 April 2022 - 01:12 PM, said:


I used to think the same, but the math says otherwise, and it took a bit to convince me. Over a long enough time frame the only thing that determines your win loss ratio is you.

You can put together anecdotal situations about how you played with a high level of skill and still lost, but play enough games and you're going to get the crap teams and the great teams enough times that those factors are neutralized as a determinant of WLR. The drawback is that it takes a couple hundred games to get a WLR that is an accurate representation of your skill level. That's why I've always favored some sort of additional metric to help seed players, but hey, we all went through the matchmaker changes and tier reset so we can tolerate a couple hundred games of WLR settling to get better matchmaking.

The great thing about this is that it wouldn't be particularly hard to do. WLR is already present in the statistics and Nightbird put together a proposal for how to do it within the structure of PGI's existing architecture. He's put out a lot of data on this (probably have to go back to 2020 to find some of the early posts) that's worth reading if you're interested.


The issue is that when PGI asked for modifications to the present PSR system MM (after Solo/Group queue merge), PGI was not willing to change the MM up completely to a W/L setting. Nightbird had even twittered PGI and was shot down, essentially it was not within the requirement range that they were asking for. Coding is and has been difficult with PGI. Even with the current MM, there is no secondary consideration, it is just plain Tier+tonnage, and they really cannot get that close either...

And using a W/L MM, if a 4-man group has a high W/L, be aware that (chuckles) Sorry.. I do not believe PGI could get that coded correctly, but what would happen is that most of remaining 8 players would have the worse W/L record, while the other players on the other side would have the better W/L of the that selection, which would mean that 4-man would need to actually carry, or at least try to get the remaining 8 to actually work with them.

Elo selected players/groups within a set range of the seeded player. Per Paul, the MM alternated between teams. Still matching weight class and the groups were limited to 1 mech/weight class. Then once the teams were filled out, one team with the higher Elo average was the team expected to win. If they won their Elo went up very little, the losing team Elo numbers also dropped very little. It was when the expected winners lost that their Elo drop quite a bit, the expected losers went up quite a bit.

Then original PSR went live. It was actually okay at first, but PGI never revealed the actual PSR numbers. The players though had calculated when PSR > MS thresholds, and it ensured all but the lowest MS scoring players would eventually reach Tier 1 with enough time and games played.. Nothing like it is today. Some players, one with an avg 171 MS, 0.83 W/L ratio were able to reach Tier 2 and Tier 1 by blunt force games played. In late 2019 said player was in the top 10 most games, with at 25k games having been played. He had setup his web profile to show tiers, and he had gone from Tier 2 to Tier 1, then bounced back and forth between those tiers.

Could the MM be better? It could be different, even if PGI kept the current tier system for visible queues but utilized a hidden stat, ie W/L for the actual MM. But could PGI actually code that change effectively? Math is hard.. especially for PGI... :)

#114 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 April 2022 - 05:25 PM

View PostTarl Cabot, on 11 April 2022 - 04:08 PM, said:

The issue is that when PGI asked for modifications to the present PSR system MM (after Solo/Group queue merge), PGI was not willing to change the MM up completely to a W/L setting. Nightbird had even twittered PGI and was shot down, essentially it was not within the requirement range that they were asking for. Coding is and has been difficult with PGI. Even with the current MM, there is no secondary consideration, it is just plain Tier+tonnage, and they really cannot get that close either...


Their response showed they never read the suggestion, unless you really think that the current system can't return a 1 or 0 for each player based on win or loss.

It's not doomsday that the Match Maker sucks. As I showed in my graphs, all it means is that monthly retention rate is 10% worse than with the simple WLR PSR. If you know how to do compounding, that translates to a huge number of lost players and revenue.

#115 DaZur

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 7,511 posts
  • LocationWisconsin

Posted 11 April 2022 - 07:08 PM

View PostSFC174, on 11 April 2022 - 01:12 PM, said:

Do you really think that Match Score is somehow a better representation of a person's skill? That is really the question is here, if MS is better than WLR then it is fine to use to calculate someone's PSR with. Unfortunately MS is terrible which is why PSR is terrible and the MM is terrible. The main reason I suggest WLR is because creating a good formulation for MS is 10x harder.

Nope... MS is not a good metric but then again WLR isn't either nor is damage. Two of the three are derived from team outcome. The remainder is hard to quantify as a direct indicator of an individuals skill quotient.

I'm not a statistical expert or even a certifiable mathematician (Did stay at a Holiday Inn once though)... But I'm not convinced using a binary outcome to determine an individuals skill quotient when that binary outcome is directly influenced by peripheral effect. In my somewhat addled and feedable mind it's a weak variable.

View PostKnownswift, on 11 April 2022 - 02:01 PM, said:


This game only resolves in three ways.

1. You win.
2. You lose.
3. You tie (both teams lose)


Correction: This team game resolves in three ways... Win, lose draw. 24 individual players contribute to that binary outcome. The point of my examples was to infer you can have a great game and still loss the match. Conversely, you can contribute very little to a team win yet still net a statistical win.

A win/loss binary outcome is a dependent variable as it relates to the other metrics that contribute to that binary outcome.

Et all, I'm mealy stating opinion and responding to questions and propositions laid out in front of me... I'm to old for a pissing match but do believe any discussion is good discussion so long as all parties play nice and ""someone/anyone" might glean something useful from it.

#116 pbiggz

    Member

  • PipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 4,826 posts
  • LocationOutreach

Posted 11 April 2022 - 07:39 PM

View PostNightbird, on 11 April 2022 - 03:02 PM, said:

Only showing the insanity of your previous statement.


The point im making is that taking a huge dookie on the existing match maker because you made some graphs that say WLR is "n% better" is misleading. WLR is not a silver bullet that will solve all our problems. All it will do is push different people into the margins. Your comments regarding player retention are all assumptions, and without a window into the actual source code you have no grounds to speak authoritatively. If you're honest about the assumptions you're making, thats fine, be honest about them, but don't talk like an authority because you figured out how to use microsoft excel.

And further to that point, taking a huge dookie on everyone who suggests ways to tune up the existing matchmaker is non-constructive and just makes you look like a ****.

Edited by pbiggz, 11 April 2022 - 07:46 PM.


#117 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 April 2022 - 08:45 PM

View Postpbiggz, on 11 April 2022 - 07:39 PM, said:

The point im making is that taking a huge dookie on the existing match maker because you made some graphs that say WLR is "n% better" is misleading. WLR is not a silver bullet that will solve all our problems. All it will do is push different people into the margins. Your comments regarding player retention are all assumptions, and without a window into the actual source code you have no grounds to speak authoritatively. If you're honest about the assumptions you're making, thats fine, be honest about them, but don't talk like an authority because you figured out how to use microsoft excel.

And further to that point, taking a huge dookie on everyone who suggests ways to tune up the existing matchmaker is non-constructive and just makes you look like a ****.


1. Being able to predict the results of an action is useful, unlike throwing random ideas around. For example, when I present evidence that a better MM would improve retention, I backed it up with data scraped from the leaderboards. When you say this or that, it's backed up by nothing.

2. math doesn't care about feelings, you are either right or wrong, there are no ribbons for trying

Edited by Nightbird, 11 April 2022 - 09:09 PM.


#118 pbiggz

    Member

  • PipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 4,826 posts
  • LocationOutreach

Posted 12 April 2022 - 07:56 AM

View PostNightbird, on 11 April 2022 - 08:45 PM, said:


1. Being able to predict the results of an action is useful, unlike throwing random ideas around. For example, when I present evidence that a better MM would improve retention, I backed it up with data scraped from the leaderboards. When you say this or that, it's backed up by nothing.

2. math doesn't care about feelings, you are either right or wrong, there are no ribbons for trying


There aren't any ribbons for ******** on anyone who questions you either.

#119 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 12 April 2022 - 09:55 AM

View Postpbiggz, on 12 April 2022 - 07:56 AM, said:


There aren't any ribbons for ******** on anyone who questions you either.


True or false, did I accurately simulate and predict the outcome of the current match maker using Jay Z's PSR before it was in game?

Blind dart tossing doesn't get any respect from me since I do this as a pro.

#120 pbiggz

    Member

  • PipPipPipPipPipPipPipPipPip
  • Overlord
  • Overlord
  • 4,826 posts
  • LocationOutreach

Posted 12 April 2022 - 10:34 AM

View PostNightbird, on 12 April 2022 - 09:55 AM, said:

True or false, did I accurately simulate and predict the outcome of the current match maker using Jay Z's PSR before it was in game?

Blind dart tossing doesn't get any respect from me since I do this as a pro.


If you can't understand why ******** on everyone who thinks differently from you isn't an effective form of communication then you should stop talking at all. It doesn't matter how qualified you think you are. It doesn't matter how qualified you actually are. You're being a ****.

Edited by pbiggz, 12 April 2022 - 10:34 AM.






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users