Jump to content

Psr Update And Hold On Patch.


723 replies to this topic

#481 OneTeamPlayer

    Member

  • PipPipPipPipPipPip
  • Bad Company
  • 390 posts

Posted 12 June 2020 - 02:30 PM

View PostNightbird, on 12 June 2020 - 02:09 PM, said:

Don't worry about it. I was in that group but my idea was rejected because of the reason... and I paraphrase "The community is too stupid to understand it". I have a screenshot lol


Post the screenshot.

#482 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 5,389 posts

Posted 12 June 2020 - 02:36 PM

View PostOneTeamPlayer, on 12 June 2020 - 02:30 PM, said:

Post the screenshot.


Can't, it insults the community. I PMed it to you.

#483 Socal Bronco

    Member

  • PipPip
  • @ui_title_%s_Rank_4
  • @ui_title_%s_Rank_4
  • 23 posts

Posted 12 June 2020 - 02:57 PM

View PostNightbird, on 10 June 2020 - 08:29 PM, said:


How would you handle teams of 2, 3, 4?

As far as how the MM organizes players from tiers, for all solo players matches this is already how it does it. The reason it doesn't work is because past MS is a poor predictor of performance on the team. (For the people that haven't hit the T1 cap, total past MS earned is their PSR bar location today.) It doesn't go by the 5 tiers but rather by a more granular number in the background. (The EXP bar line)

This is also why all the proposals based on MS won't work. Past MS is only loosely related to odds of winning future matches, whereas past WLR is strongly related. Picked any match score on Jarl's list (not adjusted) and look up and down the list at the player's WLR. You'll see a huge variance. This is clearer when you take all the data and graph it, showing a clear pattern that MS is inferior to WLR.

I analyzed Nightbirds Jarls list data vs my data for the most recent 6 months and vs the data from my first 6 months.
Nightbirds WLR varies 40% vs his MS which only varies 10%

My current average WLR is less than it was when I first started, but my match score is higher. Hmm, presumably I'm a better player now. In all three cases WLR varied much more over the period vs AVG match score. So explain to me how WLR will do a better job of identifying pilot skill?
https://www.flickr.c...2791@N05/N26583



Edited by Socal Bronco, 12 June 2020 - 03:15 PM.


#484 JoeCold

    Member

  • Pip
  • Bad Company
  • Bad Company
  • 18 posts
  • LocationMallory's World

Posted 12 June 2020 - 03:00 PM

It occurs to me that if the population forces the matchmaking system to mix tiers to any degree, shouldn't the PSR adjustments for the match handicap above and below the average of the match. That is, shouldn't players above the average tier in that match have a penalty modifier to the rest of their PSR calculation and vice versa? Otherwise mixing tiers drives everyone to the extremes, albeit at different rates depending on ones distance from average.

#485 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 5,389 posts

Posted 12 June 2020 - 03:01 PM

View PostSocal Bronco, on 12 June 2020 - 02:57 PM, said:

I analyzed Nightbirds Jarls list data vs my data for the most recent 6 months and vs the data from my first 6 months.
Nightbirds WLR varies 40% vs his MS which only varies 10%


You have to analyze the data for all players to see trends. A single person's data isn't useful.

#486 Horseman

    Member

  • PipPipPipPipPipPipPipPipPip
  • CS 2019 Top 8 Qualifier
  • CS 2019 Top 8 Qualifier
  • 3,296 posts
  • LocationPoland

Posted 12 June 2020 - 03:07 PM

View PostVonBruinwald, on 12 June 2020 - 01:04 PM, said:

Looks like the "community consensus" is coming from an off-site discord which isn't being advertised:
https://twitter.com/...845618194280448

It has been advertised... you just missed when and where.
They're working on their proposals. In an environment that's more conducive to actual work as opposed to getting constantly interrupted by armchair mechwarriors who think they have a constitutional right to be in Tier 1.
Don't worry, even if you missed the bus you'll get to see the proposals when they're presented.

Edited by Horseman, 12 June 2020 - 03:08 PM.


#487 Eatit

    Member

  • PipPipPipPipPipPip
  • Liquid Metal
  • Liquid Metal
  • 259 posts

Posted 12 June 2020 - 03:10 PM

We need more information on how the current MM works in order to make any meaningful evaluation.

My imagining of how it works currently.

One Pool of players each with a PSR value.

MM asks pool to provide 24 players that have been in que longest in chassis broken down by size. 3A-3H-3M-3L per team.
Limited by current tier valve setting.

If the MM can't provide 24 broken down by size it opens up the size limitations to allow variation on what's available. e.g. 4A-4H-2M-2L per team.

If the pool is still too small it opens the tier valve.

My understanding is that the current MM is allowing Tier 1-3, 2-4, or 3-5 normally and opens to tier 1-4 or 2-5 and in extreme cases 1-5.

Once it has opened the size and tier valves and two teams are created it sorts those teams based on Size. It only uses the tiers to create the pool to pull from. It uses size to create the 2 teams. This will create randomly skilled teams solely based on luck of the draw.



Most of the playerbase is in T1 currently and the MM still has to combine 1-3 at best and 1-5 at worst. What is the importance of moving players from T1 to T4 if the MM is going to open the valve and allow them all to be in the same match anyway? And then sort the teams randomly based on size. This still has the possibility of 12 tier one players vs. 12 tier 4 players.



ALL MATCHES ARE UNBALANCED. SHUFFLING THE PLAYER BASE AROUND IN THE TIERS WILL CHANGE NOTHING IF MM IS PULLING FROM ALL TIERS.

Sorting the teams by adding equal numbers of players from each tier on each team will help with this issue. Fixing the PSR rating will help to sort the players but they both need to be done to create balance not just one or the other.

PGI/Paul please correct me if I'm wrong.

#488 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 5,389 posts

Posted 12 June 2020 - 03:28 PM

View PostHorseman, on 12 June 2020 - 03:07 PM, said:

It has been advertised... you just missed when and where.
They're working on their proposals. In an environment that's more conducive to actual work as opposed to getting constantly interrupted by armchair mechwarriors who think they have a constitutional right to be in Tier 1.
Don't worry, even if you missed the bus you'll get to see the proposals when they're presented.


I'm eagerly awaiting the proof behind the proposals :)

#489 Zanotam

    Member

  • Pip
  • Legendary Founder
  • Legendary Founder
  • 16 posts

Posted 12 June 2020 - 03:39 PM

To everyone complaining about groups:

A group can generally expected to be maybe 4 slots, 1/3 of your team. In League of Legends however the 'solo queue' lets people queue in pairs which makes them 2/5ths of the team (~6.6666% repeating more!) and yet nobody is bitchign about how it's impossible to accurately rank people in LoL.... in fact, matchmaking in LoL is pretty damn good.

There are exactly 3 meaningful things for the future MM:

1. player population,
2. using something meaningful to predict winrate
3. Effectively composing teams in the most fair fashion to adjust for top-tier and bottom-tier players skewing things.


GROUP QUEUE IS NOT RUINING THINGS AND SMALL GROUPS AS PART OF A LARGER TEAM ARE NOT AN ISSUE FOR A MATCHMAKER!

#490 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 5,389 posts

Posted 12 June 2020 - 03:50 PM

View PostZanotam, on 12 June 2020 - 03:39 PM, said:

To everyone complaining about groups:

A group can generally expected to be maybe 4 slots, 1/3 of your team. In League of Legends however the 'solo queue' lets people queue in pairs which makes them 2/5ths of the team (~6.6666% repeating more!) and yet nobody is bitchign about how it's impossible to accurately rank people in LoL.... in fact, matchmaking in LoL is pretty damn good.

There are exactly 3 meaningful things for the future MM:

1. player population,
2. using something meaningful to predict winrate
3. Effectively composing teams in the most fair fashion to adjust for top-tier and bottom-tier players skewing things.


GROUP QUEUE IS NOT RUINING THINGS AND SMALL GROUPS AS PART OF A LARGER TEAM ARE NOT AN ISSUE FOR A MATCHMAKER!


For a working* matchmaker. LoL uses a heavily modified Elo system. It's basically W/L but with point movements adjusted for the relative strength of teams and players. PGI did ELO at first but they didn't use the right formula and so above average players went to infinity and below average players went to 0 lol. The current system is slightly less broken.

Edited by Nightbird, 12 June 2020 - 03:52 PM.


#491 VonBruinwald

    Member

  • PipPipPipPipPipPipPipPipPip
  • The Undisputed
  • The Undisputed
  • 2,647 posts
  • LocationRandis IV

Posted 12 June 2020 - 03:52 PM

View PostHorseman, on 12 June 2020 - 03:07 PM, said:

It has been advertised... you just missed when and where.


Got a link for that advert?

View PostHorseman, on 12 June 2020 - 03:07 PM, said:

They're working on their proposals. In an environment that's more conducive to actual work as opposed to getting constantly interrupted by armchair mechwarriors who think they have a constitutional right to be in Tier 1.


You do realise that's most of T1 you're talking about.

#492 OneTeamPlayer

    Member

  • PipPipPipPipPipPip
  • Bad Company
  • 390 posts

Posted 12 June 2020 - 04:01 PM

View PostZanotam, on 12 June 2020 - 03:39 PM, said:

To everyone complaining about groups:

A group can generally expected to be maybe 4 slots, 1/3 of your team. In League of Legends however the 'solo queue' lets people queue in pairs which makes them 2/5ths of the team (~6.6666% repeating more!) and yet nobody is bitchign about how it's impossible to accurately rank people in LoL.... in fact, matchmaking in LoL is pretty damn good.

There are exactly 3 meaningful things for the future MM:

1. player population,
2. using something meaningful to predict winrate
3. Effectively composing teams in the most fair fashion to adjust for top-tier and bottom-tier players skewing things.


GROUP QUEUE IS NOT RUINING THINGS AND SMALL GROUPS AS PART OF A LARGER TEAM ARE NOT AN ISSUE FOR A MATCHMAKER!


Has it occurred to you that people aren't complaining about groups in LoL because LoL isn't putting it's players into unwinnable matches one out of every three games. Not just unwinnable but so one sided as to be not enjoyable for both teams?

The rotating stomp fest started after group queue merge which is why this community is wary of the merge.

Pre-merge there were losses and bad tactics, heck even a stomp on occasion, but it wasn't so often that players could go an entire session seeing mostly stomps one way or another.

The most obvious solution would be to remove the group queue from solo then tweak a solution, but what's actually happening is keeping the two mixed, watching as players get more and more frustrated, and making a community based solution from externally based input sources after asking for community input on their own official forums.

In short, everything about the merge is a mess and MWO is obviously not LoL.

That's like saying "well zuckerberg can afford to buy a ferrari and not go into debt, why can't I?" Well, for starters, you're not Mark Zuckerberg rich, that's why.

When MWO starts getting down basics as well as LoL has we can start comparing the two, until then its an apples to dishwashers comparison.

#493 Eatit

    Member

  • PipPipPipPipPipPip
  • Liquid Metal
  • Liquid Metal
  • 259 posts

Posted 12 June 2020 - 04:02 PM

After reading many of these posts it occurs to me that there are some that think their skill is being devalued by allowing less skilled players to hold the same skill title, Tier 1.

There are also some that want the tiers to be better sorted so that they can have a more balanced experience when playing.

The current proposal will only work for the people that want the title T1 to mean something. Matches will still be unbalanced.
It seems as though this is an ego stroke and not an actual attempt to fix an unbalance MM.

#494 Zanotam

    Member

  • Pip
  • Legendary Founder
  • Legendary Founder
  • 16 posts

Posted 12 June 2020 - 04:19 PM

View PostNightbird, on 12 June 2020 - 03:50 PM, said:


For a working* matchmaker. LoL uses a heavily modified Elo system. It's basically W/L but with point movements adjusted for the relative strength of teams and players. PGI did ELO at first but they didn't use the right formula and so above average players went to infinity and below average players went to 0 lol. The current system is slightly less broken.


I mean, isn't W/L but with point movement adjusted for relative strength of teams and players even likely better than your pure W/L suggestion? Although I think your W/L suggestion is probably the best one I've seen though so far.

View PostOneTeamPlayer, on 12 June 2020 - 04:01 PM, said:

Has it occurred to you that people aren't complaining about groups in LoL because LoL isn't putting it's players into unwinnable matches one out of every three games. Not just unwinnable but so one sided as to be not enjoyable for both teams?

The rotating stomp fest started after group queue merge which is why this community is wary of the merge.

Pre-merge there were losses and bad tactics, heck even a stomp on occasion, but it wasn't so often that players could go an entire session seeing mostly stomps one way or another.

The most obvious solution would be to remove the group queue from solo then tweak a solution, but what's actually happening is keeping the two mixed, watching as players get more and more frustrated, and making a community based solution from externally based input sources after asking for community input on their own official forums.

In short, everything about the merge is a mess and MWO is obviously not LoL.

That's like saying "well zuckerberg can afford to buy a ferrari and not go into debt, why can't I?" Well, for starters, you're not Mark Zuckerberg rich, that's why.

When MWO starts getting down basics as well as LoL has we can start comparing the two, until then its an apples to dishwashers comparison.



I would point out that the statistics say stomps were happening 25% of the time pre-merge and you must not be aware of how common stomps are in LoL because 30% or so of the time sounds about right for stomps one way or another.

#495 Paul Inouye

    Lead Designer

  • Developer
  • Developer
  • 2,815 posts
  • LocationVancouver, BC

Posted 12 June 2020 - 04:20 PM

After reading and trying to keep everything organized, I will be back Tuesday after combining like minded thoughts with one another and go over the main proposals laid out here.

As mentioned on twitter (yes... I tweeted something and feel dirty doing it.), it's imporant that we keep focused on:

Match Score movement calculations - The +5, +3, +1 stuff when you reach a Match Score threshold. This includes those diagrams of overlapping movement on win/loss and even the discussions on removing the win/loss component and comparing your match score vs the 23 other players and assigning movement based on that.
Match Score thresholds - The 0-100, 101-300, 301-400+
Match Score event kicker calculations - The you should get X for a kill, Y for an assist etc.

I can't respond to every single post made here, but I'll do my best (as mentioned above) to combine any ideas that are the same but just have different variable settings into one response and I'll do my best to address any outliers outside of those groupings.

For Match Score event kickers, I'd like to release a set of data that you could run your formulas against. I don't know if you'd like that as a Google doc or an Excel spreadsheet. Let me know.

I've also been asked a number of times for current values of those Match Score kickers. At this time I can't provide you with that due to possible exploitation/trolling behaviors. I can however provide you a sample set of numbers to show you how the calculations work in general that reveals what would be your end of round match score and you could tweak from there.

-Paul

p.s. Forgot to mention something about the population numbers and some of you are aware... but no matter what system is put in place, the match maker is still going to have to open up to have matches kick off in timely manner.. so yes, while a new match score formula and PSR tier movement numbers will make a tighter skill based distribution, the numbers in the actual queue will dictate how far apart those skill numbers will be in a given match. This is why Tier 1 will continue to play against Tier 2 and Tier 3 players.

#496 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 5,389 posts

Posted 12 June 2020 - 04:29 PM

View PostZanotam, on 12 June 2020 - 04:19 PM, said:

I mean, isn't W/L but with point movement adjusted for relative strength of teams and players even likely better than your pure W/L suggestion? Although I think your W/L suggestion is probably the best one I've seen though so far.


Absolutely, but data is needed to fine tune and we have none. I'm not asking for PGI to release data either, since other companies don't release it either and do all of their math through hiring analysts.

#497 OneTeamPlayer

    Member

  • PipPipPipPipPipPip
  • Bad Company
  • 390 posts

Posted 12 June 2020 - 05:00 PM

View PostNightbird, on 12 June 2020 - 04:29 PM, said:


Absolutely, but data is needed to fine tune and we have none. I'm not asking for PGI to release data either, since other companies don't release it either and do all of their math through hiring analysts.


Except the "hiring analysts" part is missing here.

Also if the whole system is due for an imminent overhaul why would Paul care about "exploitative/trolling behavior" from data release since the system shouldn't be around long enough for any trolling behavior to matter in the short or long term?

Is this admitting that changes aren't coming any time soon?

#498 Gagis

    Member

  • PipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 765 posts

Posted 12 June 2020 - 05:03 PM

https://docs.google....t#gid=414734865
Details of my proposal are in the spreadsheet. Feel free to play with the values to see how it works.

PSR Based on Wins and Losses Evaluated by Your Share of Match Score

EXPLANATION
The goal here is to create a PSR that is convergent and does not diverge to infinities over a large number of games. In order to get rid of the ''XP Bar'' phenomenon, we need something other than a straight sum of match results. The playerbase however seems to be unhappy with tracking Win/Loss Record, so I made a model that gives your wins and losses a value based on your Match Score. I call this the WinScore/LossScore Ratio.

I even added a Feelgood Factor to make it possible to gain PSR on a loss trough ''negative LossScore'' or lose PSR on a win trough ''negative WinScore'' if your performance is too far above or below what it should be. If Feelgood Factor is 0, you won and earned the highest Match Score, you would get 1.0 Winscore or if you both lost and got the lowest Match Score, you would get a LossScore of 1.0.

HOW?
WinScore = (your_score - lowest_score - FgF) / (highest_score - lowest_score)
LossScore = 1 - (your_score - lowest_score + FgF) / (highest_score - lowest_score)

PSR approximates PSR = sum(WinScore) / sum(LossScore) over the last X games but since we don't want to store WinScores and LossScores or track multiple results between matches, it is actually updated as
new_PSR_if_winning = old_PSR + WinScore / X
new_PSR_if_losing = old_PSR / (1 + LossScore / X)
where X is whatever we want Length of Pseudo Match History to be.

I want to stress again that only information we need is the Match Scores, winners/losers and existing PSR and the only variable that needs to be saved is PSR.

Please check the accompanying google docs spreadsheet for more details.

Edited by Gagis, 12 June 2020 - 05:11 PM.


#499 Tamerlin

    Member

  • PipPipPipPipPipPip
  • The Patron Saint
  • The Patron Saint
  • 326 posts

Posted 12 June 2020 - 05:34 PM

View PostPaul Inouye, on 12 June 2020 - 04:20 PM, said:

For Match Score event kickers, I'd like to release a set of data that you could run your formulas against. I don't know if you'd like that as a Google doc or an Excel spreadsheet. Let me know.

I've also been asked a number of times for current values of those Match Score kickers. At this time I can't provide you with that due to possible exploitation/trolling behaviors. I can however provide you a sample set of numbers to show you how the calculations work in general that reveals what would be your end of round match score and you could tweak from there.


Excel would be good

#500 Socal Bronco

    Member

  • PipPip
  • @ui_title_%s_Rank_4
  • @ui_title_%s_Rank_4
  • 23 posts

Posted 12 June 2020 - 05:58 PM

View PostNightbird, on 12 June 2020 - 03:01 PM, said:


You have to analyze the data for all players to see trends. A single person's data isn't useful.


Just look at the jarls list global stats. Match score improves with more games played. 99% of players have WLR below 2, 50% are below 1, you can’t slice that small enough to tell good from bad. I looked at 3 sets of data, and all of the WLR numbers varied greatly month to month.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users