Jump to content

Psr Update And Hold On Patch.


723 replies to this topic

#501 thievingmagpi

    Member

  • PipPipPipPipPipPipPipPip
  • 1,377 posts

Posted 12 June 2020 - 06:49 PM

Any kind of "drift", as may occur in the proposed systems might be an issue with the assumption that the game is going to last another 8 years.

Or that any kind of ranking system couldn't just be reset periodically.

Edited by thievingmagpi, 12 June 2020 - 06:50 PM.


#502 Dar1ngOne

    Member

  • PipPipPipPipPipPipPipPip
  • Star Colonel IV
  • Star Colonel IV
  • 1,248 posts
  • LocationPerth WA

Posted 12 June 2020 - 06:53 PM

This caught my eye, and made me log in to comment.

The issue is not the formula or the player base. It's in the balance of weapons and tech, and experience.

For example - If the game was locked to 1 mech, with the same tech base, same weapons, across the 24 players, then you would get a real sense of who knows what to do and the skill level involved.

So for instance with have 24 Commando's on the field, all the same, and I would expect the account named Mycrus to dominate most of the matches.

Then what happens is people remember that name as being a threat, so the other players start to focus Mycrus. Now Mycrus may make an alt account, or change his account name to keep the focus of him.

So then PGI introduces mech X and weapon X, and then suddenly, the meta changes, and a great player in a Commando is very average in a Hunchback.

Your punishing a players account based on the fact that he hasn't got the right mech, the right weapons, despite having the skill in another chassis?

If it were me, and I have said this to you in the past PGI, you would have the Tiers renamed as Elite, Veteran, Rookie, and Green.

You would have a mode where the mechs have to be the same builds, and then sort the best from the worst in that mode.

Currently, your trying to assess a player's worth on shifting sands, and if your building a system on that, it will fail.

Good luck with that. Thanks again for MW5.

#503 Dogmeat1

    Member

  • PipPipPip
  • CS 2019 Silver Champ
  • CS 2019 Silver Champ
  • 87 posts
  • LocationButte Hold

Posted 12 June 2020 - 07:17 PM

I'm posting this on behalf of kamikaze viking.

Kami's Lookup Table PSR Shift Proposal.

the reasoning behind My proposal is

a) To fit within the scope given. Minimal work for Best effort gains.

Posted Image A compromise to cater to both camps of "everything is Win/loss" and "everything is personal skill"

I realised early in the discussion regarding how Match Score affect PSR movement that due to score increase based on the WIN The current system is 'double dipping'. Hence the Solution appeared to me to be to neutralise the Raw Match score by using a relative system separated by team. This would seem to be to cancel out the Match Score granted for winning, so that the total MS for each player reflects better on their skill within the team.

The result is that this system grants a factor for the WIN/loss as well as your personal skill relative to your team, and these should be easily tweakable within this system (values within a lookup table).

Method
==============

This graphic may be familiar to some as ive posted it a few times in recent weeks.

https://i.imgur.com/nSl4rQ9.jpg

PSR Shift Values
================
These are base shift values - totalled as zero sum.
Win = -1 +0 +1 +2 = +1
Loss = -2 -1 +0 +1 = -1
Win + loss = +1 -1 = 0

The higher the number the faster people will move through the psr range per match. initial Simulations they found that 0, 12 & 24 worked best (and fit with Pauls proposed range).

Applying this across the match. Many systems suggested will sort all 24 players and then rank them, mine MUST keep the two teams separate and rank them by MS within their team.
==================
When applied to each team in a 3/3/3/3 team split

Win = -1x3 +0x3 +1x3 +2x3 = -3 +0 +3 +6= +6
Loss = -2x3 -1x3 +0x3 +1x3 = -6 -3 +0 +3= -6
Win + loss = +6 -6 = 0

Example Match using 3/3/3/3 split
https://i.imgur.com/FJoNrQO.jpg

==================

As Jay Z was testing in his simulator (as seen in this post https://mwomercs.com...66#entry6337266), he implemented this using a lookup table. Which showed once tweaked, a more granular version seemed a better distribution and still remains zero sum. I derived my own version as can be seen below;

Winning team PSR Shift



Base versionTweaked version
2424
24 20
24 16
12 12
12 8
12 4
00
00
00
-12-4
-12-8
-12-12


Losing team PSR Shift


Base versionTweaked version
1212
12 8
124
00
0 0
0 0
-12-4
-12-8
-12-12
-24-16
-24-20
-24-24


Sum = 0

Edited by Dogmeat1, 13 June 2020 - 02:55 AM.


#504 MischiefSC

    Member

  • PipPipPipPipPipPipPipPipPipPipPipPip
  • The Benefactor
  • The Benefactor
  • 16,695 posts

Posted 12 June 2020 - 07:18 PM

I don't play enough to be a real contributor to this discussion but I want to point this out -

Match Score doesn't equate to how effective someone is at winning matches. It's skewed by damage farming, kill stealing, etc. While top tier players often have a high average match score that's a product of a lot more than just damage numbers and component destruction.

The only useful metric for determining how effective someone is at winning matches is.... how often what they do, how they play, helps their team win matches.

So the question very honestly becomes:

Do you want a matchmaker that is as effective as possible at building teams who are evenly matched at winning

OR

Do you want a matchmaker that rewards the average players perception of 'I dun good' and punishes the average players perception of 'I dun bad'.

Maybe the game lacks the population to make the first super effective but the second is largely worthless as a matchmaker anyway. If the purpose of the new MM/PSR system is to just give player a sense of validation for 'I dun good' or 'I dun bad' that needs established. That's not necessarily a bad thing even; the community is a lot more 'stompy robbits EXPLOSIONS boom whee' than 'LEARN GIT GUD'.

#505 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 5,389 posts

Posted 12 June 2020 - 07:30 PM

View PostKurlon, on 08 June 2020 - 04:30 PM, said:

The community driven PSR project

In response to PGI’s announcement on this thread, the discord and reddit communities have decided to work together to try to find the best solution to this issue. At first this was a JGx led initiative but quickly expanded to include dozens of representatives from most of the major units, as well as a large number of voices from the solo community. After many hours exchanging and refining ideas, we believe we have come up with a solution that both meets PGI’s requirements and that will improve the gaming experience of the community as a whole.

Arbitrary fixed PSR gains/losses present a number of problems in terms of balance and fairness to all players involved in an individual match. Therefore our solution is to use scaling PSR gains/losses based on an individual player’s match score matchscore as compared to the team's average. The system uses the Win/Loss status to compute a 0 sum PSR shift that is scaled fairly to the pilot’s contribution in that match. The maths involved is no more complex than the current XP, C-bill and Matchscore calculations requiring minimal development time.

This system was developed by Jay Z with heavy input from the veteran community. A link to a full explanation is below;

https://docs.google....3LgMBQJ4hE/edit

And a spreadsheet simulating the system can be found here;

https://docs.google....dit?usp=sharing

In summary, this proposal uses simple arithmetic similar to existing calculations to assign fair and reasonable PSR shifts in accordance to pilot contribution to the match result. The aim of PSR is to segregate the playerbase on their respective match impact to give the Matchmaker the best possible information to create fair and interesting matches.

The version of this proposal you see presented is simplified from a more complicated model we have been testing which has more adjusting and weighting factors. Within the parameters set for this task, we believe this to be a straightforward implementation which will have community support. Finally, given there are only two values that control the maths, the long term management is simple and easy to hand over. If you have any questions, or concerns, please let us know as we have investigated this and numerous other solutions in great detail.

The matchscore formula itself does need quite a bit of adjusting. We plan on posting an update regarding that soon.

Finally during our discussions we also came up with 2 other proposals we strongly believe would be worth considering;

Kamikazi viking’s PSR lookup table https://mwomercs.com...05#entry6338105

Gagis’ PSR gains based on matchscore ratios https://mwomercs.com...st__p__6338091




View PostDogmeat1, on 12 June 2020 - 07:17 PM, said:

I'm posting this on behalf of kamikaze viking.

Kami's Lookup Table PSR Shift Proposal.

the reasoning behind My proposal is

a) To fit within the scope given. Minimal work for Best effort gains.

Posted Image A compromise to cater to both camps of "everything is Win/loss" and "everything is personal skill"

I realised early in the discussion regarding how Match Score affect PSR movement that due to score increase based on the WIN The current system is 'double dipping'. Hence the Solution appeared to me to be to neutralise the Raw Match score by using a relative system separated by team. This would seem to be to cancel out the Match Score granted for winning, so that the total MS for each player reflects better on their skill within the team.

The result is that this system grants a factor for the WIN/loss as well as your personal skill relative to your team, and these should be easily tweakable within this system (values within a lookup table).

Method
==============

This graphic may be familiar to some as ive posted it a few times in recent weeks.

https://i.imgur.com/nSl4rQ9.jpg

PSR Shift Values
================
These are base shift values - totalled as zero sum.
Win = -1 +0 +1 +2 = +1
Loss = -2 -1 +0 +1 = -1
Win + loss = +1 -1 = 0

The higher the number the faster people will move through the psr range per match. initial Simulations they found that 0, 12 & 24 worked best (and fit with Pauls proposed range).

Applying this across the match. Many systems suggested will sort all 24 players and then rank them, mine MUST keep the two teams separate and rank them by MS within their team.
==================
When applied to each team in a 3/3/3/3 team split

Win = -1x3 +0x3 +1x3 +2x3 = -3 +0 +3 +6= +6
Loss = -2x3 -1x3 +0x3 +1x3 = -6 -3 +0 +3= -6
Win + loss = +6 -6 = 0

Example Match using 3/3/3/3 split
https://i.imgur.com/FJoNrQO.jpg

==================

As Jay Z was testing in the simulator (as seen in this post https://mwomercs.com...66#entry6337266), he implemented this using a lookup table. Which showed once tweaked, a more granular version seemed a better distribution and still remains zero sum.

Winning team PSR Shift



Base versionTweaked version
2424
24 20
24 16
12 12
12 8
12 4
00
00
00
-12-4
-12-8
-12-12


Sum = 0

Losing team PSR Shift


Base versionTweaked version
1212
12 8
124
00
0 0
0 0
-12-4
-12-8
-12-12
-24-16
-24-20
-24-24


Sum = 0


The issue with the current PSR + MM is that you can calculate an average PSR movement per match for player, call it X, then the player's "skill" = # matches_played * X.

After 100 games, PSR=100X. After 2000 games, PSR=2000X. Whether the PSR really represents skill is convoluted with the number of matches.

It means you can have two players with PSR = 1000 and 3000, and the PSR = 1000 player reached it with 10 * 100matches, and the PSR = 3000 player reached it with 3 * 1000matches. It is clear the PSR = 3000 player is not as skilled. Such mix ups across the population means the Match Maker is blind when using this PSR.


With either PSR shift proposal above, we're changing X for Y. Is it obvious how much of an impact this will have on match quality?

Edited by Nightbird, 12 June 2020 - 07:42 PM.


#506 Jochi Kondur

    Member

  • PipPipPip
  • Shredder
  • 66 posts

Posted 12 June 2020 - 08:52 PM

View PostKurlon, on 08 June 2020 - 04:30 PM, said:


- s n i p -

This system was developed by Jay Z with heavy input from the veteran community. A link to a full explanation is below;

https://docs.google....3LgMBQJ4hE/edit

And a spreadsheet simulating the system can be found here;

https://docs.google....dit?usp=sharing


- s n i p -



For Case 3 (Stomp), although prefaced with "hopefully become less common", players 13 - 16 on the Losing Team in the cited example were assessed to have a far greater JZ PSR Shift because they “valiantly fought on a sinking ship” under the proposed system.

Was this scenario tested by usage of actual stomp data, could they be shown? Ideally, a video analysis from the Losing Team POV to show that those who stood to valiantly fight and fell as a result are indeed deserving to be in the company of players 3 - 5 (from Case 3 Stomp example) on the Winning Team.

Edited by Jochi Kondur, 12 June 2020 - 09:08 PM.


#507 Dogmeat1

    Member

  • PipPipPip
  • CS 2019 Silver Champ
  • CS 2019 Silver Champ
  • 87 posts
  • LocationButte Hold

Posted 12 June 2020 - 10:35 PM

View PostPaul Inouye, on 12 June 2020 - 04:20 PM, said:

I've also been asked a number of times for current values of those Match Score kickers. At this time I can't provide you with that due to possible exploitation/trolling behaviors. I can however provide you a sample set of numbers to show you how the calculations work in general that reveals what would be your end of round match score and you could tweak from there.


That would be much appreciated. We understand the reservations PGI has with giving out the exact formula. Many of us already have a fair idea of how the formula works ,however, any data you can give us will be very helpful.

Is there any timeline yet on when we might be able to receive that data, as well as the deadline for submitting matchscore proposals? I have been working with many in the comp and casual communities and we have already submitted some PSR ideas and now are focusing on improving the matchscore balance so that it better reflects key values that contribute strongly towards wins. We will probably submit our matchscore formula ideas soon but it would be good to know when the cutoff for that will be.

#508 Horseman

    Member

  • PipPipPipPipPipPipPipPipPip
  • CS 2019 Top 8 Qualifier
  • CS 2019 Top 8 Qualifier
  • 3,296 posts
  • LocationPoland

Posted 12 June 2020 - 10:54 PM

View PostNightbird, on 12 June 2020 - 02:09 PM, said:

Don't worry about it. I was in that group but my idea was rejected because of the reason... and I paraphrase "The community is too stupid to understand it". I have a screenshot lol
You forget that some of us in this thread also have access to the discord. Your idea has been applauded by quite a few people. The problem they had with it was that it would be extremely difficult to convince most of the playerbase of its' value.

Edited by Horseman, 12 June 2020 - 10:56 PM.


#509 Gagis

    Member

  • PipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 765 posts

Posted 13 June 2020 - 12:08 AM

View PostJochi Kondur, on 12 June 2020 - 05:57 PM, said:


I can't reconcile with the fact that the worst performing player (i.e. worst of 24) has no negative PSR movement just because the person happens to be on the winning team. I can't cast my support behind such a system.

If their score is low enough, the Feelgood Factor will push their WinScore to the negatives. The Feelgood Factor was added so you would feel good about this. :P

#510 Jochi Kondur

    Member

  • PipPipPip
  • Shredder
  • 66 posts

Posted 13 June 2020 - 12:25 AM

View PostGagis, on 13 June 2020 - 12:08 AM, said:

If their score is low enough, the Feelgood Factor will push their WinScore to the negatives. The Feelgood Factor was added so you would feel good about this. Posted Image


What value should it be set to for that to happen? Alternating between 0 and 1 doesn't make a difference in the spreadsheet.

#511 Gagis

    Member

  • PipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 765 posts

Posted 13 June 2020 - 12:30 AM

View PostJochi Kondur, on 13 June 2020 - 12:25 AM, said:

What value should it be set to for that to happen? Alternating between 0 and 1 doesn't make a difference in the spreadsheet.

Ah, right. Should have written default values down so people can go back to them after someone has played with it. It gets compared to your Match Score so something between 50 and 150 might work. However, if the match is weird and the highest score anyone gets is 100 and FgF is 150, the PSR calculations go really wild. Thats why I'll probably recommend binding it to Average Score / 2.

#512 TheUltimateGhost

    Member

  • PipPipPipPipPip
  • Fire
  • Fire
  • 197 posts
  • Twitch: Link
  • LocationStrana Mechty

Posted 13 June 2020 - 12:38 AM

View PostKurlon, on 08 June 2020 - 04:30 PM, said:

This system was developed by Jay Z with heavy input from the veteran community. A link to a full explanation is below;

https://docs.google....3LgMBQJ4hE/edit

In summary, this proposal uses simple arithmetic similar to existing calculations to assign fair and reasonable PSR shifts in accordance to pilot contribution to the match result. The aim of PSR is to segregate the playerbase on their respective match impact to give the Matchmaker the best possible information to create fair and interesting matches.

The version of this proposal you see presented is simplified from a more complicated model we have been testing which has more adjusting and weighting factors. Within the parameters set for this task, we believe this to be a straightforward implementation which will have community support. Finally, given there are only two values that control the maths, the long term management is simple and easy to hand over. If you have any questions, or concerns, please let us know as we have investigated this and numerous other solutions in great detail.


just one little thing I'd add, for a Tie just let W be 0 instead of 1 for a win and -1 for a loss

#513 Jochi Kondur

    Member

  • PipPipPip
  • Shredder
  • 66 posts

Posted 13 June 2020 - 12:43 AM

View PostGagis, on 13 June 2020 - 12:30 AM, said:

Ah, right. Should have written default values down so people can go back to them after someone has played with it. It gets compared to your Match Score so something between 50 and 150 might work. However, if the match is weird and the highest score anyone gets is 100 and FgF is 150, the PSR calculations go really wild. Thats why I'll probably recommend binding it to Average Score / 2.


Ok, it works as advertised now. Cheers.

#514 Kamikaze Viking

    Member

  • PipPipPipPipPipPip
  • Bad Company
  • Bad Company
  • 368 posts
  • LocationStay on Topic... STAY ON TOPIC!!!

Posted 13 June 2020 - 02:46 AM

I vote For Jay Z's System
and Secondarily for my own (especially as tweaking is as simple as adjusting the values in the lookup table).

And for anyone asking. Jay Z actually built and tested mine in parallel with his and once we got it working, that was when this idea was put back on the table (im guessing it was just hidden along with others that were tested to remove clutter from his own presentation).

EDIT: Im now leaning towards voting for Gagis's sytem after nightbird and Xiphias have done their analysis.

Edited by Kamikaze Viking, 14 June 2020 - 01:51 AM.


#515 MechTech Dragoon

    Member

  • PipPipPipPipPipPip
  • Elite Founder
  • Elite Founder
  • 289 posts

Posted 13 June 2020 - 03:39 AM

Jay Z's system looks good. Go with it.

#516 Xiphias

    Member

  • PipPipPipPipPipPipPip
  • Littlest Helper
  • Littlest Helper
  • 852 posts

Posted 13 June 2020 - 05:19 AM

View PostMischiefSC, on 12 June 2020 - 07:18 PM, said:

I don't play enough to be a real contributor to this discussion but I want to point this out -

Match Score doesn't equate to how effective someone is at winning matches. It's skewed by damage farming, kill stealing, etc. While top tier players often have a high average match score that's a product of a lot more than just damage numbers and component destruction.

The only useful metric for determining how effective someone is at winning matches is.... how often what they do, how they play, helps their team win matches.

So the question very honestly becomes:

Do you want a matchmaker that is as effective as possible at building teams who are evenly matched at winning

OR

Do you want a matchmaker that rewards the average players perception of 'I dun good' and punishes the average players perception of 'I dun bad'.

Maybe the game lacks the population to make the first super effective but the second is largely worthless as a matchmaker anyway. If the purpose of the new MM/PSR system is to just give player a sense of validation for 'I dun good' or 'I dun bad' that needs established. That's not necessarily a bad thing even; the community is a lot more 'stompy robbits EXPLOSIONS boom whee' than 'LEARN GIT GUD'.

I agree that W/L is the best predictor for determining W/L, however there is a significant contingent that disagrees with this (I think a lot of people are too focused on individual matches and not the bigger picture). That said, with the combining of queues, groups do have a potential to skew a W/L based metric, whether that really matters is a different question.

The way I see it, any good PSR system needs to includes some element of W/L (winning should increase your ability to climb), but a pure W/L metric could take a long time to converge to an appropriate level (and be skewed by teams). Looking at global stats, MS and WLR are correlated, players with better match score tend to win more. By including MS into the calculation you can increase the convergence rate (time to get to correct PSR) at the cost of introducing some steady state error (settling at a PSR that isn't exactly what WLR would eventually converge to). The top two players on the losing team are usually better than the bottom two players on the winning team, so the system moves players more quickly if they demonstrate consistent performance.

Given that T1-T3 players are already going to end up playing against each other when the valves are opened, I think you can make a decent argument that converging faster is worth sacrificing a bit of prediction accuracy. It also has the added benefit of appeasing players who dislike going down in PSR despite scoring well. No system is going to be perfect and I feel like a pure WLR system is unfeasible with the community opinion at the moment (old ELO system was based on WLR/ELO and people complained a lot about going down on a loss).

At the end of the day, a lot of players care more about the system rewarding them how they feel they should be rewarded, rather than how they mathematically should be shifted around to get the best matches. Lots of players care more about individual matches (e.g. going down on a loss with 8 kills 1200 damage) than overall trends.

#517 Jochi Kondur

    Member

  • PipPipPip
  • Shredder
  • 66 posts

Posted 13 June 2020 - 05:28 AM

View PostKurlon, on 08 June 2020 - 04:30 PM, said:

- s n i p -

This system was developed by Jay Z with heavy input from the veteran community. A link to a full explanation is below;

https://docs.google....3LgMBQJ4hE/edit

And a spreadsheet simulating the system can be found here;

https://docs.google....dit?usp=sharing

- s n i p -



How-what-why is the difference between the values in columns R and S? Thanks.

Edited by Jochi Kondur, 13 June 2020 - 05:30 AM.


#518 Gagis

    Member

  • PipPipPipPipPipPipPip
  • FP Veteran - Beta 1
  • FP Veteran - Beta 1
  • 765 posts

Posted 13 June 2020 - 06:05 AM

View PostJochi Kondur, on 13 June 2020 - 05:28 AM, said:


How-what-why is the difference between the values in columns R and S? Thanks.

If I remember right he had one column where your score was compared to only your own team and another one where it was compared to both teams as two alternative ways to do it. Not 100% sure since it was an earlier version of his spreadsheet that I stole the layout from for my proposal.

#519 Jochi Kondur

    Member

  • PipPipPip
  • Shredder
  • 66 posts

Posted 13 June 2020 - 06:20 AM

View PostGagis, on 13 June 2020 - 06:05 AM, said:

If I remember right he had one column where your score was compared to only your own team and another one where it was compared to both teams as two alternative ways to do it. Not 100% sure since it was an earlier version of his spreadsheet that I stole the layout from for my proposal.


Could you pass on the feedback to update the header descriptions to something more descriptive (the Little Details™...) Thanks!

#520 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 5,389 posts

Posted 13 June 2020 - 07:34 AM

View PostHorseman, on 12 June 2020 - 10:54 PM, said:

You forget that some of us in this thread also have access to the discord. Your idea has been applauded by quite a few people. The problem they had with it was that it would be extremely difficult to convince most of the playerbase of its' value.



Normally the burden of proof is on the one proposing a solution to a complex mathematical problem. Going without it is the same as slapping down a 5 on a math test question without going through any steps to "show your work" and hope.

I already made an extra effort, not only to provide mathematical evidence that my solution is sound, but also that the proposed systems won't work. It's up to the rest of you that decide whether to leave the fate of the MM to sound evidence or to hope. If you all go with hope, then live with the results.

Edited by Nightbird, 13 June 2020 - 07:34 AM.






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users