Jump to content

Tuning Up Jay Z's Psr System And Stabilizing It

Balance Gameplay General

26 replies to this topic

#1 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 30 June 2020 - 06:54 AM

EDIT : I have reviewed all the information here and documented it through Google Docs.
Google Doc available here:
https://docs.google....dit?usp=sharing

This has now been simulated using season 48 data for validation. Post here:
https://mwomercs.com...psr-comparison/

Getting a stable and functional PSR is a two step process. However it is quite easy to achieve building on JayZ's system.
  • Step 1. Modify the current X, Y, C.
  • Step 2. Add a formula to stabilize PSR. Additional Input : PSR before the match - that's it.
  • Step 3. New players start at 500 and there is no cap.
PRIORITY 1 : STABILIZE THE PSR

Uses a simple formula and PSR itself to make it stable. It doesn't really matter how the PSR shift is calculared at the end of a match for this to work.

Posted Image

PRIORITY 2 : X,Y,C TUNE UP
We need a PSR that fairly represents player skill across the player base so that the MM can rely on this information to balance teams. Otherwise, garbage in, garbage out.
Posted Image

Note : I discuss two other possible lines in the full document.
Answering the first question :

Why such a big X ?

Win and Loss is a hard +/-. You can’t 55% Win, you either 100% win or 100% loose. MS on the other hand changes smoothly and can take any value, mostly changing little by little. They behave very differently over multiple games.

Here's an example of how Win/Loss and MS act over many games. Note: there are many ways to get to the end values, but the end points do not change unless the resulting AvgMS and W-L changes.
Posted Image

Why are the X and Y important ?

Because the ratio between X and Y changes the steepness of the skill line. Too steep and you are MS heavy, like now, too shallow and it's all about WLR.

Next image shows how this fits together in my system. Movement along the PSR is represented by it's X and Y components in their own axis for clarity. Movement is always along the PSR line.
Posted Image

Can I test my end of match ?

Yes, but keep in mind that you can't look at a single match to see what PSR will do. To help with this, I added a portion where you can check the team composition if you were to play another match with the same players, based on the PSR shift alone. The tuned up X,Y,C provided will almost always split the winning and losing team.

Try it here with your own values on Jay Z’s spreadsheet (modified to show next match) :
https://docs.google.com/spreadsheets/d/1NewNKHTn4nPw9iMgoTIkIxoZYAxeM1YaI3m-hvX8LT8/edit?usp=sharing (Original X = 5, Y = 20, C=0.6)

https://docs.google.com/spreadsheets/d/15VQqOmtKDm9QinTQYZT_vBEu6upGV2qiWKCSVV99M1s/edit?usp=sharing (CFox Mod)

To know which PSR you would get, try the PSR calculator:
https://docs.google....dit?usp=sharing
Original post in the spoiler below - I got nothing to hide but the review made the analysis much better.
NOTE : Some of the findings are out of date.

Spoiler

Edited by Cluster Fox, 14 October 2020 - 06:46 AM.


#2 Snowhawk

    Member

  • PipPipPipPipPipPip
  • Survivor
  • Survivor
  • 433 posts

Posted 30 June 2020 - 06:57 AM

That's the second time I read that they will make today the psr update. But I can't find any informations About this in the pgi announcements. Did you find it on Twitter?

#3 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 30 June 2020 - 07:13 AM

View PostSnowhawk, on 30 June 2020 - 06:57 AM, said:

That's the second time I read that they will make today the psr update. But I can't find any informations About this in the pgi announcements. Did you find it on Twitter?


Nope. Didn't see or read anything. I'm not too worried about it to be honest.

This is one of the most important changes to potentially reducing stomps and making the matches so much better. PGI taking their time to do it right is the best way.

#4 John Bronco

    Member

  • PipPipPipPipPipPipPip
  • The Fighter
  • The Fighter
  • 966 posts

Posted 30 June 2020 - 07:19 AM

It's literally the top post in announcements.

#5 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 30 June 2020 - 07:22 AM

Very true, I read the question too fast. I haven't heard any updates about it today.

Edited by Cluster Fox, 30 June 2020 - 07:22 AM.


#6 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 01 July 2020 - 10:09 AM

Answering three questions that were asked somewhere else:




Q1. What happens to the PSR if a player changes skill?

A. With the stabilizing PSR formula, Constant P determines how fast the PSR responds to change and how "noisy" it'll get. The onset stays the same regardless of when a shift is happening.

Here is an example of a fictional player with the "Weak" archetype for the first 200 games, then with the "Strong" archetype. Unlikely to happen but it illustrates the point well.
It took roughly 60 games to move down 1000PSRpoints from the initial 2500 PSR, and roughly 120 games to move up 2000PSR points.

Posted Image

For this example:
NetPSRshift = 2500 + (1 - 0.02) * (PSRin - 2500) + (0.02 * 150 * PSRjayz) - PSRin

I think P = 0.02 is a very reasonable response time/noise compromise.




Q2. So how what does the standard deviation mean in terms of MS and WLR, why is it equivalent?

A. To put it simply, standard deviation represents a proportion of players. So, for instance:
Player A: 40th percentile in MS and the 60th percentile in WLR
Player B: 60th percentile MS and the 40th percentile WLR

Both players are of equivalent skill, they simply progressed on different aspects of the game.




Q3. So where are the Tiers going to stabilize using your formula?

A. Since my formula uses the player distribution and standard deviation, we can actually plot an estimate on the graph as long as X=11.53, Y=15 and C=1 in Jay Z's system.

Changing parameter A will change how wide the bands are and tighten or spread everything around the average (mid Tier3). The lines are not perpendicular because the axes are not proportional.

Once in steady state, this is approximately how Tiers will be divided, assuming each Tier is worth 1000 PSR from 0-5000. The boundaries will always be blended due to noise.

For A = 150, boundaries will be more or less:

Posted Image

I personally should fall around low Tier2 and stay there unless I improve.

Edited by Cluster Fox, 01 July 2020 - 11:06 AM.


#7 DevilCrayon

    Member

  • PipPipPipPipPipPip
  • The Nightmare
  • The Nightmare
  • 274 posts
  • LocationOakland, CA

Posted 04 July 2020 - 07:47 AM

This looks promising! I don't understand all the statistics involved, but it sounds very plausible. I appreciate the time you put into it!

#8 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 05 July 2020 - 07:22 AM

View PostDevilCrayon, on 04 July 2020 - 07:47 AM, said:

This looks promising! I don't understand all the statistics involved, but it sounds very plausible. I appreciate the time you put into it!


Thanks, I know this is very dry. I'll try to explain it better in a Google Docs soon.

I'm in the process of changing WLR (W/L) "the X axis" to :
(Wins - Losses) / (matches played) -> This gives a scale between -1.0(all losses) and 1.0(all wins) and it's closer to how the PSR is added up.

It matches better with avgMS than WLR. Also it might change X,Y,C a bit.

I also extracted individual some season data in order to compare seasons. The original analysis was taking all the history of the s46 and s47 players.

#9 Sjorpha

    Member

  • PipPipPipPipPipPipPipPipPip
  • Philanthropist
  • Philanthropist
  • 4,477 posts
  • LocationSweden

Posted 06 July 2020 - 05:02 AM

This model posits that doing things that does not contribute to the chances of winning should be regarded as some kind of "alternative" way of being good at the game. And even worse, it posits that this alternative way of being good should have equal weight in matchmaking.

How about if I said chess player A, that loses most of his games, is as good at chess as player B, who wins most of his games, because player A captures a lot more pieces in his games? Does that make any kind of sense? Nope obviously player B is better that player A, it's irrelevant how many pieces they capture.

You are good at a game if you're good at winning, you're bad at a game if you're bad at winning. MWO isn't some special case that is somehow exempt from game theory and competitive principles.

The purpose of the matchmaker is to predict your impact on the chances of winning a match, not to predict your matchscore or damage or anything else. Stop the nonsense please.

Edited by Sjorpha, 06 July 2020 - 05:05 AM.


#10 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 06 July 2020 - 06:38 AM

I see that this is based on correlation between MS and WLR in one season. This is hindsight. You need correlation to future matches to have any predictive ability.

#11 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 06 July 2020 - 07:03 AM

Sjorpha, I'm trying to improve on Jay Z's system since this is what got put in the game. I still support the WLR MM. I do see what you mean by considering the two as equal.

View PostNightbird, on 06 July 2020 - 06:38 AM, said:

I see that this is based on correlation between MS and WLR in one season. This is hindsight. You need correlation to future matches to have any predictive ability.


It's actually the full historical data of the players who participated in s46 and s47. I agree this is hindsight.

Edited by Cluster Fox, 06 July 2020 - 07:04 AM.


#12 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 06 July 2020 - 07:10 AM

I mean try using season 46 data to predict season 47 results. Don't look within one season.

#13 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 06 July 2020 - 02:17 PM

View PostNightbird, on 06 July 2020 - 07:10 AM, said:

I mean try using season 46 data to predict season 47 results. Don't look within one season.


I have only analyzed so far, I did not model to simulate. But thanks for clarifying.

I might try season 44 into 45 since 47 is weird because of mixed queue. 46 had about a month of mixed stuff.

#14 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 09 July 2020 - 03:05 PM

So, let's forget how much WL and AvgMS is important for a second.

I documented my PSR stability solution. This one works no matter what PSR is. Check it out, it can probably be improved, maybe with a quadratic gain or something.

Google Docs:
https://docs.google....dit?usp=sharing
Posted Image

Edited by Cluster Fox, 09 July 2020 - 03:06 PM.


#15 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 10 July 2020 - 09:14 PM

I reviewed my PSR skill line and documented it. W-L correlated much better than WLR.

Google Docs - Summary for both PSR mods with links to the -very- painful details:
https://docs.google....=h.rxz7ogwblorw

Posted Image

#16 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 July 2020 - 10:23 AM

Take 1000 players from Jarl's with WLR=1. IMO they are as average skilled as one can be. They have avgMS from 175 - 300. Will you give them the same PSR value? Or will you give them PSR values from 1500 to 3500 messing up the Match Maker?

Stat has the saying "garbage in garbage out". If you take bad data such as match score, and do anything with it, the results will be bad. This is true of Jay Z's model, and since you're using it, your model as well.

Since we can't fix MS (not even if we change the kickers, it's beyond saving), then the next best thing we can do is ignore it.

Lastly, the PSR scale you're assigning to pilots are arbitrary. If someone has 3000 pts, are they 50% better than a pilot with 2000 pts? If the points don't mean anything, then it makes no difference when you sum two teams up to have the same value, since a 4000 pt and 2000 pt pilots added together will not be equal to 2 3000pt pilots.

Edited by Nightbird, 11 July 2020 - 10:24 AM.


#17 Cluster Fox

    Member

  • PipPipPipPipPip
  • 104 posts
  • LocationStuck on a rock in Grim Plexus

Posted 11 July 2020 - 05:14 PM

Hey Nightbird, MS is not worth much right now, very true. But in my opinion, that doesn't mean it's completely worthless and to be ignored. Of course I might prove myself otherwise once I simulate this system.

To answer your question:
1.0 WLR and 100 AvgMS -> end value oscillates around 1800
1.0 WLR and 300 AvgMS -> end value oscillates around 2700
Not that bad of a spread on a range of 0-5000. There's always the option to make A smaller and tighten it.

The P-factor of 0.01 is high and induces noise right now. Seeing how a mess it is out there, PSR should get in the ballpark ASAP this season, noise is a compromise for now. A smaller P-factor doesn't change -where- values stabilize but it does on how fast they travel. So a P-factor of 0.003, once everything has kind of settled would make everything smoother and the PSR noise for a player would be reduced. Heck, maybe seeding P would be a good idea.

I disagree about the scale. The scale isn't arbitrary since it's based on standard deviation. So you know that a set percentage of player is above or below this PSR. A PSR of 3400 is 1 standard deviation both in W-L and AvgMS above the average. So about 16% of players are better. A PSR of 1600 is 1 standard deviation below average. 2500 is exactly average.
You can tell how much WLR and MS a player is likely to have if you know their PSR.

My proposition tries to make a distinction between those players who have the same WLR and are spread out in MS. I have not simulated it yet and maybe/probably the X,Y,C will change.

Oh and if I end up with Y=0 after simulating it, that basically made it a W-L system and you'll have full rights to "I told you so".

Cheers!

Edited by Cluster Fox, 11 July 2020 - 05:15 PM.


#18 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 11 July 2020 - 05:30 PM

Cool, please sim Jay Z's system first and match the current results, then add your correction so we can see the difference in performance.

#19 Anomalocaris

    Member

  • PipPipPipPipPipPipPip
  • Bad Company
  • 671 posts

Posted 12 July 2020 - 10:01 AM

Hey look everyone, a rational discussion where the two proponents disagree but are looking for information and a solution rather than scoring points on each other.

Kill it fast before it spreads.....

;)

#20 Nightbird

    Member

  • PipPipPipPipPipPipPipPipPipPip
  • The God of Death
  • The God of Death
  • 7,518 posts

Posted 12 July 2020 - 10:29 AM

View PostAnomalocaris, on 12 July 2020 - 10:01 AM, said:

Hey look everyone, a rational discussion where the two proponents disagree but are looking for information and a solution rather than scoring points on each other.

Kill it fast before it spreads.....

Posted Image


The trolls are part of Cluster Fox's group, they're not gonna troll in their own member's thread lol





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users