I built the simulator using the same logic as NightBird. To try and emulate all four different systems, and to provide common grounds for comparison. Simulator was adjusted and checked for accuracy against real player data. The objective was to pit models against one another.

Conclusion/Findings (TL;DR)

- Stability is required.
- JayZ's values, are inadequate. Stability will not fix it.
- There is no free lunch (aka perfect solution)
- Both NB's system and CFox tweaks are good solutions.
- Cluster Fox with X,Y,C tuning, stability and seeding gives a good PSR representation of skill. This builds on the system already in place.
- Nightbird's system is another viable option but has some shortcomings as well. With no moving average, PSR would need to be reset every season otherwise PSR will not evolve with skill. A moving average is a compromise. Seeding is worth exploring.

What are Cluster Fox's PSR fixes?

It's what we got now, but tuned up and stabilized.

- Step 1. Change the X,Y,C values in the current system (jayZ).
- Step 2. Add a simple formula using the player's PSR to stabilize it.
- Step 3. Remove maximum PSR cap.
- Step 4. Start new players at PSR=500.
- No PSR reset.

Details in my original post: Tuning up JayZ and stabilizing it and detailed in google Docs

My propositions

This simulation proves that they have to be done together.

The systems simulated (for now)

**Cluster Fox**- My X,Y,C values in JayZ's system, with PSR stability**Cluster Fox + Seed**- Same as above, but seeding Afactors and Pfactors**NightBird's (no MA)**- His original WLR suggestion for the MM**NightBird's (100 and 300 MA)**- Same with a moving average - I'm using NB's own formula for the Moving average**JayZ**- The system currently implemented, with X,Y,C values initially suggested.**JayZ + Stability**- The X,Y,C initially suggested, with Cluster Fox's PSR stability

*NOTE: JayZ explicitly mentioned the X,Y,C should be tuned as more data becomes available. This is more data*

**In progress : X = 22, Y = 0, C = 1 with stability and seeding.**

Building the simulator

Results

Let's skip to the interesting stuff.

**Comparison 1 : W-L distribution**

If the W-L distribution is narrow, the MM created more balanced matches. Players tend to get a WLR closer to 1.0 and both teams have a more equal chance of winning a game.

After 200k games:

**NightBird's system (no MA)**, gives the best distribution of players in their W-L brackets.

**Cluster Fox + Seeding**is the second best system for W-L distribution.

NB (300 MA) is slightly worse than C Fox (Seed) - omitted for clarity.

Cluster Fox (no seed) and NightBird's system (100 MA) are comparable.

JayZ is by far the worst W-L distribution.

JayZ + Stability gives the about same results - omitted for clarity.

**What does it mean?**

In simple terms : The more you factor W-L in PSR, the closer the matches you get.

Also, once a moving average is included, PSR accuracy depends on noise and response time.

**Comparison 2 : Team %W-L chance distribution per games**

Ideally you want teams with 50%-50% win chance. An ideal MM would be able to create this every single game. This is not possible, so as close to 50-50 as possible is best.

**All 4 base systems:**

Again, NB (no MA) and C Fox (Seed) are best and this time, comparable. NB (no MA) is slightly better.

**Effect of NightBird's Moving Average:**

Including a moving average in NB's system has a negative effect. It makes everyone tend heavily towards 1.0 WLR at the start and as such, has problems from noise or slow response time of PSR. NB (100 MA) suffers from W-L noise. NB (300 MA) suffers mostly from slow response time (more games played would become more accurate).

**Effect of Stability and seeding:**

This one is interesting. Stabilizing JayZ has very little effect. Reason is not enough games were played at 200k to get the full divergence effect towards T1 and T5. This is visible later on in the PSR graphs.

Also visible is that seeding C Fox system drastically improves accuracy as this noise / response time trade off for 50 games pays off.

**What does it mean?**

Again, W-L is important, and seeding is also important with any kind of moving average to balance reponse time and noise.

**Comparison 3 : PSR to skill. AFTER 200k games**

Here, Skill is a measure of the natural trendline for player skill progression. To see how skill is estimated, please check out my other post. It's a mix of AvgMS and W-L.

A perfect PSR would display player distribution on a single PSR to Skill line and on a single PSR to W-L line. This is not possible, so getting as close as possible is best.

JayZ's system as initially suggested does not reflect skill, nor W-L.

AvgMS is too heavily weighted. Player movement is visibile as they move away from 2500 PSR at different rates.

**NIGHTBIRD'S OPTION**

Nighbird's option is calculated differently so the ideal W-L a different line, but a perfect match. However player skill is not as accurately represented.

The MM I simulated is imperfect and could not prevent a 33/1 WLR, this player's PSR got as high as 82000 !

Adding moving averages to NB's system is a compromise. Skill is better represented but W-L suffers from noise and response time.

**ADDING CLUSTER FOX STABILITY TO JAY Z's SYSTEM**

Adding CFox stability has a positive effect on player movement, effectively keeping some from T1 and T5. However effect on W-L is minimal because of the X,Y,C.

**ADDING CLUSTER FOX X,Y,C TUNING TO THE ABOVE**

Changing the X,Y,C has a tremendous impact on the PSR reflection of both skill and W-L. Now the biggest effect is noise and slow player movement.

**FINAL CLUSTER FOX PROPOSITION**

**ADDING CLUSTER FOX SEEDING**

Here, the effect of seeding is most visible on W-L. The initial fast movement followed by stability with the seeding has the best combined effect.

Errata : PSR vs Skill graphs are showing a X scale of 200 to 2000. The ideal line should intersect at (0,0) vice (200,0). This is not significant enough to change all the graphs.

Conclusion

- JayZ's system, used as is, is inadequate. Stability will not fix it.
- There is no free lunch (aka perfect solution)
- Both NB's system and CFox tweaks are good solutions.
- Cluster Fox with X,Y,C tuning, stability and seeding gives a good PSR representation of skill. This builds on the system already in place.
- Nightbird's system is another viable option but has some shortcomings as well. With no moving average, PSR would need to be reset every season otherwise PSR will not evolve with skill. A moving average is a compromise. Seeding is worth exploring.

So, What PSR will I get under the different Systems?

Check it out here : Cluster Fox's PSR prediction

References

https://mwomercs.com/forums/topic/277351-tuning-up-jay-zs-psr-system-and-stabilizing-it/

https://mwomercs.com...cy-with-graphs/

**Edited by Cluster Fox, 24 October 2020 - 01:10 PM.**