379 replies to this topic

[Paul Inouye]

Hey Folks,

So we sat down and analyzed the core proposals that were submitted, including those outside of the 4 cores presented. At the end of the day, we believe Core 2B + Modifications, known as Core 2C from the initial pitch from Jay Z, would get the community what it needs in terms of PSR distribution. With corrections and including average team based match score comparisons, it provides a more fair method of PSR movement.

That being said, here is some info and clarification of what's happening in "Core 2C".

As mentioned in Jay Z's original proposition, win/loss still has a factor in terms of overall movement but is mitigated by the way the amount of movement is calculated.

In short, here is the formula being used:

PSR Change = W(X) - Y + Y( C(P/A) + (1-C)(P/M) )

Where:
W - The win/loss variable used to determine direction of movement.
X - Win/Loss constant used in weighting average team PSR shift
Y - Matchscore constant used in weighting PSR shift compared to the value of X
P - Individual Pilot's Matchscore
A - Team's average Matchscore
C - Team's average Matchscore constant used in weighting calculated team effects
M - Average Matchscore across all 24 players in the match

Example:
This is the data from a game that happened recently. Using the above calculation, the following PSR changes will happen:



Running the top player on the winning team's numbers we will get:

PSR Change = +1(5) - 20 + 20(0.6 (537/300) + (1-0.6)(537/242))
= +24.3

Running the bottom player on the winning team's numbers we get:
PSR Change = +1(5) - 20 + 20(0.6 (77/300) + (1-0.6)(77/242))
= -9.4

Running the top player on the losing team's numbers we get:
PSR Change = -1(5) - 20 + 20(0.6(229/183) + (1-0.6)(229/242))
= -2.4

Running the bottom player on the losing team's numbers we get:
PSR Change = -1(5) - 20 + 20(0.6(84/183) + (1-0.6)(84/242))
= -16.7

While it may seem to not be equal since the change in opposite directions is not the same for the winning top player, and bottom losing player, there are more people with bigger negative values on the losing team to counter it.

Sounds confusing but it's actually pretty straight forward and maintains win/loss performance with zero sum movement amounts.

So here's what we're trying to fix. This image below is a data pull of active players in the last 3 months and the current Tier distribution (How many players are currently in each Tier):



As you can see, we have a large number of players in Tier 5 and a relatively flat line of players across Tier 4, 3, 2 and 1. While this 'looks' theoretically okay, the large spike in Tier 5 and even distribution across the other tiers is not ideal when trying to match make players against one another.

With this new system, we're hoping to see the following (orange version) distribution across the playerbase:




In an ideal world, this graph is a bell curve graph. With full zero sum implementation, we might get there.


So here's the rundown of upcoming changes:

PSR range is being increased from 0-3750, to 0-5000 to allow a little more room for distribution with the new system.
PSR WILL be reset to 2500 for all players.
There is no longer a PSR placement booster for new players/reset players that used to bump players in their first 10 matches.
The new formula for calculating PSR movement is as proposed by the community: W(X) - Y + Y( C(P/A) + (1-C)(P/M) )
The constants used for weighting will be set to X=5, Y=20 and C=0.6.


This change will require a full patch to the client and servers. I do not have a direct ETA but it will be in the VERY near future. I will update this post as soon as I get the estimate tomorrow (Jun 24). This new system is currently being tested by QA and is working as intended. It's not a big patch, just needs to be timed correctly.

UPDATE: ETA is Tuesday June 30th.

Edited by Ilyahweh, 22 March 2021 - 04:57 PM.
Added planned release date.

[PeteZonee]

Awesome changes! I can't wait for the PSR reset. Thanks for listening to us. Hopefully we get slightly better quality matches in the long run.

[Brauer]

Paul Inouye, on 23 June 2020 - 05:12 PM, said:

Running the top player on the losing team's numbers we get:
PSR Change = -1(5) - 20 + 20(0.6(229/183) + (1-0.6)(229/242)
= -2.4

At a glance this looks like a positive change to me. Thank you for the update and for pursuing a change to PSR.

FYI there is an error in your breakdown of the system. The top performing player on the losing team is actually Player 14 and they will see a PSR increase of 6.1.

Edited by Brauer, 23 June 2020 - 05:44 PM.

[Jon McFuzzy]

Encouraging news. Looking forward to it.

[Krasnopesky]

Very pleasing result. Glad to see a community suggestion being implemented.

[Nightbird]

Well, we know the update will fail because:

1. the formula fails to stabilize people at any PSR value other than 0 or 5000 (forget that bell curve, 0 chance of that happening)
2. groups are given a large unfair advantage in the formula, solos get ready
3. number of stomps will not change (and may get worse for two months)
4. 60% of the player base will be sent - fast or slow - to Tier 5. Once the good players hit the 5000 max, they can't earn points anymore so points awarded to them get lost to entropy, gradually reducing the total player points average from 2500 and forcing all the casuals towards T5.
5. T5 players are 2.5x more likely to quit MWO

Because Math is not an Opinion.

So rather than waiting for 2 months to see proof of whether 1+1=5, I give you the result now:

Spoiler

P.S. The tier/population graph needs to be players*games to present the relative number of people in each tier as relevant for the queue. See 5 above.

Edited by Nightbird, 25 June 2020 - 03:45 PM.

[Monkey Lover]

Paul what is the plan to balance teams? They always going to end up in tier 1 even if they for example have 4, tier 5 players?

Edited by Monkey Lover, 23 June 2020 - 06:06 PM.

[OneTeamPlayer]

If the preferred player distribution graph is a bell curve how does that:

1. Reduce the number of stomps
2. Decrease the amount of matches with a large tier spread.
3. Account for Group Queue in the Solo Queue imbalances.

Wouldn't T3 players be more likely to get solely T3 matches while T1, T2, T4, and T5 are more likely to be sorted into other tiers to complete a match in a timely manner?

Also, the numbers you are providing indicate that there are 47 thousand players available from which to create matches.

How is this possible if you were openly stating matchmaker didn't have the population to work well and when steam has trouble breaking 600 at peak times?

Are you saying that steam (the largest gaming platform on PC) has an install base of less than 1000 while the solo launcher has 46 thousand currently active installs?

Quote

Currently MWO peaks at about ~1500 players online in the evening and about half that at the low point of the 24 hour period meaning, it averages in the ~1000 online range taking into account weekends are higher than weekdays. The queue the MM draws from is in the range of 24-50 players at low points and maybe as high as 80-100 at peak play times on the weekends with the current release valve settings (aka wait times).

Given that quote was given to us a few weeks ago by Paul Inoyue, how does the current matchmaker implementation work within the current playerbase limitations, rather than in a massive hypothetical playerbase that does not log in simultaneously or much at all, apparently.

If 47K "active" players = 1.5K Peak and 1K average....well, what does the 1.5K look like when sorted into these tiers, and how will players actually be separated and sorted into balanced matches?

Edited by OneTeamPlayer, 23 June 2020 - 06:07 PM.

[55555]

This is amazing news. Thank you for the work and providing many examples of its use. This is going to keep this game around for a long time to come hopefully. <3

[Tamerlin]

Thanks, Paul. Looks like a good change. Will there be a periodic PSR reset, like every three months?

Did we miss the Excel sheet with Match Score testing you spoke about here (https://mwomercs.com...ost__p__6338086)? Can we change the weighting of in-game events that drive Match Score (like damage) without also changing the CBill and XP rewarded for them?

Edited by Tamerlin, 23 June 2020 - 06:40 PM.

[PeteZonee]

OneTeamPlayer, on 23 June 2020 - 06:06 PM, said:

If the preferred player distribution graph is a bell curve how does that:

1. Reduce the number of stomps
2. Decrease the amount of matches with a large tier spread.
3. Account for Group Queue in the Solo Queue imbalances.

Wouldn't T3 players be more likely to get solely T3 matches while T1, T2, T4, and T5 are more likely to be sorted into other tiers to complete a match in a timely manner?

Also, the numbers you are providing indicate that there are 47 thousand players available from which to create matches.

How is this possible if you were openly stating matchmaker didn't have the population to work well and when steam has trouble breaking 600 at peak times?

Are you saying that steam (the largest gaming platform on PC) has an install base of less than 1000 while the solo launcher has 46 thousand currently active installs?



Given that quote was given to us a few weeks ago by Paul Inoyue, how does the current matchmaker implementation work within the current playerbase limitations, rather than in a massive hypothetical playerbase that does not log in simultaneously or much at all, apparently.

If 47K "active" players = 1.5K Peak and 1K average....well, what does the 1.5K look like when sorted into these tiers, and how will players actually be separated and sorted into balanced matches?

The active player counts do seem quite a bit off but I think this PSR reset is just the first step towards more quality matches and a stomp rate of under 30% or whatever the figure was. Next step will most likely be more number/value changes to the "tier valves" if the bell curve becomes reality.

[ESC 907]

IMHO, just going to WLR would be a longer, but easier solution, since this system seems to depend upon MS - which many believe is still imbalanced. I just hope that this system works out, and does not require too much future tweaking of the MS calculations.

[Dogmeat1]

Is there any ETA on the matchscore sample data you mentioned previously?

[Kamikaze Viking]

Thankyou again Paul For working with us on this.

I'm really excited to see this working.

--------------------------
Quick reminder when using Jay Z spreadsheet

After entering matchscores into each team remember to sort each team by Z-> A so the matchscores are descending.

Edited by Kamikaze Viking, 23 June 2020 - 06:59 PM.

[Cluster Fox]

Thanks for the update Paul,

EDIT : There's still two priorities/issues with the accepted system.

• Stabilizing the PSR so it's end value becomes independent of # of matches played.
  
• Ensuring PSR fairly represents player Skill. AKA: X,Y,C. Not that useful until 1 is done.

Link to my analysis for both issues.

https://docs.google....dit?usp=sharing

---

ORIGINAL POST:

Questions:

• Can we expect any change to the MM regarding sorting teams by WLR? - If the result is reducing stomps even half as much as expected from simulations, it would be a tremendous improvement.
  
• Would you please release data on how often the kickers happen per player per match, with the match outcome (i.e. 12v4). Average is nice, but a good amount of bulk data would be best.
  
  I've heard publishing actual kicker values is a problem, but this data doesn't disclose any of that and would allow to model how the kickers are going to impact the PSR system.

As posted in the other topic:

Here's something I brewed over the weekend. Feel free to download a copy, tweak the values, try things out,
etc.



https://docs.google....dit?usp=sharing

This spreadsheet allows anyone to rate kickers by contribution to teamplay, then assign multipliers to each group. On page two, one can enter scoring screen data from actual matches and see the comparison.

DISCLAIMER: I do not claim anything with the kicker values included. I entered some values off the top of my head which ended up ballpark similar. Data or proof of concept is necessary to get the kickers to work.

Without data, all those kickers are meaningless, therefore the tuning should be done by those in the know. I strongly hope those tuning the kicker can get some data on the frequency of each kicker vs match results.

Edited by Cluster Fox, 12 July 2020 - 07:48 AM.

[Magic Pain Glove]

Yay .

[ShogunKid]

Thank you PGI for the upcoming reset. I look forward to being able to play non-meta mechs, that I've spent real money on, in an appropriate lower Tier where I won't be chastised for doing so. Farewell Tier 2 - I won't miss ya! (Although I'm sure I'll see some of y'all again when MM needs a few lower Tier folks to fill in the gaps) Time to go decorate and tidy up some of my old stock mechs with some fresh, new one-shots and colors to celebrate!

[Horseman IIC]

OneTeamPlayer, on 23 June 2020 - 06:06 PM, said:

Also, the numbers you are providing indicate that there are 47 thousand players available from which to create matches.

No, they do not. That 47K is the number of players who played at least once in past three months. However, cross-reference Jarl's List and you'll see only 26K people appear on the leaderboards in past three months. That means 21K - nearly half - are casuals who have played less than ten matches in any given month in these three months

Quote

How is this possible if you were openly stating matchmaker didn't have the population to work well and when steam has trouble breaking 600 at peak times?

Are you saying that steam (the largest gaming platform on PC) has an install base of less than 1000 while the solo launcher has 46 thousand currently active installs?

You misunderstand the implications of the numbers.
* Only part of the game's playerbase will be concurrently online at any given time.
* The game has a 1000 concurrent players on average, peak 1500
* The steam version counts 500 of those players, peak 750
* Ergo, Steam most likely still accounts for ~50% of the active population.
* Only a fraction of those 1000 to 1500 players will be actively queued at any given time

[kalashnikity]

Looks like a good hypothesis, can't wait.

[Sjorpha]

Well, kudos for the community interaction and best of luck.

I have to say the current distribution of players surprises me, I thought a much higher percentage of the player base was tier 1 because of the current upwards bias of the psr calculation. I'm also surprised to see so many players in tier 5, is that because there are a lot of new accounts?