490 replies to this topic

[Paul Inouye]

Overview:

Well it seems that the overwhelming majority is fine with a Tier/PSR reset. And with that reset, we will make all future calculations zero sum in order to have PSR more accurately reflect player skill rather than experience playing the game.

Regarding Stomp Numbers:

But first, let's look at some numbers. We mentioned stomp stats that are matches that end with a score of 12-4 or worse for the losing side. While this metric isn't perfect, it does give us a good baseline for comparison purposes.

When we first implemented the queue merge, we mentioned that the current stomp rate increased by 5%. But as some of you mentioned, that 5% was not clear as to the number of actual stomps, just an increase in an unknown value.

How did we get the 5% increase? We took an average across a number of days prior to the merge and a number of days after the merge.

Prior to queue merge, our sample set gave us this:

1123 Stomps out of 3755 matches in an average 24hr period resulting in a 29.91% stomp rate.

Post queue merge, our sample set gave us this:
935 Stomps out of 2751 matches in an averages 24hr period resulting in a 33.99% stomp rate.

This was the 5% increase we saw.

When we first made some tuning changes to the match maker for Tier/PSR balance and tonnage balance, the sample set showed this:

1801 Stomps out of 5506 matches in an average 24hr period resulting in a 32.71% stomp rate. A 1.28% decrease in stomps.

When I did the stealth tweak to Tier/PSR and tonnage balance in the match maker, it resulted in this:

1057 Stomps out of 3453 matches in an average 24hr period resulting in a 30.61% stomp rate. A bigger reduction from peak of 33.99% down by 3.38%.

Essentially, overall stomp numbers increased by only 0.7% compared to pre-queue merge.

Now on to PSR and PSR Changes

Right now, PSR changes to Tiers are non-zero sum. If a player wins but doesn't perform well, they do not move. While this is okay, the problem was if the player lost and did well, they moved up. This leads to the "XP bar" people are commenting on. This was actually intended as we needed a way to pull experienced players away from new and lower performing players. While PSR does not reflect skill, it did do the job of separating experienced players from newer players when player count was much higher.

As time rolled on, and player count dropped, the match maker had a much tougher job of finding equally experienced opponents. Release valves in the strictness of the Match Maker meant over a period of time, more players from lower tiers would be included in matches. This had to be done in order to have matches kick off in a timely manner.

Because of the above situation, Tier 1 players could be matched with Tier 3 players. If queues sat for even longer, it would eventually open up to include Tier 4 players.

To change this, we will be implementing the following:

Current PSR values:

Player LOSES:
Match Score: 0-100 goes down in PSR by -2
Match Score: 101-250 goes down in PSR by -1
Match Score: 251-400 does not move.
Match Score: 401+ goes up in PSR by +1

Player WINS:
Match Score: 0-100 does not move.
Match Score: 101-250 goes up in PSR by +1
Match Score: 251-400 goes up in PSR by +3
Match Score: 401+ goes up in PSR by +5

New PSR values:
Player LOSES:
Match Score: 0-100 goes down in PSR by -5
Match Score: 101-250 goes down in PSR by -3
Match Score: 251-400 goes down in PSR by -1
Match Score: 401+ does not move.

Player WINS:
Match Score: 0-100 does not move.
Match Score: 101-250 goes up in PSR by +1
Match Score: 251-400 goes up in PSR by +3
Match Score: 401+ goes up in PSR by +5

With these new numbers in place, players performing well and winning will be able to climb out of mid tiers to higher tiers and players losing and not performing well will be dropping lower as they should.

The new numbers make team work critical if players want to climb in tiers. As you can see, the win is the biggest component of moving up or down in PSR. To win, you have to be a team player.

What does this mean for the Match Maker?

The Match Maker will still retain it's current settings and release valves. We cannot restrict only Tier 1 can play against Tier 1 with our current population. Doing so would result in match wait times close to an hour.

Changing the release valve numbers to be very strict during a quick test proved this, as wait times for Tier 1 AND Tier 5 climbed exponentially with each tier step removed from possible matches. Tier 1 will still have to play matches with Tier 3 no matter what PSR system is in place.

As the bell curve is reformed after the PSR reset (people shift into their respective tiers), matches should feel tighter and more competitive but it's not a magic bullet for equal skill gameplay. We are making these changes to build the best matches with the tools and numbers we have.

We will continue to monitor the stomp rate, game win/loss scenarios and tier matching to make sure the new PSR system is working as intended.

TL;DR:

- Queues merged
- PSR made zero sum
- PSR resetting
- PGI to monitor changes and act on data

Target Eta: June 9th, 2020

[denAirwalkerrr]

~30% stomps is still very high number tbh. Thanks for finally resetting PSR and making it closer zero sum I might actually come back to playing MWO from time to time to check new PSR system.

Edited by denAirwalkerrr, 04 June 2020 - 10:07 AM.

[Anomalocaris]

Shorter summary:

1. Match quality went down even in the best case scenario post merge. Goal not met.
2. You've missed the pitch on zero-sum. A player winning with a low match score should drop. They were a drag on their team and only won because of their teammates. Same thing with a losing player scoring 400+. They did more than anyone else to win the match but were penalized by their team.

Given that you still have groups dropping which will skew WLR, this attempt is not going to fix matchmaking problems because it does not address players climbing in PSR when they fail to produce in wins. You need to reward/penalize player on contribution regardless of win or loss. And unless you find away to account for groups, people are still going to end up ranked in ways they should not. You also need to consider dynamic PSR ranges for rewards/penalties. Keeping a fixed level does no reflect the reality of each game. Many people have spelled this out quite eloquently in other posts. Please consider them.

[My Lord and Saviour Jesus Christ]

Is there no way to have more than just match score impact PSR? Because LRM and AMS boats very easily farm matchscore.

[Korvus Knull]

Thank you for all the effort to make the wait times more managable. For all those experienced players out there lets be supportive of the genre and encourage team play and bring new people into the pool.

[Krasnopesky]

Excellent decision to reset the tiers.

However the changes to PSR in regards to zero sum completely miss the mark.

I have one question for you:

Why should a player who wins a match with a matchscore of 0 receive the same PSR change as a player who loses with a matchscore of 600?

[Remover of Obstacles]

So, a players gets 400 matchscore and happens to be on the losing side and goes down in PSR....

While the winning player gets 251 (with has the bonus points already added to matchscore for winning) and goes up?



You need to be able to go down with a 'bad effort' win and up with a 'great effort' loss.

[Firefox54]

I agree with the above post that part of the PSR reset was missed. There's too much weight to W-L in this format. At a minimum, some one that loses but gets a 400 match score or better should go up ... someone with a win but under 100 match score should go down.

Putting this much emphasis on team work might have been fine with a bigger player base ... but right now, it's luck to drop with 12 individuals that want to play as a team ... or luck to not drop with/against a 3/4-man team that just dominates

Bottom line (IMO) ... too much weight to W-L.

This is a start, and hopefully PGI will review this in a month or so ... or revise the PSR based on the comments being posted.

Edited by Firefox54, 04 June 2020 - 11:06 AM.

[Sebastyan Black]

Essentially, overall stomp numbers increased by only 0.7% compared to pre-queue merge.

GJ on increasing the stomp rate?

Edited by Sebastyan Black, 04 June 2020 - 11:17 AM.

[Pseudo98]

So the reward for carrying the team during a loss is basically nothing? gg

Have you maybe considered rewarding individual performance instead of penalising players for the fact that the vast majority of the player base is low skill?

How will this not encourage boosting by groups?

[Dogmeat1]

As I posted in the previous thread, the matchscore targets should scale based on the player's current PSR compared to the server PSR average/median. This will prevent players from infinitely increasing their PSR if they never lose (as happens with groups). An easy fix to this issue is to use these formulas;

Player LOSES:
IF Matchscore 401 * (PlayerPSR/MedianPSR) THEN no PSR change
ELSE Matchscore 251 * (PlayerPSR/MedianPSR) THEN PSR goes down by -1
ELSE Matchscore 101 * (PlayerPSR/MedianPSR) THEN PSR goes down by -3
ELSE PSR goes down by -5

Player WINS:
IF Matchscore 401*(PlayerPSR/MedianPSR) THEN PSR goes up by +5
ELSE Matchscore 251*(PlayerPSR/MedianPSR) THEN PSR goes up by +3
ELSE Matchscore 101*(PlayerPSR/MedianPSR) THEN PSR goes up by +1
ELSE PSR doesn't change

That should be quite easy to implement with the current code and while it won't solve all the issues, it will make the system more flexible and allow it to scale better.

In addition the issue with the buckets becoming unbalanced with not enough players in each division can be solved by using dynamic PSR targets for each bucket that are recalculated on a regular basis. For example you can simply divide the population in percentile groups (20/20/20/20/20 or even 25/25/25/15/10) and reassign players to the buckets every 24 hours, week, month or however often you feel is necessary to keep the buckets balanced. That will allow games to be matched far more quickly while keeping the tiers within each match relatively close compared to just using arbitrary PSR targets for each bucket.

<EDIT>

As many people have mentioned the system put forward by Paul/PGI rewards wins too much and punishes losses too heavily. Therefore if PGI was actually winning to modify their system a bit more then I'd recommend going with something like this;
Player LOSES:
IF Matchscore 400 * (PlayerPSR/MedianPSR) THEN PSR goes up by +3
ELSE Matchscore 325 *(PlayerPSR/MedianPSR) THEN no PSR change
ELSE Matchscore 250 * (PlayerPSR/MedianPSR) THEN no PSR change by -1
ELSE Matchscore 175 * (PlayerPSR/MedianPSR) THEN PSR goes down by -3
ELSE PSR goes down by -5

Player WINS:
IF Matchscore 400 * (PlayerPSR/MedianPSR) THEN PSR goes up by +5
ELSE Matchscore 325 * (PlayerPSR/MedianPSR) THEN PSR goes up by +3
ELSE Matchscore 250 * (PlayerPSR/MedianPSR) THEN no PSR change
ELSE Matchscore 175 * (PlayerPSR/MedianPSR) THEN PSR goes down by -1
ELSE PSR goes down by -3

That would still keep winning as a major factor in PSR progression but doesn't unfairly punish as many players.

Edited by Dogmeat1, 04 June 2020 - 03:49 PM.

[Flea Harvey Oswald]

Making it zero-sum is useful and the reset is warranted for any PSR change, but having no change if your team wins when you do nothing makes no sense for "personal" skill rating, instead it makes it almost entirely based on what random 11 people you get, not based off of your performance.

[suffocater]

Stupid. A player who got a matchscore of over 400 while carriing the team and still loosing deserves to go up!

Getting the numbers game wrong 100% of the time: PGI

[M E M E M A C H I N E]

The changes to PSR revealed the flaws of the match score system. The number one way to gain match score is doing damage. Therefore doing anything aside from tons of damage reflects even worse "skill" than in the old system. When will you fix the match score system?

[SirMad]

Overall a good idea.
In almost every online game the W-L ratio is the main factor for ranking up/down... because it is the only reliable metric.
if you loose more than 50% of your games you should drop in rank, and if you win more than 50% of your games you should go up!.

And i will never understand the thinking "going up in rank is a reward" and "going down is a punishment". The tier is just a value used by the matchmaker. it has no other value. so what should i care as a player?

[Kurlon]

Thank you for providing the base numbers the stats are derived from. I was one of those who asked for that, happy to see it happen.

I'm not quite sure on how this is a zero sum format, rather than more W/L biased than MS biased as the prior setup was? In my mind zero sum means that for X amount of upward movement after a match, there must also be X amount of downward movement. The proposed setup still allows for overall upward or downward movement in PSRs across the table in a match depending on results. Could you elaborate on what level you see the zero sum aspect working at?

[Paul Meyers DEST]

The PSR should represent the players skill and not how often hes on the winning site. So a bad player with avg matchscore of 100 get carried by his group/unit will reach tier 1. So the same bad MM as before.

[BogartTheStingy]

I am just getting out of Tier 3!...BUT my GF is better than me and STUCK in Tier 5!...soo..

[LastActionHobbit]

Brilliant news; I'm looking forward to playing more once PSR reset kicks in.

I agree with above comments as regards rewarding players that REALLY carried a losing team and and penalising those that REALLY hampered a winning team though; that seems a no brainer.

Once the reset has happened I'm pretty excited to get my game face on, challenge myself to see where I *actually* stand in terms of skill base in MWO, and hope to enjoy many more challenging yet fair feeling mathces going forward. It's taken a loooooong old time, but thanks for finally listening to what many have been asking for; this is a step in the right direction.

[nuttyrat]

Glad to hear that this is coming! Taking to Kurlon he brought up a good question .. how big is the PSR bar in each Tier?