713 replies to this topic

[Brauer]

LethalDeterrent, on 11 June 2020 - 10:07 AM, said:

I absolutely agree with this. One of the biggest frustrations are teams that play every mode like skirmish and completely ignore objectives. These should have had higher weights all along. Nothing worse than doing your job on objectives when no one else is, losing because of it, and still losing rank. Otherwise, might as well just keep Skirmish and Domination, and drop the other modes altogether. I'd agree that it wouldn't be much fun, but as it stands now, that's pretty much what we're playing every match anyway.

As to the rest, I'm no engineer so I'm cool with whatevs.

The issue with this is you are trying to re-engineer how people play the game without recognizing that all of those "playing the objective/game mode" factors are less important to winning that getting kills and damage.

By inflicting damage on the enemy team and by destroying their mechs you are simultaneously degrading their ability to play the objective AND degrading their ability to inflict damage and kills on your team. This is why kills and damage fundamentally are more important than sitting on a cap point, or shuttling batteries. Caps, batteries, and other "play the objective/game mode" factors can be important, BUT they are far more relevant in an organized setting like competitive play where a team is far more likely to have a concept of how they want to play the map and mode and can execute it in a coordinated way.

Not to mention sitting on cap points or running back and forth with batteries requires far less skill than fighting an opposing team, and as such does not, imo, merit being rewarded as much as more skillful play.

[Luscious Dan]

Win/Loss weighting

Obviously winning means a lot. Match score, K/D, damage dealt, etc. aren’t the whole picture. But group drops vs. solo drops obviously messes with the reliability of W/L as a metric of player skill in a pretty big way. Just this week, I witnessed a player on red team put up 1,900+ damage in a MCII-B and they still lost (it was 10-10 on kills, but the last enemies alive were assaults located outside the circle on Domination). I don't support a system that penalizes all losing players, or treats a 12-0 game the same as a 12-11 game.

So win/loss isn’t everything, and match score isn’t everything. Maybe the easier solution here is just to make the match score value for a win slightly bigger than whatever it currently is. It’s not perfect, but it’s extremely simple and gives just a little more weight to wins without over-complicating things or messing around with separate tier ratings for solo vs group drops, or anything like that.

Zero Sum thoughts

Honestly, I feel this is a trap. To many, it *sounds like* a zero sum solution would cure the “XP bar effect” but I don’t really think that’s true. And clearly there is a lack of consensus for what people really mean by zero sum in the first place. Most people seem to want something that looks symmetrical, but isn't really data-driven. A zero sum system could still push players towards the poles rather than sorting them into 5 meaningful tiers.

The real issue in my mind is that it takes a huge number of games to change your tier right now. This is the “XP bar grind” effect of working your way up through the tiers. The PSR/tier system can’t be ranked on ancient career statistics, it should factor in more recent data as much as possible.

I know it's yet another case where anecdotal evidence doesn't prove anything, but my alt account has career ratios of about 1.25 W/L (all solo drops) and 1.9 K/D, with career match score average 300+. Jarl’s List puts those numbers in the 93rd percentile, if you care for such things, and those numbers are going up based on recent weeks. At the start of this month, the alt account was still ranked very low in tier 2 due to under 500 games played. Is that 500 games played a lot? Compared to this account (which has ~4500 games played, but with lower W/L, K/D, and match score averages), no. But it shouldn’t take years worth of games to determine what tier an account belongs in.

Conclusions

I want to see a PSR/Tier system that shows you an accurate portrayal of a player’s current performance. Not an XP bar. Not something that requires you to play thousands of matches to get pushed up to tier 1, or down to tier 5. Not something that can't differentiate between a close match and a stomp.

I really think the best bet is to rank players 1-24 at the end of each game, and use a player’s average ranking for the current season (or X number of most recent seasons) to determine their tier. Positive bias, negative bias, zero sum etc. doesn’t matter if PSR is an average rather than a total.

I’d be ok with losing all visible traces of the tier system, and just having MM pairing up players based on these values of 1-24 (trying to keep the top 10% and bottom 10% away from each other, ideally).

[rascje]

Ajantise, on 11 June 2020 - 11:15 AM, said:

Your thinking is wrong. Dont fix the MM.
Fix the game.
Than you will have 100 000 players and you will have a perfect MM even if the logic of building a match is bad.

Sooooo ironic, dude!
...
...
Wait...
Are u really serious?!?!?..
For "fix" mwo, the game must be a complete new game... as a pvp mw5...

Edited by chel plui biel, 11 June 2020 - 11:30 AM.

[Nightbird]

Luscious Dan, on 11 June 2020 - 11:23 AM, said:

Win/Loss weighting

Obviously winning means a lot. Match score, K/D, damage dealt, etc. aren’t the whole picture. But group drops vs. solo drops obviously messes with the reliability of W/L as a metric of player skill in a pretty big way. Just this week, I witnessed a player on red team put up 1,900+ damage in a MCII-B and they still lost (it was 10-10 on kills, but the last enemies alive were assaults located outside the circle on Domination). I don't support a system that penalizes all losing players, or treats a 12-0 game the same as a 12-11 game.

Why not see that as a reward? You can have a great match, but your team lost it for you, so the MM will reward you with an easier match next round.

Another way to see it is: You have 24 players to put into two teams, for the first 8 players 4 are in a group and 4 are solo
Player 1: 400 avgMS 10 WLR (group)
Player 2: 400 avgMS 10 WLR (group)
Player 3: 400 avgMS 10 WLR (group)
Player 4: 400 avgMS 10 WLR (group)
Player 5: 400 avgMS 2 WLR (solo)
Player 6: 400 avgMS 2 WLR (solo)
Player 7: 400 avgMS 2 WLR (solo)
Player 8: 400 avgMS 2 WLR (solo)

MM puts them into two teams:

Team A
Player 1: 400 avgMS 10 WLR (group)
Player 2: 400 avgMS 10 WLR (group)
Player 3: 400 avgMS 10 WLR (group)
Player 4: 400 avgMS 10 WLR (group)

Team B
Player 5: 400 avgMS 2 WLR (solo)
Player 6: 400 avgMS 2 WLR (solo)
Player 7: 400 avgMS 2 WLR (solo)
Player 8: 400 avgMS 2 WLR (solo)

At this point, do you
1. want the MS based PSR match maker that says, Team A and Team B are equal, split the remaining players equally?
2. want the WL match maker that says, Team A is stronger than Team B, put more good players from the remaining players on Team B?

[OneTeamPlayer]

Cipher2012, on 11 June 2020 - 08:17 AM, said:

with a nerf to armament capacity of light and medium mechs, we might see a slowdown of nascars, which are 90% of the games played.

Your take on all this is yet another nerf to lights and mediums?

If lights and mediums are so overwhelming in their armament why is it that most days Heavies and Assaults make up 70% of the queue combined with lights and mediums splitting the rest of the 30%.

Why are there multiple hours where the light portion of the queue never breaks 10%?

Think people are just bored of lights being so amazingly overpowered that they wanted to "challenge" themselves with 100 tons of weapons and armor?

[Teknomancer]

Luscious Dan, on 11 June 2020 - 11:23 AM, said:

Win/Loss weighting

Obviously winning means a lot. Match score, K/D, damage dealt, etc. aren’t the whole picture. But group drops vs. solo drops obviously messes with the reliability of W/L as a metric of player skill in a pretty big way. Just this week, I witnessed a player on red team put up 1,900+ damage in a MCII-B and they still lost (it was 10-10 on kills, but the last enemies alive were assaults located outside the circle on Domination). I don't support a system that penalizes all losing players, or treats a 12-0 game the same as a 12-11 game.

So win/loss isn’t everything, and match score isn’t everything. Maybe the easier solution here is just to make the match score value for a win slightly bigger than whatever it currently is. It’s not perfect, but it’s extremely simple and gives just a little more weight to wins without over-complicating things or messing around with separate tier ratings for solo vs group drops, or anything like that.

You are correct. And it is very important to note - and this gets missed a lot - that match score is already weighted in favor of wins. I stress this: your match score is boosted if you win. It has been this way for a very long time. So for people who want to see win/loss as a key metric, match score already accounts for it.

Luscious Dan, on 11 June 2020 - 11:23 AM, said:

Zero Sum thoughts

Honestly, I feel this is a trap. To many, it *sounds like* a zero sum solution would cure the “XP bar effect” but I don’t really think that’s true. And clearly there is a lack of consensus for what people really mean by zero sum in the first place. Most people seem to want something that looks symmetrical, but isn't really data-driven. A zero sum system could still push players towards the poles rather than sorting them into 5 meaningful tiers.

The real issue in my mind is that it takes a huge number of games to change your tier right now. This is the “XP bar grind” effect of working your way up through the tiers. The PSR/tier system can’t be ranked on ancient career statistics, it should factor in more recent data as much as possible.

I know it's yet another case where anecdotal evidence doesn't prove anything, but my alt account has career ratios of about 1.25 W/L (all solo drops) and 1.9 K/D, with career match score average 300+. Jarl’s List puts those numbers in the 93rd percentile, if you care for such things, and those numbers are going up based on recent weeks. At the start of this month, the alt account was still ranked very low in tier 2 due to under 500 games played. Is that 500 games played a lot? Compared to this account (which has ~4500 games played, but with lower W/L, K/D, and match score averages), no. But it shouldn’t take years worth of games to determine what tier an account belongs in.

A good point too. Zero sum across 24 players in a match evenly distributes those 24 players by their match scores (which favor the winners), and that is good. It does not immediately balance players by their performance compared to the entire player pool (the ideal end goal of balancing). If those 24 players are not middle of road across the entire player base, then yes, players can be shuffled towards the poles.

However, it is an acceptable compromise I think. Ideally zero sum would be compared to the average (mean? median? mode?) match scores of the entire player pool... But that is a much trickier moving target. As the 24 players in a match are pulled from roughly comparable Tiers (as best it can from the available players online), then over time zero sum with the 24 in the match will average out well enough. You are gaining or losing PSR with respect to the Tiers you are in and immediately adjacent to. You are performing better, equal to, or below a comparable subset of the whole player base, and move accordingly. As your Tier shifts, that comparison shifts too, so you are always scored relative to, well, relative equals.

Luscious Dan, on 11 June 2020 - 11:23 AM, said:

Conclusions

I want to see a PSR/Tier system that shows you an accurate portrayal of a player’s current performance. Not an XP bar. Not something that requires you to play thousands of matches to get pushed up to tier 1, or down to tier 5. Not something that can't differentiate between a close match and a stomp.

I really think the best bet is to rank players 1-24 at the end of each game, and use a player’s average ranking for the current season (or X number of most recent seasons) to determine their tier. Positive bias, negative bias, zero sum etc. doesn’t matter if PSR is an average rather than a total.

I’d be ok with losing all visible traces of the tier system, and just having MM pairing up players based on these values of 1-24 (trying to keep the top 10% and bottom 10% away from each other, ideally).

Losing a visible Tier or PSR move indicator has pros and cons. Ultimately when it is used simply for matchmaking, then yes, seeing it is irrelevant. It's math under the hood, and mainly we care about a fun battle with matched talents regardless of a Tier number.

But there are a lot of competitive players in this game, and there is a significant psychological reward there for having some kind of ranking right there in the basic UI (not having to look it up on a website somewhere). It feels good to see your Tier change up; it gives incentive to play better if you see your Tier move down. That chevron after a match is something that has an effect when you see it, a lot of players look for it. May as well keep it, yes? It doesn't hurt if you don't care about it, and it adds a risk/reward emotional boost for those who do!

[Nightbird]

Teknomancer, on 11 June 2020 - 12:13 PM, said:

You are correct. And it is very important to note - and this gets missed a lot - that match score is already weighted in favor of wins. I stress this: your match score is boosted if you win. It has been this way for a very long time. So for people who want to see win/loss as a key metric, match score already accounts for it.

False, match score based PSR is your <average match score earned> x < # matches played>. Your PSR is as much affected by your performance as the # matches you play, it is meaningless.

W/L does not have this problem. Putting W/L into a MS formula is throwing gold into the ocean.

[Paul Meyers DEST]

Ajantise, on 11 June 2020 - 11:15 AM, said:

Your thinking is wrong. Dont fix the MM.
Fix the game.
Than you will have 100 000 players and you will have a perfect MM even if the logic of building a match is bad.

Fixing MM is the 1st step for 100k players.

[Cluster Fox]

Ok, here's a simple suggestion. Because I feel this is going nowhere in the current way things are being discussed.
=====
PSR = 50 games moving average WLR * 50 games moving average match score
Matches below 50 threshold count as 0 for both towards the moving average.
=====
That's it.

This basically factors in your average match score and balances it to how good you are in a team.
50 is quick enough to move when you get better or worse, and big enough to dampen new player/alt accounts noise.

The per-game little PSR arrow shows you up/down/equal based on how much the moving average just moved with that game. Good news new player, you'll get a motivating up arrow for your first 40-50ish matches because of the moving average spooling up.

Then you keep an ordered list of players, divided for approximately 20% of players each as... say...
0-180 Tier 1
180-220 Tier 2
220-250 Tier 3
250-350 Tier 4
350+ Tier 5

Disclaimer: Tier values non-scientifically eyeballed from Jarl's list global stats to illustrate the point. Ideally, the tier cutoff values would represent exactly 20% of active players in each tier. Possibly tiers are even irrelevant with this.

Edited by Cluster Fox, 11 June 2020 - 01:01 PM.

[Nightbird]

Cluster Fox, on 11 June 2020 - 12:43 PM, said:

Ok, here's a simple suggestion. Because I feel this is going nowhere in the current way things are being discussed.
=====
PSR = 50 games moving average WLR * 50 games moving average match score

No complaints from me for this. I don't know how well it works without a sim, but it should work.

[Jesslp]

Slothasaurus, on 08 June 2020 - 07:30 PM, said:

Sorry if these have been mentioned before but giving more weight to damage taken would be nice. There's something to be said for a mech leading a charge taking lots of shots so the rest of the team can get into position to fire. I have done it many times being in a light or medium find a bigger mech to buddy up with. He eats the damage. He should get rewarded for that.

Also more penalty than there currently is for team damage. There are times before the fight really gets into it I have taken more team damage than enemy fire. I have been spectating people and seen people unload on an enemy but a teammate might of been a bit off to the side and take half the volley. If people have even a bit of fire discipline or if they make the one-off mistake it won't affect them. It will affect the people that continue to not be careful.

I fully agree with both ideas.

Also, there are too many people that dont care not just doing superficial damage, but also blowing up teammates arms (and weapons with them), legs, torsos... which prevents you for doing more on the match.
Some even go further and do it ON PURPOSE, so they get more score, etc., once the match is decided for one side, by seriously crippling teammates.

This should be seriously punished.
(Once, its an accident. Twice, a nasty coincidence. Three or more, its a bad habit.)

Edited by Jesslp, 11 June 2020 - 01:12 PM.

[Horseman]

Cluster Fox, on 11 June 2020 - 12:43 PM, said:

Ok, here's a simple suggestion. Because I feel this is going nowhere in the current way things are being discussed.
=====
PSR = 50 games moving average WLR * 50 games moving average match score
Matches below 50 threshold count as 0 for both towards the moving average.

As I recall, Paul has addressed that one on twitter stating that anything that uses a rolling average is unfeasible.

[Nightbird]

Horseman, on 11 June 2020 - 01:52 PM, said:

As I recall, Paul has addressed that one on twitter stating that anything that uses a rolling average is unfeasible.

Yep, we know. My only purpose is this thread is to help people understand WHY what PGI has made feasible will have 0 impact on improving match making, and quite possibly make it worse.

The community has two options:

1. Choose one of the available options that is mathematically proven to do nothing, and have PGI say "we did what you asked, it didn't work, oh well"

2. Go back to PGI and say we don't want your update and we can't make any changes within the limitations of this thread to improve things. Allow the MM to be changed and here's mathematical proof of changes that will make matches better.

My personal expectation is for the community to ask for 1, but I have seen glimmers of hope.

Edited by Nightbird, 11 June 2020 - 02:05 PM.

[Willard Phule]

Jesslp, on 11 June 2020 - 12:49 PM, said:

I fully agree with both ideas.

Also, there are too many people that dont care not just doing superficial damage, but also blowing up teammates arms (and weapons with them), legs, torsos... which prevents you for doing more on the match.
Some even go further and do it ON PURPOSE, so they get more score, etc., once the match is decided for one side, by seriously crippling teammates.

This should be seriously punished.
(Once, its an accident. Twice, a nasty coincidence. Three or more, its a bad habit.)

Or it's about simply not caring about hitting friendlies once you've received friendly fire. I do my best to hold fire if a friendly is in front of me right about up until I get shot in the back by someone that didn't care. But you get that a lot in Tier 1.

Here's the thing I don't understand about this whole new PSR calculation thing. Why is it still based on a whole win/loss scenario as opposed to individual performance? I mean, sure, winning matters but why isn't it just a flat rate modifier slapped on at the end, after everything else is calculated? Being the guy that gets a 500+ MS on the team that got stomped 1/12 kinda says more about your ability than the 11 other guys that struggled to break 100.

[Capt Deadpool]

Nightbird W/L-based PSR plus modifiers that are applied if someone is in a group or solo or in terrible or meta mechs like Dauntless suggested seems like a good idea.

(Edited because rambling)

Edited by Capt Deadpool, 11 June 2020 - 10:21 PM.

[Nightbird]

Horseman, on 11 June 2020 - 01:52 PM, said:

As I recall, Paul has addressed that one on twitter stating that anything that uses a rolling average is unfeasible.

Another point on how easy it is to do rolling average:

Today we have a saved variable per player called PSR, and after a match we have a PSR_change. So the code for the new PSR is:
new_PSR = old_PSR + PSR_change.

To do a rolling average for MS over the last 50 games, we only need to save two variables, avgMS and N (N is matches played). After a match we have currentMS score.The code for the new avgMS is:

if N<50 then new_avgMS = old_avgMS * N/(N+1) + currentMS / (N+1)
if N>=50 then new_avgMS = old_avgMS * 49/50 + currentMS / 50
new_N = old_N+1

To do a rolling average for W/L over 50 games, likewise we need to save just two variables WLR and N. After a match we get a RESULT which is 1 for a win or 0 for a loss. Code:

if N<50 then new_WLR = {N * [old_WLR/(old_WLR + 1)] + RESULT} / {N * [1 - old_WLR/(old_WLR + 1)] - RESULT + 1}
if N>=50 then new_WLR = {49 * [old_WLR/(old_WLR + 1)] + RESULT} / {49 * [1 - old_WLR/(old_WLR + 1)] - RESULT + 1}


Takes a programmer 1 hr to do all of this. A new MM would take time but just a rolling MS or WL average took me 10 minutes to figure out.

Edited by Nightbird, 11 June 2020 - 05:06 PM.

[Nearly Dead]

I will ask here for a clarification.

Russ, Paul

If PSR were perfected, how many tiers would the MM be able to actually use to build balanced teams and still maintain the desired wait times?

[crazytimes]

Nearly Dead, on 11 June 2020 - 05:13 PM, said:

I will ask here for a clarification.

Russ, Paul

If PSR were perfected, how many tiers would the MM be able to actually use to build balanced teams and still maintain the desired wait times?

He said in the first post there are literally 24 people in the queue at times, meaning any tier will still end up in the same match.

This is why I see all the fancy formulas people are coming up with as an utter waste of time. It doesn't matter how you sort your players skill wise it they all end up in the same match anyhow. Some of the ridiculous nuances people are trying to apply just to write a different number on the same potatoes going into the same bucket.

They can't narrow the MM spread because that would cause a bunch of the few hundred people left playing to throw in the towel. It wouldn't take many to leave to hit sub critical mass and be unable to form matches at all outside peak.

[MovinTarget]

crazytimes, on 11 June 2020 - 05:34 PM, said:

He said in the first post there are literally 24 people in the queue at times, meaning any tier will still end up in the same match.

This is why I see all the fancy formulas people are coming up with as an utter waste of time. It doesn't matter how you sort your players skill wise it they all end up in the same match anyhow. Some of the ridiculous nuances people are trying to apply just to write a different number on the same potatoes going into the same bucket.

They can't narrow the MM spread because that would cause a bunch of the few hundred people left playing to throw in the towel. It wouldn't take many to leave to hit sub critical mass and be unable to form matches at all outside peak.

Because logic and maths will never trump what people feel and think they see.

[Nightbird]

crazytimes, on 11 June 2020 - 05:34 PM, said:

He said in the first post there are literally 24 people in the queue at times, meaning any tier will still end up in the same match.

This is why I see all the fancy formulas people are coming up with as an utter waste of time. It doesn't matter how you sort your players skill wise it they all end up in the same match anyhow. Some of the ridiculous nuances people are trying to apply just to write a different number on the same potatoes going into the same bucket.

They can't narrow the MM spread because that would cause a bunch of the few hundred people left playing to throw in the towel. It wouldn't take many to leave to hit sub critical mass and be unable to form matches at all outside peak.

1. He said the queue is 24-100 people depending on time of day
2. My proposal works just fine with 24 people across all skill levels. I did a simulation including a wide skill spectrum, where the most skilled players can 1vs9 the least skilled players and win. And this model produced 3 times the good matches the current MM makes with half the stomps.